Why Go
- Case Studies
  
  Common problems companies solve with Go
- Use Cases
  
  Stories about how and why companies use Go
- Security Policy
  
  How Go can help keep you secure by default
Learn
Docs
- Effective Go
  
  Tips for writing clear, performant, and idiomatic Go code
- Go User Manual
  
  A complete introduction to building software with Go
- Standard library
  
  Reference documentation for Go's standard library
- Release Notes
  
  Learn what's new in each Go release
Packages
Community
- Recorded Talks
  
  Videos from prior events
- Meetups
  
  Meet other local Go developers
- Conferences
  
  Learn and network with Go developers from around the world
- Go blog
  
  The Go project's official blog.
- Go project
  
  Get help and stay informed from Go
- Get connected

pipeline

package

Go to main page

Versions in this module

v0

v0.6.20

Feb 26, 2019

v0.6.15

May 16, 2018

v0.6.14

Apr 20, 2018

v0.6.12

Mar 11, 2018

Changes in this version

type Part

+ func (p *Part) MD5Bytes() []byte

v0.5.25

Jan 30, 2018

v0.5.23

Jan 18, 2018

v0.5.22

Jan 17, 2018

v0.5.14

Dec 8, 2017

v0.5.13

Dec 2, 2017

v0.5.12

Nov 30, 2017

v0.5.10

Nov 13, 2017

Changes in this version

+ var MD5ToBlockID = make(map[string]int)

+ var MD5ToBlockIDLock sync.RWMutex

+ func NewBytesBufferChan(bufferSize uint64) chan []byte

+ type Part struct

+ BlockID string

+ BlockSize uint32

+ BufferQ chan []byte

+ BytesToRead uint32

+ Data []byte

+ DuplicateOfBlockOrdinal int

+ NumberOfBlocks int

+ Offset uint64

+ Ordinal int

+ SourceURI string

+ TargetAlias string

+ func ConstructPartsQueue(size uint64, blockSize uint64, sourceURI string, targetAlias string, ...) (parts []Part, numOfBlocks int)

+ func NewPart(offset uint64, bytesCount uint32, ordinal int, sourceURI string, ...) *Part

+ func (p *Part) GetBuffer()

+ func (p *Part) IsMD5Computed() bool

+ func (p *Part) LookupMD5DupeOrdinal() (ordinal int)

+ func (p *Part) MD5() string

+ func (p *Part) NewBuffer() *bytes.Buffer

+ func (p *Part) ReturnBuffer()

+ func (p *Part) ToString() string

+ type PartsPartition struct

+ NumOfParts int

+ Offset int64

+ PartitionSize int64

+ Parts []Part

+ TotalNumOfParts int64

+ TotalSize int64

+ func ConstructPartsPartition(numberOfPartitions int, size int64, blockSize int64, sourceURI string, ...) []PartsPartition

+ type SourceInfo struct

+ Size uint64

+ SourceName string

+ TargetAlias string

+ type SourcePipeline interface

+ ConstructBlockInfoQueue func(blockSize uint64) (partitionQ chan PartsPartition, partsQ chan Part, numOfBlocks int, Size uint64)

+ ExecuteReader func(partitionQ chan PartsPartition, partsQ chan Part, readPartsQ chan Part, id int, ...)

+ GetSourcesInfo func() []SourceInfo

+ type StorageAccountCredentials struct

+ AccountKey string

+ AccountName string

+ type TargetCommittedListInfo struct

+ List interface{}

+ type TargetPipeline interface

+ CommitList func(listInfo *TargetCommittedListInfo, numberOfBlocks int, targetName string) (msg string, err error)

+ PreProcessSourceInfo func(source *SourceInfo) (err error)

+ ProcessWrittenPart func(result *WorkerResult, listInfo *TargetCommittedListInfo) (requeue bool, err error)

+ WritePart func(part *Part) (duration time.Duration, startTime time.Time, numOfRetries int, err error)

+ type WorkerResult struct

+ BlockSize int

+ DuplicateOfBlockOrdinal int

+ ItemID string

+ NumberOfBlocks int

+ Offset uint64

+ Ordinal int

+ Result string

+ SourceURI string

+ Stats *WorkerResultStats

+ TargetName string

+ WorkerID int

+ type WorkerResultStats struct

+ CumWriteDuration time.Duration

+ Duration time.Duration

+ NumOfWrites int64

+ Retries int

+ StartTime time.Time