Documentation
¶
Index ¶
Constants ¶
const DefaultSampleSize = 5
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Document ¶
type Document struct {
ID []byte // Unique identifier
Size int64 // Size of the document
UploadDate time.Time // When the document was uploaded
Filename string // Name of the file
Metadata Metadata // Contextual data
ContentType string // Type of data
Data []byte // Data
}
Document is the data structure that is either pulled from a remote host or that must be constructed to push to a remote host. Note that this structure contains only descriptive information of the document, not the contents.
type DocumentBuffer ¶
type DocumentBuffer struct {
// contains filtered or unexported fields
}
DocumentBuffer manages a dynamically-sized buffer of Documents.
func NewDocumentBuffer ¶
func NewDocumentBuffer() DocumentBuffer
NewDocumentBuffer creates a new DocumentBuffer.
func (*DocumentBuffer) Close ¶
func (db *DocumentBuffer) Close()
func (*DocumentBuffer) Next ¶
func (db *DocumentBuffer) Next() (*Document, error)
Next returns the next document and any associated error.
func (*DocumentBuffer) Send ¶
func (db *DocumentBuffer) Send(doc *Document, err error)
Send adds a document to the buffer and sends any error if encountered.
type Metadata ¶
type Metadata struct {
Tags []string `bson:"tags,omitempty"` // Tags associated with the document
}
type PullDescription ¶
type PullDescription struct {
Count int
}
type PullOption ¶
type PullOption func(*PullOptions)
func WithMaskName ¶
func WithMaskName() PullOption
func WithPullDescribe ¶
func WithPullDescribe() PullOption
WithPullDescribe will only describe the pull operation and not actually pull the documents.
func WithPullFilter ¶
func WithPullFilter(filter string) PullOption
func WithPullSampleSize ¶
func WithPullSampleSize(size int) PullOption
func WithPullSealOpener ¶
func WithPullSealOpener(so dcrypto.SealOpener) PullOption
func WithWorkers ¶
func WithWorkers(workers int) PullOption
type PullOptions ¶
type PullOptions struct {
SampleSize int // The number of documents to pull.
Filter string // Filter string
SealOpener dcrypto.SealOpener
DescribeOnly bool
Workers int
MaskName bool // Use a UUID as a mask name
}
PullOptions is a type for setting options for the pull operation.
type Puller ¶
type Puller interface {
// Pull will retrieve a slice of documents from a remote host.
Pull(ctx context.Context, b DocumentBuffer, opts ...PullOption) (*PullDescription, error)
}
Puller is an interface that defines the behavior of pulling a slice of documents from a remote host.
type PushOption ¶
type PushOption func(*PushOptions)
func WithPushFilter ¶
func WithPushFilter(filter string) PushOption
WithPushFilter will allow the user to set a filter for the push operation, specifically to avoid downloading chunk data for migration.
func WithPushSealOpener ¶
func WithPushSealOpener(so dcrypto.SealOpener) PushOption
WithPushSealOpener sets the sealer and opener for the object for encryption.
func WithPushTags ¶
func WithPushTags(tags ...string) PushOption
WithPushTags sets the tags for the object.
type PushOptions ¶
type PushOptions struct {
Tags []string // Metadata tags to associate with the object.
SealOpener dcrypto.SealOpener
Filter string // Filter string
}
PushOptions defines the options for pushing an object.
type Pusher ¶
type Pusher interface {
Push(ctx context.Context, name string, r io.ReadSeeker, opts ...PushOption) (string, error)
}
Pusher is an interface that defines the behavior of pushing.
type Reverter ¶
type Reverter interface {
// Revert will DELETE the files associated with the SHA in ALL cases.
// This is a WIP and will be updated to support more complex behavior.
//
// Deprecatd: DO NOT USE IN PRODUCTION, SEE DESCRIPTION.
Revert(ctx context.Context, sha string) error
}
Reverter is an interface that defines the behavior of reverting.
Source Files
Directories