combine

Documentation

Overview

Package combine implements a backend to combine multiple remotes in a directory tree

Index

• func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (outFs fs.Fs, err error)
• type Fs
  • func (f *Fs) About(ctx context.Context) (*fs.Usage, error)
  • func (f *Fs) ChangeNotify(ctx context.Context, notifyFunc func(string, fs.EntryType), ...)
  • func (f *Fs) CleanUp(ctx context.Context) error
  • func (f *Fs) Copy(ctx context.Context, src fs.Object, remote string) (fs.Object, error)
  • func (f *Fs) DirCacheFlush()
  • func (f *Fs) DirMove(ctx context.Context, src fs.Fs, srcRemote, dstRemote string) (err error)
  • func (f *Fs) DirSetModTime(ctx context.Context, dir string, modTime time.Time) error
  • func (f *Fs) Features() *fs.Features
  • func (f *Fs) Hashes() hash.Set
  • func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error)
  • func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (err error)
  • func (f *Fs) MergeDirs(ctx context.Context, dirs []fs.Directory) error
  • func (f *Fs) Mkdir(ctx context.Context, dir string) error
  • func (f *Fs) MkdirMetadata(ctx context.Context, dir string, metadata fs.Metadata) (fs.Directory, error)
  • func (f *Fs) Move(ctx context.Context, src fs.Object, remote string) (fs.Object, error)
  • func (f *Fs) Name() string
  • func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error)
  • func (f *Fs) OpenWriterAt(ctx context.Context, remote string, size int64) (fs.WriterAtCloser, error)
  • func (f *Fs) Precision() time.Duration
  • func (f *Fs) PublicLink(ctx context.Context, remote string, expire fs.Duration, unlink bool) (string, error)
  • func (f *Fs) Purge(ctx context.Context, dir string) error
  • func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)
  • func (f *Fs) PutStream(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)
  • func (f *Fs) PutUnchecked(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)
  • func (f *Fs) Rmdir(ctx context.Context, dir string) error
  • func (f *Fs) Root() string
  • func (f *Fs) Shutdown(ctx context.Context) error
  • func (f *Fs) String() string
• type Object
  • func (o *Object) Fs() fs.Info
  • func (o *Object) GetTier() string
  • func (o *Object) ID() string
  • func (o *Object) Metadata(ctx context.Context) (fs.Metadata, error)
  • func (o *Object) MimeType(ctx context.Context) (mimeType string)
  • func (o *Object) Remote() string
  • func (o *Object) SetMetadata(ctx context.Context, metadata fs.Metadata) error
  • func (o *Object) SetTier(tier string) error
  • func (o *Object) String() string
  • func (o *Object) UnWrap() fs.Object
• type Options

Functions

func NewFs

func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (outFs fs.Fs, err error)

NewFs constructs an Fs from the path.

The returned Fs is the actual Fs, referenced by remote in the config

Types

type Fs

type Fs struct {
	// contains filtered or unexported fields
}

Fs represents a combine of upstreams

func (*Fs) About

func (f *Fs) About(ctx context.Context) (*fs.Usage, error)

About gets quota information from the Fs

func (*Fs) ChangeNotify

func (f *Fs) ChangeNotify(ctx context.Context, notifyFunc func(string, fs.EntryType), ch <-chan time.Duration)

ChangeNotify calls the passed function with a path that has had changes. If the implementation uses polling, it should adhere to the given interval. At least one value will be written to the channel, specifying the initial value and updated values might follow. A 0 Duration should pause the polling. The ChangeNotify implementation must empty the channel regularly. When the channel gets closed, the implementation should stop polling and release resources.

func (*Fs) CleanUp

func (f *Fs) CleanUp(ctx context.Context) error

CleanUp the trash in the Fs

Implement this if you have a way of emptying the trash or otherwise cleaning up old versions of files.

func (*Fs) Copy

func (f *Fs) Copy(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Copy src to this remote using server-side copy operations.

This is stored with the remote path given.

It returns the destination Object and a possible error.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantCopy

func (*Fs) DirCacheFlush

func (f *Fs) DirCacheFlush()

DirCacheFlush resets the directory cache - used in testing as an optional interface

func (*Fs) DirMove

func (f *Fs) DirMove(ctx context.Context, src fs.Fs, srcRemote, dstRemote string) (err error)

DirMove moves src, srcRemote to this remote at dstRemote using server-side move operations.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantDirMove

If destination exists then return fs.ErrorDirExists

func (*Fs) DirSetModTime

func (f *Fs) DirSetModTime(ctx context.Context, dir string, modTime time.Time) error

DirSetModTime sets the directory modtime for dir

func (*Fs) Features

func (f *Fs) Features() *fs.Features

Features returns the optional features of this Fs

func (*Fs) Hashes

func (f *Fs) Hashes() hash.Set

Hashes returns hash.HashNone to indicate remote hashing is unavailable

func (*Fs) List

func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error)

List the objects and directories in dir into entries. The entries can be returned in any order but should be for a complete directory.

dir should be "" to list the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found.

func (*Fs) ListR

func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (err error)

ListR lists the objects and directories of the Fs starting from dir recursively into out.

dir should be "" to start from the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found.

It should call callback for each tranche of entries read. These need not be returned in any particular order. If callback returns an error then the listing will stop immediately.

Don't implement this unless you have a more efficient way of listing recursively that doing a directory traversal.

func (*Fs) MergeDirs

func (f *Fs) MergeDirs(ctx context.Context, dirs []fs.Directory) error

MergeDirs merges the contents of all the directories passed in into the first one and rmdirs the other directories.

func (*Fs) Mkdir

func (f *Fs) Mkdir(ctx context.Context, dir string) error

Mkdir makes the root directory of the Fs object

func (*Fs) MkdirMetadata

func (f *Fs) MkdirMetadata(ctx context.Context, dir string, metadata fs.Metadata) (fs.Directory, error)

MkdirMetadata makes the root directory of the Fs object

func (*Fs) Move

func (f *Fs) Move(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Move src to this remote using server-side move operations.

This is stored with the remote path given.

It returns the destination Object and a possible error.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantMove

func (*Fs) Name

func (f *Fs) Name() string

Name of the remote (as passed into NewFs)

func (*Fs) NewObject

func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error)

NewObject creates a new remote combine file object

func (*Fs) OpenWriterAt

func (f *Fs) OpenWriterAt(ctx context.Context, remote string, size int64) (fs.WriterAtCloser, error)

OpenWriterAt opens with a handle for random access writes

Pass in the remote desired and the size if known.

It truncates any existing object

func (*Fs) Precision

func (f *Fs) Precision() time.Duration

Precision is the greatest Precision of all upstreams

func (*Fs) PublicLink

func (f *Fs) PublicLink(ctx context.Context, remote string, expire fs.Duration, unlink bool) (string, error)

PublicLink generates a public link to the remote path (usually readable by anyone)

func (*Fs) Purge

func (f *Fs) Purge(ctx context.Context, dir string) error

Purge all files in the directory

Implement this if you have a way of deleting all the files quicker than just running Remove() on the result of List()

Return an error if it doesn't exist

func (*Fs) Put

func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

Put in to the remote path with the modTime given of the given size

May create the object even if it returns an error - if so will return the object and the error, otherwise will return nil and the error

func (*Fs) PutStream

func (f *Fs) PutStream(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

PutStream uploads to the remote path with the modTime given of indeterminate size

May create the object even if it returns an error - if so will return the object and the error, otherwise will return nil and the error

func (*Fs) PutUnchecked

func (f *Fs) PutUnchecked(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

PutUnchecked in to the remote path with the modTime given of the given size

May create the object even if it returns an error - if so will return the object and the error, otherwise will return nil and the error

May create duplicates or return errors if src already exists.

func (*Fs) Rmdir

func (f *Fs) Rmdir(ctx context.Context, dir string) error

Rmdir removes the root directory of the Fs object

func (*Fs) Root

func (f *Fs) Root() string

Root of the remote (as passed into NewFs)

func (*Fs) Shutdown

func (f *Fs) Shutdown(ctx context.Context) error

Shutdown the backend, closing any background tasks and any cached connections.

func (*Fs) String

func (f *Fs) String() string

String converts this Fs to a string

type Object

type Object struct {
	fs.Object
	// contains filtered or unexported fields
}

Object describes a wrapped Object

This is a wrapped Object which knows its path prefix

func (*Object) Fs

func (o *Object) Fs() fs.Info

Fs returns read only access to the Fs that this object is part of

func (*Object) GetTier

func (o *Object) GetTier() string

GetTier returns storage tier or class of the Object

func (*Object) ID

func (o *Object) ID() string

ID returns the ID of the Object if known, or "" if not

func (*Object) Metadata

func (o *Object) Metadata(ctx context.Context) (fs.Metadata, error)

Metadata returns metadata for an object

It should return nil if there is no Metadata

func (*Object) MimeType

func (o *Object) MimeType(ctx context.Context) (mimeType string)

MimeType returns the content type of the Object if known

func (*Object) Remote

func (o *Object) Remote() string

Remote returns the remote path

func (*Object) SetMetadata

func (o *Object) SetMetadata(ctx context.Context, metadata fs.Metadata) error

SetMetadata sets metadata for an Object

It should return fs.ErrorNotImplemented if it can't set metadata

func (*Object) SetTier

func (o *Object) SetTier(tier string) error

SetTier performs changing storage tier of the Object if multiple storage classes supported

func (*Object) String

func (o *Object) String() string

String returns the remote path

func (*Object) UnWrap

func (o *Object) UnWrap() fs.Object

UnWrap returns the Object that this Object is wrapping or nil if it isn't wrapping anything

type Options

type Options struct {
	Upstreams fs.SpaceSepList `config:"upstreams"`
}

Options defines the configuration for this backend