local

package
v2.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 8, 2020 License: MIT Imports: 47 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// Skip skips the path merge.
	Skip MergeStrategy = "Skip"
	// Merge attempts to merge the paths (directories only).
	Merge = "Merge"
	// Replace replaces the old path with the new path.
	Replace = "Replace"
)

Variables

View Source 
var (
	// ErrUpToDate indicates there are no locally staged changes.
	ErrUpToDate = errors.New("everything up-to-date")
	// ErrAborted indicates the caller aborted the operation via a confirm function.
	ErrAborted = errors.New("operation aborted by caller")
)
View Source 
var (
	// ErrNotABucket indicates the given path is not within a bucket.
	ErrNotABucket = errors.New("not a bucket (or any of the parent directories): .textile")
	// ErrBucketExists is used during initialization to indicate the path already contains a bucket.
	ErrBucketExists = errors.New("bucket is already initialized")
	// ErrThreadRequired indicates the operation requires a thread ID but none was given.
	ErrThreadRequired = errors.New("thread ID is required")
)
View Source 
var ArchiveStatusTimeout = time.Second * 5

ArchiveStatusTimeout is the timeout used when requesting a single status message.

Functions

func ChangeColor 

func ChangeColor(t dagutils.ChangeType) func(arg interface{}) aurora2.Value

ChangeColor returns an appropriate color for the given change type.

func ChangeType 

func ChangeType(t dagutils.ChangeType) string

ChangeType returns a string representation of a change type.

func DefaultConfConfig 

func DefaultConfConfig() cmd.ConfConfig

DefaultConfConfig returns the default ConfConfig.

func Diff 

func Diff(ctx context.Context, ds ipld.DAGService, a, b ipld.Node) ([]*du.Change, error)

Diff returns a set of changes that transform node 'a' into node 'b'. Modified from https://github.com/ipfs/go-merkledag/blob/master/dagutils/diff.go#L104 It only traverses links in the following cases: 1. two node's links number are greater than 0. 2. both of two nodes are ProtoNode. 3. neither of the nodes is a file node, which contains only unnamed raw blocks Otherwise, it compares the cid and emits a Mod change object.

func Ignore 

func Ignore(pth string) bool

Ignore returns true if the path contains an ignored file.

Types

type AddOption 

type AddOption func(*addOptions)

AddOption is used when staging a remote Unixfs dag cid in a local bucket.

func WithAddEvents 

func WithAddEvents(ch chan<- PathEvent) AddOption

WithAddEvents allows the caller to receive path events when staging files from a remote Unixfs dag.

func WithSelectMerge 

func WithSelectMerge(f SelectMergeFunc) AddOption

WithSelectMerge allows the caller to select the path merge strategy.

type ArchiveConfig added in v2.1.0 

type ArchiveConfig struct {
	// RepFactor (ignored in Filecoin mainnet) indicates the desired amount of active deals
	// with different miners to store the data. While making deals
	// the other attributes of FilConfig are considered for miner selection.
	RepFactor int
	// DealMinDuration indicates the duration to be used when making new deals.
	DealMinDuration int64
	// ExcludedMiners (ignored in Filecoin mainnet) is a set of miner addresses won't be ever be selected
	// when making new deals, even if they comply to other filters.
	ExcludedMiners []string
	// TrustedMiners (ignored in Filecoin mainnet) is a set of miner addresses which will be forcibly used
	// when making new deals. An empty/nil list disables this feature.
	TrustedMiners []string
	// CountryCodes (ignored in Filecoin mainnet) indicates that new deals should select miners on specific
	// countries.
	CountryCodes []string
	// Renew indicates deal-renewal configuration.
	Renew ArchiveRenew
	// Addr is the wallet address used to store the data in filecoin
	Addr string
	// MaxPrice is the maximum price that will be spent to store the data
	MaxPrice uint64
	// FastRetrieval indicates that created deals should enable the
	// fast retrieval feature.
	FastRetrieval bool
	// DealStartOffset indicates how many epochs in the future impose a
	// deadline to new deals being active on-chain. This value might influence
	// if miners accept deals, since they should seal fast enough to satisfy
	// this constraint.
	DealStartOffset int64
}

ArchiveConfig is the desired state of a Cid in the Filecoin network.

type ArchiveMessageType 

type ArchiveMessageType int

ArchiveMessageType is the type of status message. 

const (
	// ArchiveMessage accompanies an informational message.
	ArchiveMessage ArchiveMessageType = iota
	// ArchiveError accompanies an error state.
	ArchiveError
)

type ArchiveRemoteOption added in v2.1.0 

type ArchiveRemoteOption func(*archiveRemoteOptions)

func WithArchiveConfig added in v2.1.0 

func WithArchiveConfig(config ArchiveConfig) ArchiveRemoteOption

WithArchiveConfig allows you to provide a custom ArchiveConfig for a single call to ArchiveRemote.

type ArchiveRenew added in v2.1.0 

type ArchiveRenew struct {
	// Enabled indicates that deal-renewal is enabled for this Cid.
	Enabled bool
	// Threshold indicates how many epochs before expiring should trigger
	// deal renewal. e.g: 100 epoch before expiring.
	Threshold int
}

ArchiveRenew contains renew configuration for a ArchiveConfig.

type ArchiveStatusMessage 

type ArchiveStatusMessage struct {
	Type            ArchiveMessageType
	Message         string
	Error           error
	InactivityClose bool
}

ArchiveStatusMessage is used to wrap an archive status message.

type AuthFunc 

type AuthFunc func(context.Context) context.Context

AuthFunc is a function that's used to add additional context information to the outgoing API requests.

type Bucket 

type Bucket struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Bucket is a local-first object storage and synchronization model built on ThreadDB, IPFS, and Filecoin. A bucket represents a dynamic Unixfs directory with auto-updating IPNS resolution, website rendering, and Filecoin archivng.

Private buckets are fully encrypted using AES-CTR + AES-512 HMAC (see https://github.com/textileio/dcrypto for more). Both Unixfs node metadata (size, links, etc.) and node data (files) are obfuscated by encryption. The AES and HMAC keys used for bucket encryption are stored in the ThreadDB collection instance. This setup allows for bucket access to inherit from thread ACL rules.

Additionally, files can be further protected by password-based encryption before they are added to the bucket. See EncryptLocalPath and DecryptLocalPath for more.

func (*Bucket) AddRemoteCid 

func (b *Bucket) AddRemoteCid(ctx context.Context, c cid.Cid, dest string, opts ...AddOption) error

AddRemoteCid stages the Unixfs dag at cid in the local bucket, optionally allowing the caller to select a merge strategy.

func (*Bucket) ArchiveRemote 

func (b *Bucket) ArchiveRemote(ctx context.Context, opts ...ArchiveRemoteOption) error

ArchiveRemote requests an archive of the current remote bucket.

func (*Bucket) ArchiveWatch added in v2.2.0 

func (b *Bucket) ArchiveWatch(ctx context.Context) (<-chan ArchiveStatusMessage, error)

ArchiveWatch delivers messages about the archive status.

func (*Bucket) Archives added in v2.2.0 

func (b *Bucket) Archives(ctx context.Context) (*pb.ArchivesResponse, error)

Archives returns information about current and historical archives.

func (*Bucket) CatRemotePath 

func (b *Bucket) CatRemotePath(ctx context.Context, pth string, w io.Writer) error

CatRemotePath writes the content of the remote path to writer.

func (*Bucket) DBInfo 

func (b *Bucket) DBInfo(ctx context.Context) (info db.Info, cc db.CollectionConfig, err error)

DBInfo returns info about the bucket's ThreadDB. This info can be used to add replicas or additional peers to the bucket.

func (*Bucket) DecryptLocalPath 

func (b *Bucket) DecryptLocalPath(pth string, key []byte, w io.Writer) error

DecryptLocalPath decrypts the file at path with key, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC.

func (*Bucket) DecryptLocalPathWithPassword 

func (b *Bucket) DecryptLocalPathWithPassword(pth, password string, w io.Writer) error

DecryptLocalPathWithPassword decrypts the file at path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC. https://godoc.org/golang.org/x/crypto/scrypt is used to derive the keys from the password.

func (*Bucket) DecryptRemotePath 

func (b *Bucket) DecryptRemotePath(ctx context.Context, pth string, key []byte, w io.Writer) error

DecryptRemotePath decrypts the file at the remote path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC.

func (*Bucket) DecryptRemotePathWithPassword 

func (b *Bucket) DecryptRemotePathWithPassword(ctx context.Context, pth, password string, w io.Writer) error

DecryptRemotePathWithPassword decrypts the file at the remote path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC. https://godoc.org/golang.org/x/crypto/scrypt is used to derive the keys from the password.

func (*Bucket) DefaultArchiveConfig added in v2.1.0 

func (b *Bucket) DefaultArchiveConfig(ctx context.Context) (config ArchiveConfig, err error)

DefaultArchiveConfig gets the default archive config for the specified Bucket.

func (*Bucket) Destroy 

func (b *Bucket) Destroy(ctx context.Context) error

Destroy completely deletes the local and remote bucket.

func (*Bucket) DiffLocal 

func (b *Bucket) DiffLocal() ([]Change, error)

DiffLocal returns a list of locally staged bucket file changes.

func (*Bucket) EncryptLocalPath 

func (b *Bucket) EncryptLocalPath(pth string, key []byte, w io.Writer) error

EncryptLocalPath encrypts the file at path with key, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC.

func (*Bucket) EncryptLocalPathWithPassword 

func (b *Bucket) EncryptLocalPathWithPassword(pth, password string, w io.Writer) error

EncryptLocalPathWithPassword encrypts the file at path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC. https://godoc.org/golang.org/x/crypto/scrypt is used to derive the keys from the password.

func (*Bucket) Info 

func (b *Bucket) Info(ctx context.Context) (info Info, err error)

Info returns info about a bucket from the remote.

func (*Bucket) Key 

func (b *Bucket) Key() string

Key returns the bucket's unique key identifier, which is also an IPNS public key.

func (*Bucket) ListRemotePath 

func (b *Bucket) ListRemotePath(ctx context.Context, pth string) (items []BucketItem, err error)

ListRemotePath returns a list of all bucket items under path.

func (*Bucket) LocalSize 

func (b *Bucket) LocalSize() (int64, error)

LocalSize returns the cumalative size of the bucket's local files.

func (*Bucket) Path 

func (b *Bucket) Path() (string, error)

Path returns the bucket's top-level local filesystem path.

func (*Bucket) PullPathAccessRoles 

func (b *Bucket) PullPathAccessRoles(ctx context.Context, pth string) (roles map[string]buckets.Role, err error)

PullPathAccessRoles returns access roles for a path.

func (*Bucket) PullRemote 

func (b *Bucket) PullRemote(ctx context.Context, opts ...PathOption) (roots Roots, err error)

PullRemote pulls remote files. By default, only missing files are pulled. See PathOption for more info.

func (*Bucket) PushLocal 

func (b *Bucket) PushLocal(ctx context.Context, opts ...PathOption) (roots Roots, err error)

PushRemote pushes local files. By default, only staged changes are pushed. See PathOption for more info.

func (*Bucket) PushPathAccessRoles 

func (b *Bucket) PushPathAccessRoles(ctx context.Context, pth string, roles map[string]buckets.Role) (merged map[string]buckets.Role, err error)

PushPathAccessRoles updates path access roles by merging the pushed roles with existing roles and returns the merged roles. roles is a map of string marshaled public keys to path roles. A non-nil error is returned if the map keys are not unmarshalable to public keys. To delete a role for a public key, set its value to buckets.None. 

func (b *Bucket) RemoteLinks(ctx context.Context, pth string) (links Links, err error)

RemoteLinks returns the remote links for the bucket.

func (*Bucket) Roots 

func (b *Bucket) Roots(ctx context.Context) (roots Roots, err error)

Roots returns the bucket's current local and remote root cids.

func (*Bucket) SendPathInvite 

func (b *Bucket) SendPathInvite(ctx context.Context, from thread.Identity, to thread.PubKey, pth string) error

SendPathInvite sends a message containing a bucket key and path, which can be used to access a shared file / folder.

func (*Bucket) SetDefaultArchiveConfig added in v2.1.0 

func (b *Bucket) SetDefaultArchiveConfig(ctx context.Context, config ArchiveConfig) (err error)

SetDefaultArchiveConfig sets the default archive config for the specified Bucket.

func (*Bucket) Thread 

func (b *Bucket) Thread() (id thread.ID, err error)

Thread returns the bucket's thread ID.

func (*Bucket) Watch 

func (b *Bucket) Watch(ctx context.Context, opts ...WatchOption) (<-chan cmd.WatchState, error)

Watch watches for and auto-pushes local bucket changes at an interval, and listens for and auto-pulls remote changes as they arrive. Use the WithOffline option to keep watching during network interruptions. Returns a channel of watch connectivity states. Cancel context to stop watching.

type BucketItem 

type BucketItem struct {
	Cid        cid.Cid      `json:"cid"`
	Name       string       `json:"name"`
	Path       string       `json:"path"`
	Size       int64        `json:"size"`
	IsDir      bool         `json:"is_dir"`
	Items      []BucketItem `json:"items"`
	ItemsCount int          `json:"items_count"`
}

BucketItem describes an item (file/directory) in a bucket.

type Buckets 

type Buckets struct {
	// contains filtered or unexported fields
}

Buckets is used to create new individual buckets based on the provided clients and config.

func NewBuckets 

func NewBuckets(clients *cmd.Clients, config cmd.ConfConfig) *Buckets

NewBuckets creates Buckets from clients and config.

func NewBucketsWithAuth 

func NewBucketsWithAuth(clients *cmd.Clients, config cmd.ConfConfig, auth AuthFunc) *Buckets

NewBucketsWithAuth creates Buckets from clients and config and auth.

func (*Buckets) Clients 

func (b *Buckets) Clients() *cmd.Clients

Clients returns the underlying clients object.

func (*Buckets) Context 

func (b *Buckets) Context(ctx context.Context) context.Context

Context gets a context wrapped with auth if it exists.

func (*Buckets) GetLocalBucket 

func (b *Buckets) GetLocalBucket(ctx context.Context, pth string) (*Bucket, error)

GetLocalBucket loads and returns the bucket at path if it exists.

func (*Buckets) NewBucket 

func (b *Buckets) NewBucket(ctx context.Context, conf Config, opts ...NewOption) (buck *Bucket, err error)

NewBucket initializes a new bucket from the config. A local blockstore is created that's used to sync local changes with the remote. By default, this will be an unencrypted, unnamed, empty bucket. The remote bucket will also be created if it doesn't already exist. See NewOption for more info.

func (*Buckets) NewConfigFromCmd 

func (b *Buckets) NewConfigFromCmd(c *cobra.Command, pth string) (conf Config, err error)

NewConfigFromCmd returns a config by inflating values from the given cobra command and path. First, flags for "key" and "thread" are used if they exist. If still unset, the env vars {EnvPrefix}_KEY and {EnvPrefix}_THREAD are used.

func (*Buckets) RemoteBuckets 

func (b *Buckets) RemoteBuckets(ctx context.Context, id thread.ID) (list []Info, err error)

RemoteBuckets lists all existing remote buckets in the thread. If id is not defined, this will return buckets from all threads. In a hub context, this will only list buckets that the context has access to.

type Change 

type Change struct {
	Type dagutils.ChangeType
	Name string // Absolute file name
	Path string // File name relative to the bucket root
	Rel  string // File name relative to the bucket current working directory
}

Change describes a local bucket change.

type Config 

type Config struct {
	// Path is the path in which the new bucket should be created (required).
	Path string
	// Key is a key of an existing bucket (optional).
	// It's value may be inflated from a --key flag or {EnvPrefix}_KEY env variable.
	Key string
	// Thread is the thread ID of the target thread (required).
	// It's value may be inflated from a --thread flag or {EnvPrefix}_THREAD env variable.
	Thread thread.ID
}

Config contains details for a new local bucket.

type ConfirmDiffFunc 

type ConfirmDiffFunc func([]Change) bool

ConfirmDiffFunc is a caller-provided function which presents a list of bucket changes.

type Info 

type Info struct {
	Key       string        `json:"key"`
	Owner     string        `json:"owner"`
	Name      string        `json:"name"`
	Version   int           `json:"version"`
	Path      path.Resolved `json:"path"`
	Metadata  Metadata      `json:"metadata"`
	Thread    thread.ID     `json:"id"`
	CreatedAt time.Time     `json:"created_at"`
	UpdatedAt time.Time     `json:"updated_at"`
}

Info wraps info about a bucket. 

type Links struct {
	// URL is the thread URL, which maps to a ThreadDB collection instance.
	URL string `json:"url"`
	// WWW is the URL at which the bucket will be rendered as a website (requires remote DNS configuration).
	WWW string `json:"www"`
	// IPNS is the bucket IPNS address.
	IPNS string `json:"ipns"`
}

Links wraps remote link info for a bucket.

type MergeStrategy 

type MergeStrategy string

MergeStrategy describes the type of path merge strategy.

type Metadata 

type Metadata struct {
	Roles     map[string]buckets.Role `json:"roles"`
	UpdatedAt time.Time               `json:"updated_at"`
}

Metadata wraps metadata about a bucket item.

type NewOption 

type NewOption func(*newOptions)

NewOption is used when creating a new bucket.

func WithCid 

func WithCid(c cid.Cid) NewOption

WithCid indicates that an inited bucket should be boostraped with a particular UnixFS DAG.

func WithExistingPathEvents 

func WithExistingPathEvents(ch chan<- PathEvent) NewOption

WithExistingPathEvents allows the caller to receive path events when pulling files from an existing bucket on initialization.

func WithName 

func WithName(name string) NewOption

WithName sets a name for the bucket.

func WithPrivate 

func WithPrivate(private bool) NewOption

WithPrivate specifies that an encryption key will be used for the bucket.

type PathEvent 

type PathEvent struct {
	// Path relative to the bucket's cwd.
	Path string
	// Path cid if known.
	Cid cid.Cid
	// Type of event.
	Type PathEventType
	// Path total size.
	Size int64
	// Progress of the event if known (useful for upload/download progress).
	Progress int64
}

PathEvent describes a path event that occurred. These events can be used to display live info during path uploads/downloads.

type PathEventType 

type PathEventType int

PathEventType is the type of path event. 

const (
	// PathStart indicates a path has begun uploading/downloading.
	PathStart PathEventType = iota
	// PathComplete indicates a path has completed uploading/downloading.
	PathComplete
	// FileStart indicates a file under path has begun uploading/downloading.
	FileStart
	// FileProgress indicates a file has made some progress uploading/downloading.
	FileProgress
	// File complete indicates a file has completed uploading/downloading.
	FileComplete
	// FileRemoved indicates a file has been removed.
	FileRemoved
)

type PathInvite 

type PathInvite struct {
	Thread string `json:"thread"`
	Key    string `json:"key"`
	Path   string `json:"path"`
}

PathInvite wraps information needed to collaborate on a bucket path.

type PathOption 

type PathOption func(*pathOptions)

PathOption is used when pushing or pulling bucket paths.

func WithConfirm 

func WithConfirm(f ConfirmDiffFunc) PathOption

WithConfirm allows the caller to confirm a list of bucket changes.

func WithForce 

func WithForce(b bool) PathOption

WithForce indicates that all remote files should be pulled even if they already exist.

func WithHard 

func WithHard(b bool) PathOption

WithHard indicates that locally staged changes should be pruned.

func WithPathEvents 

func WithPathEvents(ch chan<- PathEvent) PathOption

WithPathEvents allows the caller to receive path events when pushing or pulling files.

type Repo 

type Repo struct {
	// contains filtered or unexported fields
}

Repo tracks a local bucket tree structure.

func NewRepo 

func NewRepo(pth, name string, layout options.Layout) (*Repo, error)

NewRepo creates a new bucket with the given path.

func (*Repo) CidVersion 

func (b *Repo) CidVersion() int

CidVersion gets the repo cid version (0 or 1).

func (*Repo) Close 

func (b *Repo) Close() error

Close closes the store and blocks service.

func (*Repo) Diff 

func (b *Repo) Diff(ctx context.Context, pth string) (diff []*du.Change, err error)

Diff returns a list of changes that are present in path compared to the bucket.

func (*Repo) GetNode 

func (b *Repo) GetNode(ctx context.Context, c cid.Cid) (ipld.Node, error)

GetNode returns the node at cid from the bucket.

func (*Repo) GetPathMap 

func (b *Repo) GetPathMap(pth string) (local, remote cid.Cid, err error)

GetPathMap returns details about a local path.

func (*Repo) HashFile 

func (b *Repo) HashFile(pth string) (cid.Cid, error)

HashFile returns the cid of the file at path. This method does not alter the bucket.

func (*Repo) MatchPath 

func (b *Repo) MatchPath(pth string, local, remote cid.Cid) (bool, error)

MatchPath returns whether or not the path exists and has matching local and remote cids.

func (*Repo) Path 

func (b *Repo) Path() string

Path returns the repo path.

func (*Repo) RemovePath 

func (b *Repo) RemovePath(ctx context.Context, pth string) error

RemovePath removes a local path map from the store.

func (*Repo) Root 

func (b *Repo) Root() (local, remote cid.Cid, err error)

Root returns the local and remote root cids.

func (*Repo) Save 

func (b *Repo) Save(ctx context.Context) error

Save saves the bucket as a node describing the file tree at the current path.

func (*Repo) SaveFile 

func (b *Repo) SaveFile(ctx context.Context, pth string, name string) error

SaveFile saves the bucket as a node describing a directory containing reader.

func (*Repo) SetCidVersion 

func (b *Repo) SetCidVersion(v int)

SetCidVersion set the repo cid version (0 or 1). The default version is 1.

func (*Repo) SetRemotePath 

func (b *Repo) SetRemotePath(pth string, remote cid.Cid) error

SetRemotePath sets or creates a mapping from a local path to a remote cid.

type Roots 

type Roots struct {
	Local  cid.Cid `json:"local"`
	Remote cid.Cid `json:"remote"`
}

Roots wraps local and remote root cids. If the bucket is not private (encrypted), these will be the same.

type SelectMergeFunc 

type SelectMergeFunc func(description string, isDir bool) (MergeStrategy, error)

SelectMergeFunc is a caller-provided function which is used to select a merge strategy.

type WatchOption 

type WatchOption func(*watchOptions)

WatchOption is used when watching a bucket for changes.

func WithOffline 

func WithOffline(offline bool) WatchOption

WithOffline will keep watching for bucket changes while the local network is offline.

func WithWatchEvents 

func WithWatchEvents(ch chan<- PathEvent) WatchOption

WithWatchEvents allows the caller to receive path events when watching a bucket for changes.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.