repo

package
v0.7.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 11, 2020 License: Apache-2.0 Imports: 39 Imported by: 13

Documentation

Overview

Package repo implements content-addressable Repository on top of BLOB storage.

Index

Constants

View Source 
const CacheDirMarkerFile = "CACHEDIR.TAG"

CacheDirMarkerFile is the name of the marker file indicating a directory contains Kopia caches. See https://bford.info/cachedir/

View Source 
const CacheDirMarkerHeader = "Signature: 8a477f597d28d172789f06886806bc55"

CacheDirMarkerHeader is the header signature for cache dir marker files.

View Source 
const FormatBlobID = "kopia.repository"

FormatBlobID is the identifier of a BLOB that describes repository format.

Variables

View Source 
var (
	BuildInfo    = "unknown"
	BuildVersion = "v0-unofficial"
)

BuildInfo is the build information of Kopia.

View Source 
var ErrAlreadyInitialized = errors.Errorf("repository already initialized")

ErrAlreadyInitialized indicates that repository has already been initialized.

View Source 
var ErrInvalidPassword = errors.Errorf("invalid repository password")

ErrInvalidPassword is returned when repository password is invalid.

View Source 
var ErrRepositoryNotInitialized = errors.Errorf("repository not initialized in the provided storage")

ErrRepositoryNotInitialized is returned when attempting to connect to repository that has not been initialized.

View Source 
var KeyRingEnabled = false

KeyRingEnabled enables password persistence uses OS-specific keyring.

Functions

func Connect 

func Connect(ctx context.Context, configFile string, st blob.Storage, password string, opt *ConnectOptions) error

Connect connects to the repository in the specified storage and persists the configuration and credentials in the file provided.

func ConnectAPIServer added in v0.6.0 

func ConnectAPIServer(ctx context.Context, configFile string, si *APIServerInfo, password string, opt *ConnectOptions) error

ConnectAPIServer sets up repository connection to a particular API server.

func DecodeToken 

func DecodeToken(token string) (blob.ConnectionInfo, string, error)

DecodeToken decodes the provided token and returns connection info and password if persisted.

func Disconnect 

func Disconnect(ctx context.Context, configFile string) error

Disconnect removes the specified configuration file and any local cache directories.

func GetDefaultHostName added in v0.7.0 

func GetDefaultHostName(ctx context.Context) string

GetDefaultHostName returns default hostname.

func GetDefaultUserName added in v0.7.0 

func GetDefaultUserName(ctx context.Context) string

GetDefaultUserName returns default username.

func GetPersistedPassword added in v0.5.2 

func GetPersistedPassword(ctx context.Context, configFile string) (string, bool)

GetPersistedPassword retrieves persisted password for a given repository config.

func Initialize 

func Initialize(ctx context.Context, st blob.Storage, opt *NewRepositoryOptions, password string) error

Initialize creates initial repository data structures in the specified storage with given credentials.

func RecoverFormatBlob 

func RecoverFormatBlob(ctx context.Context, st blob.Storage, blobID blob.ID, optionalLength int64) ([]byte, error)

RecoverFormatBlob attempts to recover format blob replica from the specified file. The format blob can be either the prefix or a suffix of the given file. optionally the length can be provided (if known) to speed up recovery.

func SetClientOptions added in v0.7.0 

func SetClientOptions(ctx context.Context, configFile string, cliOpt ClientOptions) error

SetClientOptions updates client options stored in the provided configuration file.

Types

type APIServerInfo added in v0.6.0 

type APIServerInfo struct {
	BaseURL                             string `json:"url"`
	TrustedServerCertificateFingerprint string `json:"serverCertFingerprint"`
}

APIServerInfo is remote repository configuration stored in local configuration.

type ClientOptions added in v0.7.0 

type ClientOptions struct {
	Hostname string `json:"hostname"`
	Username string `json:"username"`

	ReadOnly bool `json:"readonly,omitempty"`

	// Description is human-readable description of the repository to use in the UI.
	Description string `json:"description,omitempty"`
}

ClientOptions contains client-specific options that are persisted in local configuration file.

func (ClientOptions) ApplyDefaults added in v0.7.0 

func (o ClientOptions) ApplyDefaults(ctx context.Context, defaultDesc string) ClientOptions

ApplyDefaults returns a copy of ClientOptions with defaults filled out.

func (ClientOptions) Override added in v0.7.0 

func (o ClientOptions) Override(other ClientOptions) ClientOptions

Override returns ClientOptions that overrides fields present in the provided ClientOptions.

type ConnectOptions 

type ConnectOptions struct {
	PersistCredentials bool `json:"persistCredentials"`
	ClientOptions

	content.CachingOptions
}

ConnectOptions specifies options when persisting configuration to connect to a repository.

type DirectRepository added in v0.6.0 

type DirectRepository struct {
	Blobs     blob.Storage
	Content   *content.Manager
	Objects   *object.Manager
	Manifests *manifest.Manager
	UniqueID  []byte

	ConfigFile string
	// contains filtered or unexported fields
}

DirectRepository is an implementation of repository that directly manipulates underlying storage.

func OpenWithConfig 

func OpenWithConfig(ctx context.Context, st blob.Storage, lc *LocalConfig, password string, options *Options, caching *content.CachingOptions) (*DirectRepository, error)

OpenWithConfig opens the repository with a given configuration, avoiding the need for a config file.

func (*DirectRepository) BlobStorage added in v0.6.0 

func (r *DirectRepository) BlobStorage() blob.Storage

BlobStorage returns the blob storage.

func (*DirectRepository) ClientOptions added in v0.7.0 

func (r *DirectRepository) ClientOptions() ClientOptions

ClientOptions returns client options.

func (*DirectRepository) Close added in v0.6.0 

func (r *DirectRepository) Close(ctx context.Context) error

Close closes the repository and releases all resources.

func (*DirectRepository) ConfigFilename added in v0.6.0 

func (r *DirectRepository) ConfigFilename() string

ConfigFilename returns the name of the configuration file.

func (*DirectRepository) ContentManager added in v0.6.0 

func (r *DirectRepository) ContentManager() *content.Manager

ContentManager returns the content manager.

func (*DirectRepository) DeleteManifest added in v0.6.0 

func (r *DirectRepository) DeleteManifest(ctx context.Context, id manifest.ID) error

DeleteManifest deletes the manifest with a given ID.

func (*DirectRepository) DeriveKey added in v0.6.0 

func (r *DirectRepository) DeriveKey(purpose []byte, keyLength int) []byte

DeriveKey derives encryption key of the provided length from the master key.

func (*DirectRepository) FindManifests added in v0.6.0 

func (r *DirectRepository) FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error)

FindManifests returns metadata for manifests matching given set of labels.

func (*DirectRepository) Flush added in v0.6.0 

func (r *DirectRepository) Flush(ctx context.Context) error

Flush waits for all in-flight writes to complete.

func (*DirectRepository) GetManifest added in v0.6.0 

func (r *DirectRepository) GetManifest(ctx context.Context, id manifest.ID, data interface{}) (*manifest.EntryMetadata, error)

GetManifest returns the given manifest data and metadata.

func (*DirectRepository) Hostname added in v0.6.0 

func (r *DirectRepository) Hostname() string

Hostname returns the hostname that connected to the repository.

func (*DirectRepository) NewObjectWriter added in v0.6.0 

func (r *DirectRepository) NewObjectWriter(ctx context.Context, opt object.WriterOptions) object.Writer

NewObjectWriter creates an object writer.

func (*DirectRepository) OpenObject added in v0.6.0 

func (r *DirectRepository) OpenObject(ctx context.Context, id object.ID) (object.Reader, error)

OpenObject opens the reader for a given object, returns object.ErrNotFound.

func (*DirectRepository) PutManifest added in v0.6.0 

func (r *DirectRepository) PutManifest(ctx context.Context, labels map[string]string, payload interface{}) (manifest.ID, error)

PutManifest saves the given manifest payload with a set of labels.

func (*DirectRepository) Refresh added in v0.6.0 

func (r *DirectRepository) Refresh(ctx context.Context) error

Refresh periodically makes external changes visible to repository.

func (*DirectRepository) RefreshPeriodically added in v0.6.0 

func (r *DirectRepository) RefreshPeriodically(ctx context.Context, interval time.Duration)

RefreshPeriodically periodically refreshes the repository to reflect the changes made by other hosts.

func (*DirectRepository) SetCachingConfig added in v0.6.0 

func (r *DirectRepository) SetCachingConfig(ctx context.Context, opt *content.CachingOptions) error

SetCachingConfig changes caching configuration for a given repository.

func (*DirectRepository) Time added in v0.6.0 

func (r *DirectRepository) Time() time.Time

Time returns the current local time for the repo.

func (*DirectRepository) Token added in v0.6.0 

func (r *DirectRepository) Token(password string) (string, error)

Token returns an opaque token that contains repository connection information and optionally the provided password.

func (*DirectRepository) UpdateDescription added in v0.7.0 

func (r *DirectRepository) UpdateDescription(d string)

UpdateDescription updates the description of a connected repository.

func (*DirectRepository) Upgrade added in v0.6.0 

func (r *DirectRepository) Upgrade(ctx context.Context) error

Upgrade upgrades repository data structures to the latest version.

func (*DirectRepository) Username added in v0.6.0 

func (r *DirectRepository) Username() string

Username returns the username that's connect to the repository.

func (*DirectRepository) VerifyObject added in v0.6.0 

func (r *DirectRepository) VerifyObject(ctx context.Context, id object.ID) ([]content.ID, error)

VerifyObject verifies that the given object is stored properly in a repository and returns backing content IDs.

type LocalConfig 

type LocalConfig struct {
	// APIServer is only provided for remote repository.
	APIServer *APIServerInfo `json:"apiServer,omitempty"`

	// Storage is only provided for direct repository access.
	Storage *blob.ConnectionInfo `json:"storage,omitempty"`

	Caching *content.CachingOptions `json:"caching,omitempty"`

	ClientOptions
}

LocalConfig is a configuration of Kopia stored in a configuration file.

func (*LocalConfig) Load 

func (lc *LocalConfig) Load(r io.Reader) error

Load reads local configuration from the specified reader.

func (*LocalConfig) Save 

func (lc *LocalConfig) Save(w io.Writer) error

Save writes the configuration to the specified writer.

type NewRepositoryOptions 

type NewRepositoryOptions struct {
	UniqueID     []byte                    `json:"uniqueID"` // force the use of particular unique ID
	BlockFormat  content.FormattingOptions `json:"blockFormat"`
	DisableHMAC  bool                      `json:"disableHMAC"`
	ObjectFormat object.Format             `json:"objectFormat"` // object format
}

NewRepositoryOptions specifies options that apply to newly created repositories. All fields are optional, when not provided, reasonable defaults will be used.

type Options 

type Options struct {
	TraceStorage         func(f string, args ...interface{}) // Logs all storage access using provided Printf-style function
	ObjectManagerOptions object.ManagerOptions
	TimeNowFunc          func() time.Time // Time provider
}

Options provides configuration parameters for connection to a repository.

type Repository 

type Repository interface {
	OpenObject(ctx context.Context, id object.ID) (object.Reader, error)
	NewObjectWriter(ctx context.Context, opt object.WriterOptions) object.Writer
	VerifyObject(ctx context.Context, id object.ID) ([]content.ID, error)

	GetManifest(ctx context.Context, id manifest.ID, data interface{}) (*manifest.EntryMetadata, error)
	PutManifest(ctx context.Context, labels map[string]string, payload interface{}) (manifest.ID, error)
	FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error)
	DeleteManifest(ctx context.Context, id manifest.ID) error

	ClientOptions() ClientOptions
	UpdateDescription(d string)

	Time() time.Time

	Refresh(ctx context.Context) error
	Flush(ctx context.Context) error
	Close(ctx context.Context) error
}

Repository exposes public API of Kopia repository, including objects and manifests.

func Open 

func Open(ctx context.Context, configFile, password string, options *Options) (rep Repository, err error)

Open opens a Repository specified in the configuration file.

Source Files

View all Source files

Directories

Path Synopsis
Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs).
 Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs).
azure
Package azure implements Azure Blob Storage.
 Package azure implements Azure Blob Storage.
b2
Package b2 implements Storage based on an Backblaze B2 bucket.
 Package b2 implements Storage based on an Backblaze B2 bucket.
filesystem
Package filesystem implements filesystem-based Storage.
 Package filesystem implements filesystem-based Storage.
gcs
Package gcs implements Storage based on Google Cloud Storage bucket.
 Package gcs implements Storage based on Google Cloud Storage bucket.
logging
Package logging implements wrapper around Storage that logs all activity.
 Package logging implements wrapper around Storage that logs all activity.
providers
Package providers registers all storage providers that are included as part of Kopia.
 Package providers registers all storage providers that are included as part of Kopia.
rclone
Package rclone implements blob storage provider proxied by rclone (http://rclone.org)
 Package rclone implements blob storage provider proxied by rclone (http://rclone.org)
readonly
Package readonly implements wrapper around readonlyStorage that prevents all mutations.
 Package readonly implements wrapper around readonlyStorage that prevents all mutations.
s3
Package s3 implements Storage based on an S3 bucket.
 Package s3 implements Storage based on an S3 bucket.
sftp
Package sftp implements blob storage provided for SFTP/SSH.
 Package sftp implements blob storage provided for SFTP/SSH.
sharded
Package sharded implements common support for sharded blob providers, such as filesystem or webdav.
 Package sharded implements common support for sharded blob providers, such as filesystem or webdav.
webdav
Package webdav implements WebDAV-based Storage.
 Package webdav implements WebDAV-based Storage.
Package compression manages compression algorithm implementations.
 Package compression manages compression algorithm implementations.
Package content implements repository support for content-addressable storage.
 Package content implements repository support for content-addressable storage.
Package encryption manages content encryption algorithms.
 Package encryption manages content encryption algorithms.
Package hashing encapsulates all keyed hashing algorithms.
 Package hashing encapsulates all keyed hashing algorithms.
Package logging provides loggers for Kopia.
 Package logging provides loggers for Kopia.
Package maintenance manages automatic repository maintenance.
 Package maintenance manages automatic repository maintenance.
Package manifest implements support for managing JSON-based manifests in repository.
 Package manifest implements support for managing JSON-based manifests in repository.
Package object implements repository support for content-addressable objects of arbitrary size.
 Package object implements repository support for content-addressable objects of arbitrary size.
Package splitter manages splitting of object data into chunks.
 Package splitter manages splitting of object data into chunks.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.