repo

package
v0.11.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 11, 2022 License: Apache-2.0 Imports: 53 Imported by: 13

Documentation

Overview

Package repo implements content-addressable Repository on top of BLOB storage.

Index

Constants

View Source 
const BlobCfgBlobID = "kopia.blobcfg"

BlobCfgBlobID is the identifier of a BLOB that describes BLOB retention settings for the repository.

View Source 
const CacheDirMarkerFile = "CACHEDIR.TAG"

CacheDirMarkerFile is the name of the marker file indicating a directory contains Kopia caches. See https://bford.info/cachedir/

View Source 
const CacheDirMarkerHeader = "Signature: 8a477f597d28d172789f06886806bc55"

CacheDirMarkerHeader is the header signature for cache dir marker files.

View Source 
const FormatBlobBackupIDPrefix = "kopia.repository.backup."

FormatBlobBackupIDPrefix is the prefix for all identifiers of the BLOBs that keep a backup copy of the FormatBlobID BLOB for the purposes of rollback during upgrade.

View Source 
const FormatBlobID = "kopia.repository"

FormatBlobID is the identifier of a BLOB that describes repository format.

View Source 
const MaxGRPCMessageSize = 20 << 20

MaxGRPCMessageSize is the maximum size of a message sent or received over GRPC API when talking to Kopia repository server. This is bigger than the size of any possible content, which is defined by supported splitters.

Variables

View Source 
var (
	BuildInfo       = "unknown"
	BuildVersion    = "v0-unofficial"
	BuildGitHubRepo = ""
)

BuildInfo is the build information of Kopia. nolint:gochecknoglobals

View Source 
var ErrAlreadyInitialized = errors.Errorf("repository already initialized")

ErrAlreadyInitialized indicates that repository has already been initialized.

View Source 
var ErrInvalidPassword = errors.Errorf("invalid repository password")

ErrInvalidPassword is returned when repository password is invalid.

View Source 
var ErrRepositoryNotInitialized = errors.Errorf("repository not initialized in the provided storage")

ErrRepositoryNotInitialized is returned when attempting to connect to repository that has not been initialized.

View Source 
var ErrRepositoryUnavailableDueToUpgrageInProgress = errors.Errorf("repository upgrade in progress")

ErrRepositoryUnavailableDueToUpgrageInProgress is returned when repository is undergoing upgrade that requires exclusive access.

Functions

func Connect 

func Connect(ctx context.Context, configFile string, st blob.Storage, password string, opt *ConnectOptions) error

Connect connects to the repository in the specified storage and persists the configuration and credentials in the file provided.

func ConnectAPIServer added in v0.6.0 

func ConnectAPIServer(ctx context.Context, configFile string, si *APIServerInfo, password string, opt *ConnectOptions) error

ConnectAPIServer sets up repository connection to a particular API server.

func DecodeToken 

func DecodeToken(token string) (blob.ConnectionInfo, string, error)

DecodeToken decodes the provided token and returns connection info and password if persisted.

func DirectWriteSession added in v0.8.0 

func DirectWriteSession(ctx context.Context, r DirectRepository, opt WriteSessionOptions, cb func(ctx context.Context, dw DirectRepositoryWriter) error) error

DirectWriteSession executes the provided callback in a DirectRepositoryWriter created for the purpose and flushes writes.

func Disconnect 

func Disconnect(ctx context.Context, configFile string) error

Disconnect removes the specified configuration file and any local cache directories.

func FormatBlobBackupID added in v0.10.6 

func FormatBlobBackupID(l content.UpgradeLock) blob.ID

FormatBlobBackupID gets the upgrade backu pblob-id fro mthe lock.

func GetCachingOptions added in v0.8.0 

func GetCachingOptions(ctx context.Context, configFile string) (*content.CachingOptions, error)

GetCachingOptions reads caching configuration for a given repository.

func GetDefaultHostName added in v0.7.0 

func GetDefaultHostName(ctx context.Context) string

GetDefaultHostName returns default hostname.

func GetDefaultUserName added in v0.7.0 

func GetDefaultUserName(ctx context.Context) string

GetDefaultUserName returns default username.

func Initialize 

func Initialize(ctx context.Context, st blob.Storage, opt *NewRepositoryOptions, password string) error

Initialize creates initial repository data structures in the specified storage with given credentials.

func RecoverFormatBlob 

func RecoverFormatBlob(ctx context.Context, st blob.Storage, blobID blob.ID, optionalLength int64) ([]byte, error)

RecoverFormatBlob attempts to recover format blob replica from the specified file. The format blob can be either the prefix or a suffix of the given file. optionally the length can be provided (if known) to speed up recovery.

func SetCachingOptions added in v0.8.0 

func SetCachingOptions(ctx context.Context, configFile string, opt *content.CachingOptions) error

SetCachingOptions changes caching configuration for a given repository.

func SetClientOptions added in v0.7.0 

func SetClientOptions(ctx context.Context, configFile string, cliOpt ClientOptions) error

SetClientOptions updates client options stored in the provided configuration file.

func WriteSession added in v0.8.0 

func WriteSession(ctx context.Context, r Repository, opt WriteSessionOptions, cb func(ctx context.Context, w RepositoryWriter) error) error

WriteSession executes the provided callback in a repository writer created for the purpose and flushes writes.

Types

type APIServerInfo added in v0.6.0 

type APIServerInfo struct {
	BaseURL                             string `json:"url"`
	TrustedServerCertificateFingerprint string `json:"serverCertFingerprint"`
	DisableGRPC                         bool   `json:"disableGRPC,omitempty"`
}

APIServerInfo is remote repository configuration stored in local configuration.

type ClientOptions added in v0.7.0 

type ClientOptions struct {
	Hostname string `json:"hostname"`
	Username string `json:"username"`

	ReadOnly bool `json:"readonly,omitempty"`

	// Description is human-readable description of the repository to use in the UI.
	Description string `json:"description,omitempty"`

	EnableActions bool `json:"enableActions"`

	FormatBlobCacheDuration time.Duration `json:"formatBlobCacheDuration,omitempty"`

	Throttling *throttling.Limits `json:"throttlingLimits,omitempty"`
}

ClientOptions contains client-specific options that are persisted in local configuration file.

func (ClientOptions) ApplyDefaults added in v0.7.0 

func (o ClientOptions) ApplyDefaults(ctx context.Context, defaultDesc string) ClientOptions

ApplyDefaults returns a copy of ClientOptions with defaults filled out.

func (ClientOptions) Override added in v0.7.0 

func (o ClientOptions) Override(other ClientOptions) ClientOptions

Override returns ClientOptions that overrides fields present in the provided ClientOptions.

func (ClientOptions) UsernameAtHost added in v0.8.0 

func (o ClientOptions) UsernameAtHost() string

UsernameAtHost returns 'username@hostname' string.

type ConnectOptions 

type ConnectOptions struct {
	ClientOptions

	content.CachingOptions
}

ConnectOptions specifies options when persisting configuration to connect to a repository.

type DirectRepository added in v0.6.0 

type DirectRepository interface {
	Repository

	ObjectFormat() object.Format
	BlobCfg() content.BlobCfgBlob
	BlobReader() blob.Reader
	BlobVolume() blob.Volume
	ContentReader() content.Reader
	IndexBlobs(ctx context.Context, includeInactive bool) ([]content.IndexBlobInfo, error)
	Crypter() *content.Crypter
	NewDirectWriter(ctx context.Context, opt WriteSessionOptions) (context.Context, DirectRepositoryWriter, error)
	AlsoLogToContentLog(ctx context.Context) context.Context
	UniqueID() []byte
	ConfigFilename() string
	DeriveKey(purpose []byte, keyLength int) []byte
	Token(password string) (string, error)
	Throttler() throttling.SettableThrottler
	DisableIndexRefresh()
}

DirectRepository provides additional low-level repository functionality.

type DirectRepositoryWriter added in v0.8.0 

type DirectRepositoryWriter interface {
	RepositoryWriter
	DirectRepository
	BlobStorage() blob.Storage
	ContentManager() *content.WriteManager
	SetParameters(ctx context.Context, m content.MutableParameters, blobcfg content.BlobCfgBlob) error
	ChangePassword(ctx context.Context, newPassword string) error
	SetUpgradeLockIntent(ctx context.Context, l content.UpgradeLock) (*content.UpgradeLock, error)
	CommitUpgrade(ctx context.Context) error
	RollbackUpgrade(ctx context.Context) error
}

DirectRepositoryWriter provides low-level write access to the repository.

type LocalConfig 

type LocalConfig struct {
	// APIServer is only provided for remote repository.
	APIServer *APIServerInfo `json:"apiServer,omitempty"`

	// Storage is only provided for direct repository access.
	Storage *blob.ConnectionInfo `json:"storage,omitempty"`

	Caching *content.CachingOptions `json:"caching,omitempty"`

	ClientOptions
}

LocalConfig is a configuration of Kopia stored in a configuration file.

func LoadConfigFromFile added in v0.8.0 

func LoadConfigFromFile(fileName string) (*LocalConfig, error)

LoadConfigFromFile reads the local configuration from the specified file.

type NewRepositoryOptions 

type NewRepositoryOptions struct {
	UniqueID        []byte                    `json:"uniqueID"` // force the use of particular unique ID
	BlockFormat     content.FormattingOptions `json:"blockFormat"`
	DisableHMAC     bool                      `json:"disableHMAC"`
	ObjectFormat    object.Format             `json:"objectFormat"` // object format
	RetentionMode   blob.RetentionMode        `json:"retentionMode,omitempty"`
	RetentionPeriod time.Duration             `json:"retentionPeriod,omitempty"`
}

NewRepositoryOptions specifies options that apply to newly created repositories. All fields are optional, when not provided, reasonable defaults will be used.

type Options 

type Options struct {
	TraceStorage       bool             // Logs all storage access using provided Printf-style function
	TimeNowFunc        func() time.Time // Time provider
	DisableInternalLog bool             // Disable internal log
	UpgradeOwnerID     string           // Owner-ID of any upgrade in progress, when this is not set the access may be restricted
}

Options provides configuration parameters for connection to a repository.

type Repository 

type Repository interface {
	OpenObject(ctx context.Context, id object.ID) (object.Reader, error)
	VerifyObject(ctx context.Context, id object.ID) ([]content.ID, error)
	GetManifest(ctx context.Context, id manifest.ID, data interface{}) (*manifest.EntryMetadata, error)
	FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error)
	ContentInfo(ctx context.Context, contentID content.ID) (content.Info, error)
	PrefetchContents(ctx context.Context, contentIDs []content.ID, hint string) []content.ID
	PrefetchObjects(ctx context.Context, objectIDs []object.ID, hint string) ([]content.ID, error)
	Time() time.Time
	ClientOptions() ClientOptions
	NewWriter(ctx context.Context, opt WriteSessionOptions) (context.Context, RepositoryWriter, error)
	UpdateDescription(d string)
	Refresh(ctx context.Context) error
	Close(ctx context.Context) error
}

Repository exposes public API of Kopia repository, including objects and manifests.

func Open 

func Open(ctx context.Context, configFile, password string, options *Options) (rep Repository, err error)

Open opens a Repository specified in the configuration file.

func OpenAPIServer added in v0.8.0 

func OpenAPIServer(ctx context.Context, si *APIServerInfo, cliOpts ClientOptions, cachingOptions *content.CachingOptions, password string) (Repository, error)

OpenAPIServer connects remote repository over Kopia API.

func OpenGRPCAPIRepository added in v0.8.0 

func OpenGRPCAPIRepository(ctx context.Context, si *APIServerInfo, cliOpts ClientOptions, contentCache *cache.PersistentCache, password string) (Repository, error)

OpenGRPCAPIRepository opens the Repository based on remote GRPC server. The APIServerInfo must have the address of the repository as 'https://host:port'

type RepositoryWriter added in v0.8.0 

type RepositoryWriter interface {
	Repository

	NewObjectWriter(ctx context.Context, opt object.WriterOptions) object.Writer
	ConcatenateObjects(ctx context.Context, objectIDs []object.ID) (object.ID, error)
	PutManifest(ctx context.Context, labels map[string]string, payload interface{}) (manifest.ID, error)
	DeleteManifest(ctx context.Context, id manifest.ID) error
	Flush(ctx context.Context) error
}

RepositoryWriter provides methods to write to a repository.

type WriteSessionOptions added in v0.8.0 

type WriteSessionOptions struct {
	Purpose        string
	FlushOnFailure bool        // whether to flush regardless of write session result.
	OnUpload       func(int64) // function to invoke after completing each upload in the session.
}

WriteSessionOptions describes options for a write session.

Source Files

View all Source files

Directories

Path Synopsis
Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs).
 Package blob implements simple storage of immutable, unstructured binary large objects (BLOBs).
azure
Package azure implements Azure Blob Storage.
 Package azure implements Azure Blob Storage.
b2
Package b2 implements Storage based on an Backblaze B2 bucket.
 Package b2 implements Storage based on an Backblaze B2 bucket.
beforeop
Package beforeop implements wrapper around blob.Storage that run a given callback before all operations.
 Package beforeop implements wrapper around blob.Storage that run a given callback before all operations.
filesystem
Package filesystem implements filesystem-based Storage.
 Package filesystem implements filesystem-based Storage.
gcs
Package gcs implements Storage based on Google Cloud Storage bucket.
 Package gcs implements Storage based on Google Cloud Storage bucket.
gdrive
Package gdrive implements Storage based on Google Drive.
 Package gdrive implements Storage based on Google Drive.
logging
Package logging implements wrapper around Storage that logs all activity.
 Package logging implements wrapper around Storage that logs all activity.
rclone
Package rclone implements blob storage provider proxied by rclone (http://rclone.org)
 Package rclone implements blob storage provider proxied by rclone (http://rclone.org)
readonly
Package readonly implements wrapper around readonlyStorage that prevents all mutations.
 Package readonly implements wrapper around readonlyStorage that prevents all mutations.
retrying
Package retrying implements wrapper around blob.Storage that adds retry loop around all operations in case they return unexpected errors.
 Package retrying implements wrapper around blob.Storage that adds retry loop around all operations in case they return unexpected errors.
s3
Package s3 implements Storage based on an S3 bucket.
 Package s3 implements Storage based on an S3 bucket.
sftp
Package sftp implements blob storage provided for SFTP/SSH.
 Package sftp implements blob storage provided for SFTP/SSH.
sharded
Package sharded implements common support for sharded blob providers, such as filesystem or webdav.
 Package sharded implements common support for sharded blob providers, such as filesystem or webdav.
throttling
Package throttling implements wrapper around blob.Storage that adds throttling to all calls.
 Package throttling implements wrapper around blob.Storage that adds throttling to all calls.
webdav
Package webdav implements WebDAV-based Storage.
 Package webdav implements WebDAV-based Storage.
Package compression manages compression algorithm implementations.
 Package compression manages compression algorithm implementations.
Package content implements repository support for content-addressable storage.
 Package content implements repository support for content-addressable storage.
index
Package index manages content indices.
 Package index manages content indices.
Package encryption manages content encryption algorithms.
 Package encryption manages content encryption algorithms.
Package hashing encapsulates all keyed hashing algorithms.
 Package hashing encapsulates all keyed hashing algorithms.
Package logging provides loggers for Kopia.
 Package logging provides loggers for Kopia.
Package maintenance manages automatic repository maintenance.
 Package maintenance manages automatic repository maintenance.
Package manifest implements support for managing JSON-based manifests in repository.
 Package manifest implements support for managing JSON-based manifests in repository.
Package object implements repository support for content-addressable objects of arbitrary size.
 Package object implements repository support for content-addressable objects of arbitrary size.
Package splitter manages splitting of object data into chunks.
 Package splitter manages splitting of object data into chunks.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.