Documentation
¶
Index ¶
Constants ¶
const MaxPartSize = 1024 * 1024 * 1024
MaxPartSize is the maximum size for each part.
The MaxPartSize reduces bandwidth usage during retires on network errors when transferring multi-TB files.
Variables ¶
var ( // DeleteAllObjectVersions is a flag for whether to prune previous object versions when deleting an object. DeleteAllObjectVersions = flag.Bool("deleteAllObjectVersions", false, "Whether to prune previous object versions when deleting an object. "+ "By default, when object storage has versioning enabled deleting the file removes only current version. "+ "This option forces removal of all previous versions. "+ "See: https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages") )
Functions ¶
func ToCanonicalPath ¶ added in v1.93.5
ToCanonicalPath returns canonical path by replacing local directory separators with `/`.
Types ¶
type OriginFS ¶
type OriginFS interface {
// MustStop must be called when the RemoteFS is no longer needed.
MustStop()
// String must return human-readable representation of OriginFS.
String() string
// ListParts must return all the parts for the OriginFS.
ListParts() ([]Part, error)
}
OriginFS is an interface for remote origin filesystem.
This filesystem is used for performing server-side file copies instead of uploading data from local filesystem.
type Part ¶
type Part struct {
// Path is the path to file for backup.
//
// Path must consistently use `/` as directory separator.
// Use ToCanonicalPath() function for converting local directory separators to `/`.
Path string
// FileSize is the size of the whole file for the given part.
FileSize uint64
// Offset is offset in the file to backup.
Offset uint64
// Size is the size of the part to backup starting from Offset.
Size uint64
// ActualSize is the actual size of the part.
//
// The part is considered broken if it isn't equal to Size.
// Such a part must be removed from remote storage.
ActualSize uint64
}
Part is an atomic unit for transfer during backup / restore.
Each source file can be split into parts with up to MaxPartSize sizes.
func PartsIntersect ¶
PartsIntersect returns the intersection of a and b
func (*Part) ParseFromRemotePath ¶
ParseFromRemotePath parses p from remotePath.
Returns true on success.
remotePath must be in canonical form received from ToCanonicalPath().
func (*Part) RemotePath ¶
RemotePath returns remote path for the part p and the given prefix.
type RemoteFS ¶
type RemoteFS interface {
// MustStop must be called when the RemoteFS is no longer needed.
MustStop()
// String must return human-readable representation of RemoteFS.
String() string
// ListParts must return all the parts for the RemoteFS.
ListParts() ([]Part, error)
// DeletePart must delete part p from RemoteFS.
DeletePart(p Part) error
// RemoveEmptyDirs must recursively remove empty directories in RemoteFS.
RemoveEmptyDirs() error
// CopyPart must copy part p from dstFS to RemoteFS.
CopyPart(dstFS OriginFS, p Part) error
// DownloadPart must download part p from RemoteFS to w.
DownloadPart(p Part, w io.Writer) error
// UploadPart must upload part p from r to RemoteFS.
UploadPart(p Part, r io.Reader) error
// DeleteFile deletes filePath at RemoteFS.
//
// filePath must use / as directory delimiters.
DeleteFile(filePath string) error
// CreateFile creates filePath at RemoteFS and puts data into it.
//
// filePath must use / as directory delimiters.
CreateFile(filePath string, data []byte) error
// HasFile returns true if filePath exists at RemoteFS.
//
// filePath must use / as directory delimiters.
HasFile(filePath string) (bool, error)
// ReadFile returns file contents at the given filePath.
//
// filePath must use / as directory delimiters.
ReadFile(filePath string) ([]byte, error)
}
RemoteFS is a filesystem where backups are stored.
Source Files