rest

package
v0.0.0-...-8be083b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2024 License: BSD-2-Clause Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source 
const MaxRepoVersion = 2
View Source 
const MinRepoVersion = 1

Variables

View Source 
var ErrInvalidData = errors.New("invalid data returned")

ErrInvalidData is used to report that a file is corrupted

View Source 
var ErrNoSnapshotFound = errors.New("no snapshot found")

ErrNoSnapshotFound is returned when no snapshot for the given criteria could be found.

View Source 
var ErrTreeNotOrdered = errors.New("nodes are not ordered or duplicate")

Functions

func ForAllSnapshots 

func ForAllSnapshots(ctx context.Context, be Lister, loader LoaderUnpacked, excludeIDs IDSet, fn func(ID, *Snapshot, error) error) error

ForAllSnapshots reads all snapshots in parallel and calls the given function. It is guaranteed that the function is not run concurrently. If the called function returns an error, this function is cancelled and also returns this error. If a snapshot ID is in excludeIDs, it will be ignored.

func IsListxattrPermissionError 

func IsListxattrPermissionError(err error) bool

func LoadJSONUnpacked 

func LoadJSONUnpacked(ctx context.Context, repo LoaderUnpacked, t FileType, id ID, item interface{}) (err error)

LoadJSONUnpacked decrypts the data and afterwards calls json.Unmarshal on the item.

func ParallelList 

func ParallelList(ctx context.Context, r Lister, t FileType, parallelism uint, fn func(context.Context, ID, int64) error) error

func SaveConfig 

func SaveConfig(ctx context.Context, r SaverUnpacked, cfg Config) error

func ZeroPrefixLen 

func ZeroPrefixLen(p []byte) (n int)

ZeroPrefixLen returns the length of the longest all-zero prefix of p.

Types

type Blob 

type Blob struct {
	BlobHandle
	Length             uint
	Offset             uint
	UncompressedLength uint
}

func (Blob) IsCompressed 

func (b Blob) IsCompressed() bool

type BlobHandle 

type BlobHandle struct {
	ID   ID
	Type BlobType
}

type BlobLoader 

type BlobLoader interface {
	LoadBlob(context.Context, BlobType, ID, []byte) ([]byte, error)
}

type BlobSaver 

type BlobSaver interface {
	SaveBlob(context.Context, BlobType, []byte, ID, bool) (ID, bool, int, error)
}

type BlobSet 

type BlobSet map[BlobHandle]struct{}

func NewBlobSet 

func NewBlobSet(handles ...BlobHandle) BlobSet

func (BlobSet) Delete 

func (s BlobSet) Delete(h BlobHandle)

Delete removes id from the set.

func (BlobSet) Has 

func (s BlobSet) Has(h BlobHandle) bool

Has returns true iff id is contained in the set.

func (BlobSet) Insert 

func (s BlobSet) Insert(h BlobHandle)

Insert adds id to the set.

type BlobType 

type BlobType uint8

BlobType specifies what a blob stored in a pack is. 

const (
	InvalidBlob BlobType = iota
	DataBlob
	TreeBlob
	NumBlobTypes // Number of types. Must be last in this enumeration.
)

func (BlobType) IsMetadata 

func (t BlobType) IsMetadata() bool

type Config 

type Config struct {
	Version           uint        `json:"version"`
	ID                string      `json:"id"`
	ChunkerPolynomial chunker.Pol `json:"chunker_polynomial"`
}

func CreateConfig 

func CreateConfig(version uint) (Config, error)

func LoadConfig 

func LoadConfig(ctx context.Context, r LoaderUnpacked) (Config, error)

LoadConfig returns loads, checks and returns the config for a repository.

type ExtendedAttribute 

type ExtendedAttribute struct {
	Name  string `json:"name"`
	Value []byte `json:"value"`
}

ExtendedAttribute is a tuple storing the xattr name and value for various filesystems.

type FileType 

type FileType = backend.FileType
const (
	PackFile     FileType = backend.PackFile
	KeyFile      FileType = backend.KeyFile
	LockFile     FileType = backend.LockFile
	SnapshotFile FileType = backend.SnapshotFile
	IndexFile    FileType = backend.IndexFile
	ConfigFile   FileType = backend.ConfigFile
)

These are the different data types a backend can store.

type GenericAttributeType 

type GenericAttributeType string

GenericAttributeType can be used for OS specific functionalities by defining specific types in node.go to be used by the specific node_xx files. OS specific attribute types should follow the convention <OS>Attributes. GenericAttributeTypes should follow the convention <OS specific attribute type>.<attribute name> The attributes in OS specific attribute types must be pointers as we want to distinguish nil values and not create GenericAttributes for them.

type ID 

type ID [idSize]byte

ID references content within a repository.

func Find 

func Find(ctx context.Context, be Lister, t FileType, prefix string) (ID, error)

Find loads the list of all files of type t and searches for names which start with prefix. If none is found, nil and ErrNoIDPrefixFound is returned. If more than one is found, nil and ErrMultipleIDMatches is returned.

func Hash 

func Hash(data []byte) ID

Hash returns the ID for data.

func IDFromHash 

func IDFromHash(hash []byte) (id ID)

IDFromHash returns the ID for the hash.

func NewRandomID 

func NewRandomID() ID

NewRandomID returns a randomly generated ID. When reading from rand fails, the function panics.

func ParseID 

func ParseID(s string) (ID, error)

ParseID converts the given string to an ID.

func SaveJSONUnpacked 

func SaveJSONUnpacked(ctx context.Context, repo SaverUnpacked, t FileType, item interface{}) (ID, error)

SaveJSONUnpacked serialises item as JSON and encrypts and saves it in the backend as type t, without a pack. It returns the storage hash.

func SaveSnapshot 

func SaveSnapshot(ctx context.Context, repo SaverUnpacked, sn *Snapshot) (ID, error)

SaveSnapshot saves the snapshot sn and returns its ID.

func (ID) Equal 

func (id ID) Equal(other ID) bool

Equal compares an ID to another other.

func (ID) IsNull 

func (id ID) IsNull() bool

IsNull returns true iff id only consists of null bytes.

func (ID) MarshalJSON 

func (id ID) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON encoding of id.

func (*ID) Str 

func (id *ID) Str() string

Str returns the shortened string version of id.

func (ID) String 

func (id ID) String() string

func (*ID) UnmarshalJSON 

func (id *ID) UnmarshalJSON(b []byte) error

UnmarshalJSON parses the JSON-encoded data and stores the result in id.

type IDSet 

type IDSet map[ID]struct{}

IDSet is a set of IDs.

func (IDSet) Has 

func (s IDSet) Has(id ID) bool

Has returns true iff id is contained in the set.

type IDs 

type IDs []ID

type Lister 

type Lister interface {
	List(ctx context.Context, t FileType, fn func(ID, int64) error) error
}

Lister allows listing files in a backend.

type ListerLoaderUnpacked 

type ListerLoaderUnpacked interface {
	Lister
	LoaderUnpacked
}

type Loader 

type Loader interface {
	LoadBlob(context.Context, BlobType, ID, []byte) ([]byte, error)
	LookupBlobSize(tpe BlobType, id ID) (uint, bool)
	Connections() uint
}

Loader loads a blob from a repository.

type LoaderUnpacked 

type LoaderUnpacked interface {
	// Connections returns the maximum number of concurrent backend operations
	Connections() uint
	LoadUnpacked(ctx context.Context, t FileType, id ID) (data []byte, err error)
}

LoaderUnpacked allows loading a blob not stored in a pack file

type MultipleIDMatchesError 

type MultipleIDMatchesError struct {
	// contains filtered or unexported fields
}

A MultipleIDMatchesError is returned by Find() when multiple IDs with a given prefix are found.

func (*MultipleIDMatchesError) Error 

func (e *MultipleIDMatchesError) Error() string

type NoIDByPrefixError 

type NoIDByPrefixError struct {
	// contains filtered or unexported fields
}

A NoIDByPrefixError is returned by Find() when no ID for a given prefix could be found.

func (*NoIDByPrefixError) Error 

func (e *NoIDByPrefixError) Error() string

type Node 

type Node struct {
	Name       string      `json:"name"`
	Type       string      `json:"type"`
	Mode       os.FileMode `json:"mode,omitempty"`
	ModTime    time.Time   `json:"mtime,omitempty"`
	AccessTime time.Time   `json:"atime,omitempty"`
	ChangeTime time.Time   `json:"ctime,omitempty"`
	UID        uint32      `json:"uid"`
	GID        uint32      `json:"gid"`
	User       string      `json:"user,omitempty"`
	Group      string      `json:"group,omitempty"`
	Inode      uint64      `json:"inode,omitempty"`
	DeviceID   uint64      `json:"device_id,omitempty"` // device id of the file, stat.st_dev, only stored for hardlinks
	Size       uint64      `json:"size,omitempty"`
	Links      uint64      `json:"links,omitempty"`
	LinkTarget string      `json:"linktarget,omitempty"`
	// implicitly base64-encoded field. Only used while encoding, `linktarget_raw` will overwrite LinkTarget if present.
	// This allows storing arbitrary byte-sequences, which are possible as symlink targets on unix systems,
	// as LinkTarget without breaking backwards-compatibility.
	// Must only be set of the linktarget cannot be encoded as valid utf8.
	LinkTargetRaw      []byte                                   `json:"linktarget_raw,omitempty"`
	ExtendedAttributes []ExtendedAttribute                      `json:"extended_attributes,omitempty"`
	GenericAttributes  map[GenericAttributeType]json.RawMessage `json:"generic_attributes,omitempty"`
	Device             uint64                                   `json:"device,omitempty"` // in case of Type == "dev", stat.st_rdev
	Content            IDs                                      `json:"content"`
	Subtree            *ID                                      `json:"subtree,omitempty"`

	Error string `json:"error,omitempty"`

	Path string `json:"-"`
}

Node is a file, directory or other item in a backup.

func NodeFromFileInfo 

func NodeFromFileInfo(path string, fi os.FileInfo, ignoreXattrListError bool) (*Node, error)

NodeFromFileInfo returns a new node from the given path and FileInfo. It returns the first error that is encountered, together with a node.

func (Node) Equals 

func (node Node) Equals(other Node) bool

type PackedBlob 

type PackedBlob struct {
	Blob
	PackID ID
}

PackedBlob is a blob stored within a file.

type SaverUnpacked 

type SaverUnpacked interface {
	// Connections returns the maximum number of concurrent backend operations
	Connections() uint
	SaveUnpacked(ctx context.Context, t FileType, buf []byte) (ID, error)
}

SaverUnpacked allows saving a blob not stored in a pack file

type Snapshot 

type Snapshot struct {
	Time     time.Time `json:"time"`
	Parent   *ID       `json:"parent,omitempty"`
	Tree     *ID       `json:"tree"`
	Paths    []string  `json:"paths"`
	Hostname string    `json:"hostname,omitempty"`
	Username string    `json:"username,omitempty"`
	UID      uint32    `json:"uid,omitempty"`
	GID      uint32    `json:"gid,omitempty"`
	Excludes []string  `json:"excludes,omitempty"`
	Tags     []string  `json:"tags,omitempty"`
	Original *ID       `json:"original,omitempty"`

	ProgramVersion string           `json:"program_version,omitempty"`
	Summary        *SnapshotSummary `json:"summary,omitempty"`
	// contains filtered or unexported fields
}

Snapshot is the state of a resource at one point in time.

func FindSnapshot 

func FindSnapshot(ctx context.Context, be Lister, loader LoaderUnpacked, s string) (*Snapshot, string, error)

FindSnapshot takes a string and tries to find a snapshot whose ID matches the string as closely as possible.

func LoadSnapshot 

func LoadSnapshot(ctx context.Context, loader LoaderUnpacked, id ID) (*Snapshot, error)

LoadSnapshot loads the snapshot with the id and returns it.

func NewSnapshot 

func NewSnapshot(paths []string, tags []string, hostname string, time time.Time) (*Snapshot, error)

NewSnapshot returns an initialized snapshot struct for the current user and time.

func (*Snapshot) HasHostname 

func (sn *Snapshot) HasHostname(hostnames []string) bool

HasHostname returns true if either - the snapshot hostname is in the list of the given hostnames, or - the list of given hostnames is empty

func (*Snapshot) HasPaths 

func (sn *Snapshot) HasPaths(paths []string) bool

HasPaths returns true if the snapshot has all of the paths.

func (*Snapshot) HasTagList 

func (sn *Snapshot) HasTagList(l []TagList) bool

HasTagList returns true if either

  • the snapshot satisfies at least one TagList, so there is a TagList in l for which all tags are included in sn, or
  • l is empty

func (*Snapshot) HasTags 

func (sn *Snapshot) HasTags(l []string) bool

HasTags returns true if the snapshot has all the tags in l.

func (Snapshot) ID 

func (sn Snapshot) ID() *ID

ID returns the snapshot's ID.

type SnapshotFilter 

type SnapshotFilter struct {
	Hosts []string
	Tags  TagLists
	Paths []string
	// Match snapshots from before this timestamp. Zero for no limit.
	TimestampLimit time.Time
	// contains filtered or unexported fields
}

A SnapshotFilter denotes a set of snapshots based on hosts, tags and paths.

func (*SnapshotFilter) FindLatest 

func (f *SnapshotFilter) FindLatest(ctx context.Context, be Lister, loader LoaderUnpacked, snapshotID string) (*Snapshot, string, error)

FindLatest returns either the latest of a filtered list of all snapshots or a snapshot specified by `snapshotID`.

type SnapshotSummary 

type SnapshotSummary struct {
	BackupStart time.Time `json:"backup_start"`
	BackupEnd   time.Time `json:"backup_end"`

	// statistics from the backup json output
	FilesNew            uint   `json:"files_new"`
	FilesChanged        uint   `json:"files_changed"`
	FilesUnmodified     uint   `json:"files_unmodified"`
	DirsNew             uint   `json:"dirs_new"`
	DirsChanged         uint   `json:"dirs_changed"`
	DirsUnmodified      uint   `json:"dirs_unmodified"`
	DataBlobs           int    `json:"data_blobs"`
	TreeBlobs           int    `json:"tree_blobs"`
	DataAdded           uint64 `json:"data_added"`
	DataAddedPacked     uint64 `json:"data_added_packed"`
	TotalFilesProcessed uint   `json:"total_files_processed"`
	TotalBytesProcessed uint64 `json:"total_bytes_processed"`
}

type TagList 

type TagList []string

TagList is a list of tags.

type TagLists 

type TagLists []TagList

TagLists consists of several TagList.

type Tree 

type Tree struct {
	Nodes []*Node `json:"nodes"`
}

Tree is an ordered list of nodes.

func LoadTree 

func LoadTree(ctx context.Context, r BlobLoader, id ID) (*Tree, error)

LoadTree loads a tree from the repository.

func (*Tree) Find 

func (t *Tree) Find(name string) *Node

Find returns a node with the given name, or nil if none could be found.

type TreeJSONBuilder 

type TreeJSONBuilder struct {
	// contains filtered or unexported fields
}

func NewTreeJSONBuilder 

func NewTreeJSONBuilder() *TreeJSONBuilder

func (*TreeJSONBuilder) AddNode 

func (builder *TreeJSONBuilder) AddNode(node *Node) error

func (*TreeJSONBuilder) Finalize 

func (builder *TreeJSONBuilder) Finalize() ([]byte, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.