set

Documentation

Index

• Constants
• type DB
  • func New(path string) (*DB, error)
  • func (d *DB) AddOrUpdate(set *Set) error
  • func (d *DB) Close() error
  • func (d *DB) GetAll() ([]*Set, error)
  • func (d *DB) GetByID(id string) *Set
  • func (d *DB) GetByRequester(requester string) ([]*Set, error)
  • func (d *DB) GetDirEntries(setID string) ([]*Entry, error)
  • func (d *DB) GetFailedEntries(setID string) ([]*Entry, int, error)
  • func (d *DB) GetFileEntries(setID string) ([]*Entry, error)
  • func (d *DB) GetPureFileEntries(setID string) ([]*Entry, error)
  • func (d *DB) SetDirEntries(setID string, paths []string) error
  • func (d *DB) SetDiscoveredEntries(setID string, paths []string) (*Set, error)
  • func (d *DB) SetDiscoveryStarted(setID string) error
  • func (d *DB) SetEntryStatus(r *put.Request) (*Entry, error)
  • func (d *DB) SetError(setID, errMsg string) error
  • func (d *DB) SetFileEntries(setID string, paths []string) error
• type Entry
  • func (e *Entry) ShouldUpload(reuploadAfter time.Time) bool
• type EntryStatus
  • func (e EntryStatus) String() string
• type Error
  • func (e Error) Error() string
• type Set
  • func (s *Set) Count() string
  • func (s *Set) Discovered() string
  • func (s *Set) ID() string
  • func (s *Set) MakeTransformer() (put.PathTransformer, error)
  • func (s *Set) Size() string
• type Status
  • func (s Status) String() string

Constants

const (
	ErrInvalidSetID   = "invalid set ID"
	ErrInvalidRequest = "request lacks Requester or Set"
	ErrInvalidEntry   = "invalid set entry"

	AttemptsToBeConsideredFailing = 3
)

const (
	ErrInvalidTransformer = "invalid transformer"
)

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is used to create and query a database for storing backup sets (lists of files a user wants to have backed up) and their backup status.

func New

func New(path string) (*DB, error)

New returns a *DB that can be used to create or query a set database. Provide the path to the database file.

Returns an error if path exists but can't be opened, or if it doesn't exist and can't be created.

func (*DB) AddOrUpdate

func (d *DB) AddOrUpdate(set *Set) error

AddOrUpdate adds or updates the given Set to the database.

func (*DB) Close

func (d *DB) Close() error

Close closes the database. Be sure to call this to finalise any writes to disk correctly.

func (*DB) GetAll

func (d *DB) GetAll() ([]*Set, error)

GetAll returns all the Sets previously added to the database.

func (*DB) GetByID

func (d *DB) GetByID(id string) *Set

GetByID returns the Sets with the given ID previously added to the database. Returns nil if such a set does not exist.

func (*DB) GetByRequester

func (d *DB) GetByRequester(requester string) ([]*Set, error)

GetByRequester returns all the Sets previously added to the database by the given requester.

func (*DB) GetDirEntries

func (d *DB) GetDirEntries(setID string) ([]*Entry, error)

GetDirEntries returns all the dir entries for the given set.

func (*DB) GetFailedEntries

func (d *DB) GetFailedEntries(setID string) ([]*Entry, int, error)

GetFailedEntries returns up to 10 of the file entries for the given set (both SetFileEntries and SetDiscoveredEntries) that have a failed status. Also returns the number of failed entries that were not returned.

func (*DB) GetFileEntries

func (d *DB) GetFileEntries(setID string) ([]*Entry, error)

GetFileEntries returns all the file entries for the given set (both SetFileEntries and SetDiscoveredEntries).

func (*DB) GetPureFileEntries

func (d *DB) GetPureFileEntries(setID string) ([]*Entry, error)

GetPureFileEntries returns all the file entries for the given set (only SetFileEntries, not SetDiscoveredEntries).

func (*DB) SetDirEntries

func (d *DB) SetDirEntries(setID string, paths []string) error

SetDirEntries sets the directory paths for the given backup set. Only supply absolute paths to directories.

func (*DB) SetDiscoveredEntries

func (d *DB) SetDiscoveredEntries(setID string, paths []string) (*Set, error)

SetDiscoveredEntries sets discovered file paths for the given backup set's directory entries. Only supply absolute paths to files.

It also updates LastDiscovery, sets NumFiles and sets status to PendingUpload.

Returns the updated set and an error if the setID isn't in the database.

func (*DB) SetDiscoveryStarted

func (d *DB) SetDiscoveryStarted(setID string) error

SetDiscoveryStarted updates StartedDiscovery and resets some status values for the given set. Returns an error if the setID isn't in the database.

func (*DB) SetEntryStatus

func (d *DB) SetEntryStatus(r *put.Request) (*Entry, error)

SetEntryStatus finds the set Entry corresponding to the given Request's Local path, Requester and Set name, and updates its status in the database, and also updates summary status for the Set.

Returns the Entry, which is a reflection of the Request, but contains additional information such as the number of Attempts if the Request is failing and you keep retrying.

Returns an error if a set or entry corresponding to the Request can't be found.

func (*DB) SetError

func (d *DB) SetError(setID, errMsg string) error

SetError updates a set with the given error message. Returns an error if the setID isn't in the database.

func (*DB) SetFileEntries

func (d *DB) SetFileEntries(setID string, paths []string) error

SetFileEntries sets the file paths for the given backup set. Only supply absolute paths to files.

type Entry

type Entry struct {
	Path        string
	Size        uint64 // size of local file in bytes.
	Status      EntryStatus
	LastError   string
	LastAttempt time.Time
	Attempts    int
	// contains filtered or unexported fields
}

Entry holds the status of an entry in a backup set.

func (*Entry) ShouldUpload

func (e *Entry) ShouldUpload(reuploadAfter time.Time) bool

ShouldUpload returns true if this Entry is pending or has failed less than 3 times or was previously uploaded before the given time.

type EntryStatus

type EntryStatus int

const (
	// Pending is an Entry status meaning the file has not yet had an upload
	// attempt.
	Pending EntryStatus = iota

	// UploadingEntry is an Entry status meaning the file has started to upload.
	UploadingEntry

	// Uploaded is an Entry status meaning the file has been uploaded
	// successfully.
	Uploaded

	// Failed is an Entry status meaning the file failed to upload due to some
	// remote issue.
	Failed

	// Failed is an Entry status meaning the local file is missing so can't be
	// uploaded. Symlinks are considered to be "missing" since they can't be
	// uploaded.
	Missing
)

func (EntryStatus) String

func (e EntryStatus) String() string

String lets you convert a EntryStatus to a meaningful string.

type Error

type Error struct {
	// contains filtered or unexported fields
}

func (Error) Error

func (e Error) Error() string

type Set

type Set struct {
	// An arbitrary (short) name for this backup set.
	Name string

	// The username of the person requesting this backup.
	Requester string

	// The method of transforming local Entries paths in to remote paths, to
	// determine the upload location. "humgen" to use the put.HumgenTransformer,
	// or "prefix=local:remote" to use the put.PrefixTransformer.
	Transformer string

	// Monitor the files and directories and re-upload them whenever they
	// change. Optional, defaults to unmonitored (a one time upload of Entries).
	Monitor bool

	// An optional longer free-text description of this backup set.
	Description string

	// Delete local paths after successful upload. Optional, defaults to no
	// deletions (ie. do a backup, not a move).
	DeleteLocal bool

	// StartedDiscovery provides the last time that discovery started. This is a
	// read-only value.
	StartedDiscovery time.Time

	// LastDiscovery provides the last time that discovery completed. This is a
	// read-only value.
	LastDiscovery time.Time

	// NumFiles provides the total number of set and discovered files in this
	// set, as of the last discovery. This is a read-only value.
	NumFiles uint64

	// SizeFiles provides the total size (bytes) of set and discovered files in
	// this set, as of the last discovery. This is a read-only value.
	SizeFiles uint64

	// Uploaded provides the total number of set and discovered files in this
	// set that have been uploaded or confirmed uploaded since the last
	// discovery. This is a read-only value.
	Uploaded uint64

	// Failed provides the total number of set and discovered files in this set
	// that have failed their upload since the last discovery. This is a
	// read-only value.
	Failed uint64

	// Missing provides the total number of set and discovered files in this set
	// that no longer exist locally since the last discovery. This is a
	// read-only value.
	Missing uint64

	// Status provides the current status for the set since the last discovery.
	// This is a read-only value.
	Status Status

	// LastCompleted provides the last time that all uploads completed
	// (regardless of success or failure). This is a read-only value.
	LastCompleted time.Time

	// LastCompletedCount provides the count of files on the last upload attempt
	// (those successfully uploaded, those which failed, but not those which
	// were missing locally). This is a read-only value.
	LastCompletedCount uint64

	// LastCompletedSize provides the size of files (bytes) counted in
	// LastCompletedCount. This is a read-only value.
	LastCompletedSize uint64

	// Error holds any error that applies to the whole set, such as an issue
	// with the Transformer. This is a read-only value.
	Error string
}

Set describes a backup set; a list of files and directories to backup, plus some metadata. All properties are required unless otherwise noted.

func (*Set) Count

func (s *Set) Count() string

Count provides a string representation of NumFiles, or if 0, returns the LastCompletedCount with a textual note to that effect.

func (*Set) Discovered

func (s *Set) Discovered() string

Discovered provides a string representation of when discovery last completed, or if it hasn't, when it started, or if it hasn't, says "not started".

func (*Set) ID

func (s *Set) ID() string

ID returns an ID for this set, generated deterministiclly from its Name and Requester. Ie. it is unique between all requesters, and amongst a requester's differently named sets. Sets with the same Name and Requester will have the same ID().

func (*Set) MakeTransformer

func (s *Set) MakeTransformer() (put.PathTransformer, error)

MakeTransformer turns our Transformer string in to a put.HumgenTransformer or a put.PrefixTransformer as appropriate.

func (*Set) Size

func (s *Set) Size() string

Size provides a string representation of SizeFiles in a human readable format, or if 0, returns the LastCompletedSize with a textual note to that effect.

type Status

type Status int

const (
	// PendingDiscovery is a Set status meaning the set's entries are pending
	// existence, size and directory content discovery.
	PendingDiscovery Status = iota

	// PendingUpload is a Set status meaning discovery has completed, but no
	// entries have been uploaded since then.
	PendingUpload

	// Uploading is a Set status meaning discovery has completed and upload of
	// entries has started.
	Uploading

	// Failing is a Set status meaning at least 1 of the entries has failed to
	// upload after 3 attempts. Other uploads are ongoing.
	Failing

	// Complete is a Set status meaning all entries have had an upload attempt
	// since the last discovery. (Some uploads may have failed, but they had
	// 3 retries.)
	Complete
)

func (Status) String

func (s Status) String() string

String lets you convert a Status to a meaningful string.