backups

package
v0.0.0-...-9ec3720 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 6, 2021 License: AGPL-3.0 Imports: 34 Imported by: 0

Documentation

Overview

Package backups contains all the stand-alone backup-related functionality for juju state. That functionality is encapsulated by the backups.Backups type. The package also exposes a few key helpers and components.

Backups are not a part of juju state nor of normal state operations. However, they certainly are tightly coupled with state (the very subject of backups). This puts backups in an odd position, particularly with regard to the storage of backup metadata and archives.

As noted above backups are about state but not a part of state. So exposing backup-related methods on State would imply the wrong thing. Thus most of the functionality here is defined at a high level without relation to state. A few low-level parts or helpers are exposed as functions to which you pass a state value. Those are kept to a minimum.

Note that state (and juju as a whole) currently does not have a persistence layer abstraction to facilitate separating different persistence needs and implementations. As a consequence, state's data, whether about how an model should look or about existing resources within an model, is dumped essentially straight into State's mongo connection. The code in the state package does not make any distinction between the two (nor does the package clearly distinguish between state-related abstractions and state-related data).

Backups add yet another category, merely taking advantage of State's mongo for storage. In the interest of making the distinction clear, among other reasons, backups uses its own database under state's mongo connection.

Index

Constants

View Source 
const (
	// FilenamePrefix is the prefix used for backup archive files.
	FilenamePrefix = "juju-backup-"

	// FilenameTemplate is used with time.Time.Format to generate a filename.
	FilenameTemplate = FilenamePrefix + "20060102-150405.tar.gz"
)
View Source 
const (
	TempFilename = "juju-backup.tar.gz"
)
View Source 
const UnknownString = "<unknown>"

UnknownString is a marker value for string fields with unknown values.

Variables

View Source 
var UnknownInt64 = int64(math.MaxInt64)

UnknownInt64 is a marker value for int64 fields with unknown values.

View Source 
var UnknownVersion = version.MustParse("9999.9999.9999")

UnknownVersion is a marker value for version fields with unknown values.

Functions

func GetFilesToBackUp 

func GetFilesToBackUp(rootDir string, paths *Paths, oldmachine string) ([]string, error)

GetFilesToBackUp returns the paths that should be included in the backup archive.

func NewStorage 

func NewStorage(st DB) filestorage.FileStorage

NewStorage returns a new FileStorage to use for storing backup archives (and metadata).

func StoreArchive 

func StoreArchive(stor filestorage.FileStorage, meta *Metadata, file io.Reader) error

StoreArchive sends the backup archive and its metadata to storage. It also sets the metadata's ID and Stored values.

Types

type ArchiveData 

type ArchiveData struct {
	ArchivePaths
	// contains filtered or unexported fields
}

ArchiveData is a wrapper around a the uncompressed data in a backup archive file. It provides access to the content of the archive. While ArchiveData provides useful functionality, it may not be appropriate for large archives. The contents of the archive are kept in-memory, so large archives could be too taxing on the host. In that case consider using ArchiveWorkspace instead.

func NewArchiveData 

func NewArchiveData(data []byte) *ArchiveData

NewArchiveData builds a new archive data wrapper for the given uncompressed data.

func NewArchiveDataReader 

func NewArchiveDataReader(r io.Reader) (*ArchiveData, error)

NewArchiveReader returns a new archive data wrapper for the data in the provided reader. Note that the entire archive will be read into memory and kept there. So for relatively large archives it will often be more appropriate to use ArchiveWorkspace instead.

func (*ArchiveData) Metadata 

func (ad *ArchiveData) Metadata() (*Metadata, error)

Metadata returns the metadata stored in the backup archive. If no metadata is there, errors.NotFound is returned.

func (*ArchiveData) NewBuffer 

func (ad *ArchiveData) NewBuffer() *bytes.Buffer

NewBuffer wraps the archive data in a Buffer.

func (*ArchiveData) Version 

func (ad *ArchiveData) Version() (*version.Number, error)

Version returns the juju version under which the backup archive was created. If no version is found in the archive, it must come from before backup archives included the version. In that case we return version 1.20.

type ArchivePaths 

type ArchivePaths struct {
	// ContentDir is the path to the directory within the archive
	// containing all the contents. It is the only file or directory at
	// the top-level of the archive and everything else in the archive
	// is contained in the content directory.
	ContentDir string

	// FilesBundle is the path to the tar file inside the archive
	// containing all the state-related files (with the exception of the
	// DB dump files) gathered in by the backup machinery.
	FilesBundle string

	// DBDumpDir is the path to the directory within the archive
	// contents that contains all the files dumped from the juju state
	// database.
	DBDumpDir string

	// MetadataFile is the path to the metadata file.
	MetadataFile string
}

ArchivePaths holds the paths to the files and directories in a backup archive.

func NewCanonicalArchivePaths 

func NewCanonicalArchivePaths() ArchivePaths

NewCanonicalArchivePaths composes a new ArchivePaths with default values set. These values are relative (un-rooted) and the canonical slash ("/") is the path separator. Thus the paths are suitable for resolving the paths in a backup archive file (which is a tar file).

func NewNonCanonicalArchivePaths 

func NewNonCanonicalArchivePaths(rootDir string) ArchivePaths

NonCanonicalArchivePaths builds a new ArchivePaths using default values, rooted at the provided rootDir. The path separator used is platform-dependent. The resulting paths are suitable for locating backup archive contents in a directory into which an archive has been unpacked.

type ArchiveWorkspace 

type ArchiveWorkspace struct {
	ArchivePaths
	RootDir string
}

ArchiveWorkspace is a wrapper around backup archive info that has a concrete root directory and an archive unpacked in it.

func NewArchiveWorkspaceReader 

func NewArchiveWorkspaceReader(archive io.Reader) (*ArchiveWorkspace, error)

NewArchiveWorkspaceReader returns a new archive workspace with a new workspace dir populated from the archive. Note that this involves unpacking the entire archive into a directory under the host's "temporary" directory. For relatively large archives this could have adverse effects on hosts with little disk space.

func (*ArchiveWorkspace) Close 

func (ws *ArchiveWorkspace) Close() error

Close cleans up the workspace dir.

func (*ArchiveWorkspace) Metadata 

func (ws *ArchiveWorkspace) Metadata() (*Metadata, error)

Metadata returns the metadata derived from the JSON file in the archive.

func (*ArchiveWorkspace) OpenBundledFile 

func (ws *ArchiveWorkspace) OpenBundledFile(filename string) (io.Reader, error)

OpenBundledFile returns an open ReadCloser for the corresponding file in the archived files bundle.

func (*ArchiveWorkspace) UnpackFilesBundle 

func (ws *ArchiveWorkspace) UnpackFilesBundle(targetRoot string) error

UnpackFilesBundle unpacks the archived files bundle into the targeted dir.

type Backups 

type Backups interface {
	// Create creates a new juju backup archive. It updates
	// the provided metadata.
	Create(meta *Metadata, paths *Paths, dbInfo *DBInfo, keepCopy, noDownload bool) (string, error)

	// Add stores the backup archive and returns its new ID.
	Add(archive io.Reader, meta *Metadata) (string, error)

	// Get returns the metadata and archive file associated with the ID.
	Get(id string) (*Metadata, io.ReadCloser, error)

	// List returns the metadata for all stored backups.
	List() ([]*Metadata, error)

	// Remove deletes the backup from storage.
	Remove(id string) error
}

Backups is an abstraction around all juju backup-related functionality.

func NewBackups 

func NewBackups(stor filestorage.FileStorage) Backups

NewBackups creates a new Backups value using the FileStorage provided.

type ControllerMetadata 

type ControllerMetadata struct {
	// UUID contains the controller UUID.
	UUID string

	// MachineID contains controller machine id from which this backup is taken.
	MachineID string

	// MachineInstanceID contains controller machine's instance id from which this backup is taken.
	MachineInstanceID string

	// HANodes contains the number of nodes in this controller's HA configuration.
	HANodes int64
}

ControllerMetadata contains controller specific metadata.

func UnknownController 

func UnknownController() ControllerMetadata

UnknownController returns a new backups origin with unknown values.

type DB 

type DB interface {
	// MongoSession returns the underlying mongodb session.
	MongoSession() *mgo.Session

	// ModelTag is the concrete model tag for this database.
	ModelTag() names.ModelTag

	// ModelConfig is the config of the model being backedup.
	ModelConfig() (*config.Config, error)

	// ControllerConfig is the config of the controller being backedup.
	ControllerConfig() (controller.Config, error)

	// StateServingInfo is the secrets of the controller.
	StateServingInfo() (controller.StateServingInfo, error)
}

DB represents the set of methods required to perform a backup. It exists to break the strict dependency between state and this package, and those that depend on this package.

type DBDumper 

type DBDumper interface {
	// Dump something to dumpDir.
	Dump(dumpDir string) error
}

DBDumper is any type that dumps something to a dump dir.

func NewDBDumper 

func NewDBDumper(info *DBInfo) (DBDumper, error)

NewDBDumper returns a new value with a Dump method for dumping the juju state database.

type DBInfo 

type DBInfo struct {
	// Address is the DB system's host address.
	Address string
	// Username is used when connecting to the DB system.
	Username string
	// Password is used when connecting to the DB system.
	Password string
	// Targets is a list of databases to dump.
	Targets set.Strings
}

DBInfo wraps all the DB-specific information backups needs to dump the database. This includes a simplification of the information in authentication.MongoInfo.

func NewDBInfo 

func NewDBInfo(mgoInfo *mongo.MongoInfo, session DBSession) (*DBInfo, error)

NewDBInfo returns the information needed by backups to dump the database.

type DBSession 

type DBSession interface {
	DatabaseNames() ([]string, error)
}

type Metadata 

type Metadata struct {
	*filestorage.FileMetadata

	// Started records when the backup was started.
	Started time.Time

	// Finished records when the backup was complete.
	Finished *time.Time

	// Origin identifies where the backup was created.
	Origin Origin

	// Notes is an optional user-supplied annotation.
	Notes string

	// FormatVersion stores format version of these metadata.
	FormatVersion int64

	// Controller contains metadata about the controller where the backup was taken.
	Controller ControllerMetadata

	// CACert is the controller CA certificate.
	CACert string

	// CAPrivateKey is the controller CA private key.
	CAPrivateKey string
}

Metadata contains the metadata for a single state backup archive.

func BuildMetadata 

func BuildMetadata(file *os.File) (*Metadata, error)

BuildMetadata generates the metadata for a backup archive file.

func NewMetadata 

func NewMetadata() *Metadata

NewMetadata returns a new Metadata for a state backup archive, in the most current format.

func NewMetadataJSONReader 

func NewMetadataJSONReader(in io.Reader) (*Metadata, error)

NewMetadataJSONReader extracts a new metadata from the JSON file.

func NewMetadataState 

func NewMetadataState(db DB, machine, series string) (*Metadata, error)

NewMetadataState composes a new backup metadata based on the current Juju state.

func (*Metadata) AsJSONBuffer 

func (m *Metadata) AsJSONBuffer() (io.Reader, error)

AsJSONBuffer returns a bytes.Buffer containing the JSON-ified metadata. This will always produce latest known format.

func (*Metadata) MarkComplete 

func (m *Metadata) MarkComplete(size int64, checksum string) error

MarkComplete populates the remaining metadata values. The default checksum format is used.

type MongoDB 

type MongoDB interface {
	UpsertUser(*mgo.User) error
}

MongoDB represents a mgo.DB.

type MongoSession 

type MongoSession interface {
	Run(cmd interface{}, result interface{}) error
	Close()
	DB(string) *mgo.Database
}

MongoSession represents mgo.Session.

type Origin 

type Origin struct {
	Model    string
	Machine  string
	Hostname string
	Version  version.Number
	Series   string
}

Origin identifies where a backup archive came from. While it is more about where and Metadata about what and when, that distinction does not merit special consideration. Instead, Origin exists separately from Metadata due to its use as an argument when requesting the creation of a new backup.

func UnknownOrigin 

func UnknownOrigin() Origin

UnknownOrigin returns a new backups origin with unknown values.

type Paths 

type Paths struct {
	BackupDir string
	DataDir   string
	LogsDir   string
}

Paths holds the paths that backups needs.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.