backups

package
v0.0.0-...-8ff1004 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 15, 2019 License: AGPL-3.0 Imports: 50 Imported by: 0

Documentation

Overview

Package backups contains all the stand-alone backup-related functionality for juju state. That functionality is encapsulated by the backups.Backups type. The package also exposes a few key helpers and components.

Backups are not a part of juju state nor of normal state operations. However, they certainly are tightly coupled with state (the very subject of backups). This puts backups in an odd position, particularly with regard to the storage of backup metadata and archives.

As noted above backups are about state but not a part of state. So exposing backup-related methods on State would imply the wrong thing. Thus most of the functionality here is defined at a high level without relation to state. A few low-level parts or helpers are exposed as functions to which you pass a state value. Those are kept to a minimum.

Note that state (and juju as a whole) currently does not have a persistence layer abstraction to facilitate separating different persistence needs and implementations. As a consequence, state's data, whether about how an model should look or about existing resources within an model, is dumped essentially straight into State's mongo connection. The code in the state package does not make any distinction between the two (nor does the package clearly distinguish between state-related abstractions and state-related data).

Backups add yet another category, merely taking advantage of State's mongo for storage. In the interest of making the distinction clear, among other reasons, backups uses its own database under state's mongo connection.

Index

Constants

View Source 
const (
	// FilenamePrefix is the prefix used for backup archive files.
	FilenamePrefix = "juju-backup-"

	// FilenameTemplate is used with time.Time.Format to generate a filename.
	FilenameTemplate = FilenamePrefix + "20060102-150405.tar.gz"
)
View Source 
const (
	TempFilename = "juju-backup.tar.gz"
)
View Source 
const UnknownString = "<unknown>"

UnknownString is a marker value for string fields with unknown values.

Variables

View Source 
var UnknownVersion = version.MustParse("9999.9999.9999")

UnknownVersion is a marker value for version fields with unknown values.

Functions

func GetFilesToBackUp 

func GetFilesToBackUp(rootDir string, paths *Paths, oldmachine string) ([]string, error)

GetFilesToBackUp returns the paths that should be included in the backup archive.

func NewStorage 

func NewStorage(st DB) filestorage.FileStorage

NewStorage returns a new FileStorage to use for storing backup archives (and metadata).

func PrepareMachineForRestore 

func PrepareMachineForRestore(mongoVersion mongo.Version) error

PrepareMachineForRestore deletes all files from the re-bootstrapped machine that are to be replaced by the backup and recreates those directories that are to contain new files; this is to avoid possible mixup from new/old files that lead to an inconsistent restored state machine.

func StoreArchive 

func StoreArchive(stor filestorage.FileStorage, meta *Metadata, file io.Reader) error

StoreArchive sends the backup archive and its metadata to storage. It also sets the metadata's ID and Stored values.

Types

type ArchiveData 

type ArchiveData struct {
	ArchivePaths
	// contains filtered or unexported fields
}

ArchiveData is a wrapper around a the uncompressed data in a backup archive file. It provides access to the content of the archive. While ArchiveData provides useful functionality, it may not be appropriate for large archives. The contents of the archive are kept in-memory, so large archives could be too taxing on the host. In that case consider using ArchiveWorkspace instead.

func NewArchiveData 

func NewArchiveData(data []byte) *ArchiveData

NewArchiveData builds a new archive data wrapper for the given uncompressed data.

func NewArchiveDataReader 

func NewArchiveDataReader(r io.Reader) (*ArchiveData, error)

NewArchiveReader returns a new archive data wrapper for the data in the provided reader. Note that the entire archive will be read into memory and kept there. So for relatively large archives it will often be more appropriate to use ArchiveWorkspace instead.

func (*ArchiveData) Metadata 

func (ad *ArchiveData) Metadata() (*Metadata, error)

Metadata returns the metadata stored in the backup archive. If no metadata is there, errors.NotFound is returned.

func (*ArchiveData) NewBuffer 

func (ad *ArchiveData) NewBuffer() *bytes.Buffer

NewBuffer wraps the archive data in a Buffer.

func (*ArchiveData) Version 

func (ad *ArchiveData) Version() (*version.Number, error)

Version returns the juju version under which the backup archive was created. If no version is found in the archive, it must come from before backup archives included the version. In that case we return version 1.20.

type ArchivePaths 

type ArchivePaths struct {
	// ContentDir is the path to the directory within the archive
	// containing all the contents. It is the only file or directory at
	// the top-level of the archive and everything else in the archive
	// is contained in the content directory.
	ContentDir string

	// FilesBundle is the path to the tar file inside the archive
	// containing all the state-related files (with the exception of the
	// DB dump files) gathered in by the backup machinery.
	FilesBundle string

	// DBDumpDir is the path to the directory within the archive
	// contents that contains all the files dumped from the juju state
	// database.
	DBDumpDir string

	// MetadataFile is the path to the metadata file.
	MetadataFile string
}

ArchivePaths holds the paths to the files and directories in a backup archive.

func NewCanonicalArchivePaths 

func NewCanonicalArchivePaths() ArchivePaths

NewCanonicalArchivePaths composes a new ArchivePaths with default values set. These values are relative (un-rooted) and the canonical slash ("/") is the path separator. Thus the paths are suitable for resolving the paths in a backup archive file (which is a tar file).

func NewNonCanonicalArchivePaths 

func NewNonCanonicalArchivePaths(rootDir string) ArchivePaths

NonCanonicalArchivePaths builds a new ArchivePaths using default values, rooted at the provided rootDir. The path separator used is platform-dependent. The resulting paths are suitable for locating backup archive contents in a directory into which an archive has been unpacked.

type ArchiveWorkspace 

type ArchiveWorkspace struct {
	ArchivePaths
	RootDir string
}

ArchiveWorkspace is a wrapper around backup archive info that has a concrete root directory and an archive unpacked in it.

func NewArchiveWorkspaceReader 

func NewArchiveWorkspaceReader(archive io.Reader) (*ArchiveWorkspace, error)

NewArchiveWorkspaceReader returns a new archive workspace with a new workspace dir populated from the archive. Note that this involves unpacking the entire archive into a directory under the host's "temporary" directory. For relatively large archives this could have adverse effects on hosts with little disk space.

func (*ArchiveWorkspace) Close 

func (ws *ArchiveWorkspace) Close() error

Close cleans up the workspace dir.

func (*ArchiveWorkspace) Metadata 

func (ws *ArchiveWorkspace) Metadata() (*Metadata, error)

Metadata returns the metadata derived from the JSON file in the archive.

func (*ArchiveWorkspace) OpenBundledFile 

func (ws *ArchiveWorkspace) OpenBundledFile(filename string) (io.Reader, error)

OpenBundledFile returns an open ReadCloser for the corresponding file in the archived files bundle.

func (*ArchiveWorkspace) UnpackFilesBundle 

func (ws *ArchiveWorkspace) UnpackFilesBundle(targetRoot string) error

UnpackFilesBundle unpacks the archived files bundle into the targeted dir.

type Backups 

type Backups interface {
	// Create creates a new juju backup archive. It updates
	// the provided metadata.
	Create(meta *Metadata, paths *Paths, dbInfo *DBInfo, keepCopy, noDownload bool) (string, error)

	// Add stores the backup archive and returns its new ID.
	Add(archive io.Reader, meta *Metadata) (string, error)

	// Get returns the metadata and archive file associated with the ID.
	Get(id string) (*Metadata, io.ReadCloser, error)

	// List returns the metadata for all stored backups.
	List() ([]*Metadata, error)

	// Remove deletes the backup from storage.
	Remove(id string) error

	// Restore updates juju's state to the contents of the backup archive,
	// it returns the tag string for the machine where the backup originated
	// or error if the process fails.
	Restore(backupId string, args RestoreArgs) (names.Tag, error)
}

Backups is an abstraction around all juju backup-related functionality.

func NewBackups 

func NewBackups(stor filestorage.FileStorage) Backups

NewBackups creates a new Backups value using the FileStorage provided.

type DB 

type DB interface {
	// MongoSession returns the underlying mongodb session.
	MongoSession() *mgo.Session

	// ModelTag is the concrete model tag for this database.
	ModelTag() names.ModelTag

	// ModelConfig is the config of the model being backedup.
	ModelConfig() (*config.Config, error)

	// ControllerConfig is the config of the controller being backedup.
	ControllerConfig() (controller.Config, error)

	// StateServingInfo is the secrets of the controller.
	StateServingInfo() (state.StateServingInfo, error)
}

DB represents the set of methods required to perform a backup. It exists to break the strict dependency between state and this package, and those that depend on this package.

type DBDumper 

type DBDumper interface {
	// Dump something to dumpDir.
	Dump(dumpDir string) error
}

DBDumper is any type that dumps something to a dump dir.

func NewDBDumper 

func NewDBDumper(info *DBInfo) (DBDumper, error)

NewDBDumper returns a new value with a Dump method for dumping the juju state database.

type DBInfo 

type DBInfo struct {
	// Address is the DB system's host address.
	Address string
	// Username is used when connecting to the DB system.
	Username string
	// Password is used when connecting to the DB system.
	Password string
	// Targets is a list of databases to dump.
	Targets set.Strings
	// MongoVersion the version of the running mongo db.
	MongoVersion mongo.Version
}

DBInfo wraps all the DB-specific information backups needs to dump the database. This includes a simplification of the information in authentication.MongoInfo.

func NewDBInfo 

func NewDBInfo(mgoInfo *mongo.MongoInfo, session DBSession, version mongo.Version) (*DBInfo, error)

NewDBInfo returns the information needed by backups to dump the database.

type DBRestorer 

type DBRestorer interface {
	// Dump something to dumpDir.
	Restore(dumpDir string, dialInfo *mgo.DialInfo) error
}

DBDumper is any type that dumps something to a dump dir.

func NewDBRestorer 

func NewDBRestorer(args RestorerArgs) (DBRestorer, error)

NewDBRestorer returns a new structure that can perform a restore on the db pointed in dialInfo.

type DBSession 

type DBSession interface {
	DatabaseNames() ([]string, error)
}

type Metadata 

type Metadata struct {
	*filestorage.FileMetadata

	// Started records when the backup was started.
	Started time.Time

	// Finished records when the backup was complete.
	Finished *time.Time

	// Origin identifies where the backup was created.
	Origin Origin

	// Notes is an optional user-supplied annotation.
	Notes string

	// CACert is the controller CA certificate.
	CACert string

	// CAPrivateKey is the controller CA private key.
	CAPrivateKey string
}

Metadata contains the metadata for a single state backup archive.

func BuildMetadata 

func BuildMetadata(file *os.File) (*Metadata, error)

BuildMetadata generates the metadata for a backup archive file.

func NewMetadata 

func NewMetadata() *Metadata

NewMetadata returns a new Metadata for a state backup archive. Only the start time and the version are set.

func NewMetadataJSONReader 

func NewMetadataJSONReader(in io.Reader) (*Metadata, error)

NewMetadataJSONReader extracts a new metadata from the JSON file.

func NewMetadataState 

func NewMetadataState(db DB, machine, series string) (*Metadata, error)

NewMetadataState composes a new backup metadata with its origin values set. The model UUID comes from state. The hostname is retrieved from the OS.

func (*Metadata) AsJSONBuffer 

func (m *Metadata) AsJSONBuffer() (io.Reader, error)

AsJSONBuffer returns a bytes.Buffer containing the JSON-ified metadata.

func (*Metadata) MarkComplete 

func (m *Metadata) MarkComplete(size int64, checksum string) error

MarkComplete populates the remaining metadata values. The default checksum format is used.

type MongoDB 

type MongoDB interface {
	UpsertUser(*mgo.User) error
}

MongoDB represents a mgo.DB.

func GetDB 

func GetDB(s string, session MongoSession) MongoDB

GetDB wraps mgo.Session.DB to ease testing.

type MongoSession 

type MongoSession interface {
	Run(cmd interface{}, result interface{}) error
	Close()
	DB(string) *mgo.Database
}

MongoSession represents mgo.Session.

func NewMongoSession 

func NewMongoSession(dialInfo *mgo.DialInfo) (MongoSession, error)

NewMongoSession wraps mgo.DialInfo to ease testing.

type Origin 

type Origin struct {
	Model    string
	Machine  string
	Hostname string
	Version  version.Number
	Series   string
}

Origin identifies where a backup archive came from. While it is more about where and Metadata about what and when, that distinction does not merit special consideration. Instead, Origin exists separately from Metadata due to its use as an argument when requesting the creation of a new backup.

func UnknownOrigin 

func UnknownOrigin() Origin

UnknownOrigin returns a new backups origin with unknown values.

type Paths 

type Paths struct {
	BackupDir string
	DataDir   string
	LogsDir   string
}

Paths holds the paths that backups needs.

type RestoreArgs 

type RestoreArgs struct {
	PrivateAddress string
	PublicAddress  string
	NewInstId      instance.Id
	NewInstTag     names.Tag
	NewInstSeries  string
}

RestoreArgs holds the args to be used to call state/backups.Restore

type RestorerArgs 

type RestorerArgs struct {
	DialInfo        *mgo.DialInfo
	NewMongoSession func(*mgo.DialInfo) (MongoSession, error)
	Version         mongo.Version
	TagUser         string
	TagUserPassword string
	GetDB           func(string, MongoSession) MongoDB

	RunCommandFn func(string, ...string) error
	StartMongo   func() error
	StopMongo    func() error
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.