backups

Documentation

Index

• Constants
• Variables
• func CopyLevelDB(src, dst *leveldb.DB) (ncopied uint64, err error)
• type Backup
• type Buffer
  • func (Buffer) Close() error
• type Config
  • func (c Config) ArchiveName() string
  • func (c Config) Storage() (_ Storage, err error)
• type Discard
  • func (Discard) Close() error
  • func (Discard) Write(p []byte) (int, error)
• type FileStorage
  • func NewFileStorage(root, prefix string) (_ *FileStorage, err error)
  • func (s *FileStorage) ListArchives() (paths []string, err error)
  • func (s *FileStorage) Open(name string) (_ io.WriteCloser, err error)
  • func (s *FileStorage) Remove(name string) error
• type LevelDB
  • func (l *LevelDB) Backup(tmpdir string) (err error)
• type Manager
  • func New(conf Config, backup Backup) *Manager
  • func (m *Manager) MkdirTemp() (path string, err error)
  • func (m *Manager) Run() (err error)
  • func (m *Manager) Shutdown() error
• type MemoryStorage
  • func (s *MemoryStorage) Backup(name string) (*Buffer, bool)
  • func (s *MemoryStorage) ListArchives() ([]string, error)
  • func (s *MemoryStorage) Open(name string) (io.WriteCloser, error)
  • func (s *MemoryStorage) Remove(name string) error
• type SQLite3
  • func (s *SQLite3) Backup(tmpdir string) (err error)
  • func (s *SQLite3) OpenDestDB(tmpdir string) (db *sql.DB, conn *sqlite.Conn, err error)
• type Storage

Constants

const (
	SQLite3Main   = "main"
	PagesPerStep  = 5
	StepSleep     = 50 * time.Millisecond
	DefaultDBName = "backup.db"
)

Variables

var (
	ErrNotEnabled        = errors.New("the backup manager is not enabled")
	ErrTmpDirUnavailable = errors.New("cannot create temporary directories for backup")
	ErrInvalidStorageDSN = errors.New("could not parse storage dsn, specify scheme:///relative/path/")
	ErrNotADirectory     = errors.New("incorrectly configured: backup storage is not a directory")
	ErrNilSQLite3Conn    = errors.New("could not fetch underlying sqlite3 connection for backup")
)

Functions

func CopyLevelDB

func CopyLevelDB(src, dst *leveldb.DB) (ncopied uint64, err error)

Types

type Backup

type Backup interface {
	Backup(tmpdir string) error
}

Backup is an interface that enables different types of database backups, e.g. for backuping up a leveldb database (or multiple leveldb databases) vs a sqlite database. It is expected that the backup is written to the temporary directory specified by the input variable. It is unnecessary to compress the contents of the backup as the backup manager will compress the contents before transferring them to the backup storage location. Once the backup is complete (error or not) the tmpdir is removed.

type Buffer

type Buffer struct {
	bytes.Buffer
}

func (Buffer) Close

func (Buffer) Close() error

type Config

type Config struct {
	// If false, the backup manager will not run.
	Enabled bool `default:"false"`

	// The interval between backups.
	Interval time.Duration `default:"24h"`

	// The path to a local disk directory to store compressed backups, e.g.
	// file:///rel/path/ or a cloud location such as s3://bucket.
	StorageDSN string `split_words:"true" required:"true"`

	// Temporary directory to perform local backup to. If not set, then the OS tmpdir
	// is used. Backups are generated in this folder and then moved to the storage
	// location. This folder needs enough space to contain the complete backup.
	TempDir string `required:"false"`

	// Prefix specifies the filename of the backup, e.g.g prefix-200601021504.tgz
	Prefix string `default:"backup"`

	// The number of previous backup versions to keep.
	Keep int `default:"1"`
}

Configure the backup manager routine to ensure it is backing up the correct databases at the correct interval; removing old backups as necessary and storing the backups in the correct storage location.

func (Config) ArchiveName

func (c Config) ArchiveName() string

ArchiveName returns the name of the next archive using the current timestamp.

func (Config) Storage

func (c Config) Storage() (_ Storage, err error)

Storage returns the storage configuration specified by the storage DSN.

type Discard

type Discard struct{}

func (Discard) Close

func (Discard) Close() error

func (Discard) Write

func (Discard) Write(p []byte) (int, error)

type FileStorage

type FileStorage struct {
	// contains filtered or unexported fields
}

FileStorage stores archived backups on a local disk. This is primarily used by mounting a second volume on a Kubernetes pod and saving the backups there or by writing to a network or RAID protected dis.

func NewFileStorage

func NewFileStorage(root, prefix string) (_ *FileStorage, err error)

NewFileStorage creates the specified root directory if it does not exist. All storage archives will be opened relative to the root directory.

func (*FileStorage) ListArchives

func (s *FileStorage) ListArchives() (paths []string, err error)

ListArchives returns all backup archives in the FileStorage directory ordered by date ascending using string sorting that depends on the backup archive name format: prefix-YYYYmmddHHMM.tgz

func (*FileStorage) Open

func (s *FileStorage) Open(name string) (_ io.WriteCloser, err error)

Open a file on disk and return the file for writting a gzip stream to.

func (*FileStorage) Remove

func (s *FileStorage) Remove(name string) error

Remove a file from the backup directory.

type LevelDB

type LevelDB struct {
	DB *leveldb.DB
}

LevelDB implements a single leveldb backup that takes a snapshot of the database and writes it to a second leveldb database at the temporary directory location.

func (*LevelDB) Backup

func (l *LevelDB) Backup(tmpdir string) (err error)

Backup executes the leveldb backup strategy.

type Manager

type Manager struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Manager runs as an independent service which periodically backs up a database to a compressed backup location on disk or to cloud storage. At a specified interval, the manager runs a db-specific backup routine that generally clones the database to a tmp directory; it then compresses the clone and moves it to a final backup location.

func New

func New(conf Config, backup Backup) *Manager

Return a new backup manager ready to be run. NOTE: the backup manager is not started on new; it must explicitly be run for the backup routine to be effective. This follows the startup/shutdown services model.

func (*Manager) MkdirTemp

func (m *Manager) MkdirTemp() (path string, err error)

Create a temporary direcory in the configured path. If no configured directory is specified then os.MkdirTemp is used. It is the callers responsibility to cleanup the directory that was created.

func (*Manager) Run

func (m *Manager) Run() (err error)

Run the main backup manager routine which periodically wakes up and creates a backup of the specified database. The backup manager can be started and stopped as necessary.

func (*Manager) Shutdown

func (m *Manager) Shutdown() error

type MemoryStorage

type MemoryStorage struct {
	// contains filtered or unexported fields
}

Memory storage is primarily used for testing and writes backups to a bytes buffer. If /dev/null is specified as the "root" path, then a noop-closer is returned.

func (*MemoryStorage) Backup

func (s *MemoryStorage) Backup(name string) (*Buffer, bool)

func (*MemoryStorage) ListArchives

func (s *MemoryStorage) ListArchives() ([]string, error)

func (*MemoryStorage) Open

func (s *MemoryStorage) Open(name string) (io.WriteCloser, error)

func (*MemoryStorage) Remove

func (s *MemoryStorage) Remove(name string) error

type SQLite3

type SQLite3 struct {
	DB   *sqlite.Conn
	Name string
}

SQLite3 implements a single sqlite3 backup that uses the online backup of a running database mechanism to copy the db over to a second sqlite3 database at the temporary directory location.

func (*SQLite3) Backup

func (s *SQLite3) Backup(tmpdir string) (err error)

Backup executes the sqlite3 backup strategy.

func (*SQLite3) OpenDestDB

func (s *SQLite3) OpenDestDB(tmpdir string) (db *sql.DB, conn *sqlite.Conn, err error)

type Storage

type Storage interface {
	Open(name string) (io.WriteCloser, error)
	Remove(name string) error
	ListArchives() ([]string, error)
}

Storage provides an interface for reading and writing compressed backups to disk.