archiver

Documentation

Overview

Package archiver zips lots of little blobs into bigger zip files and stores them somewhere. While generic, it was designed to incrementally create Amazon Glacier archives from many little blobs, rather than creating millions of Glacier archives.

Index

• Constants
• Variables
• type Archiver
  • func (a *Archiver) RunOnce() error

Constants

const DefaultMinZipSize = 16 << 20

DefaultMinZipSize is the default value of Archiver.MinZipSize.

Variables

var ErrSourceTooSmall = errors.New("archiver: not enough blob data on source to warrant a new zip archive")

ErrSourceTooSmall is returned by RunOnce if there aren't enough blobs on Source to warrant a new zip archive.

Types

type Archiver

type Archiver struct {
	// Source is where the blobs should come from.
	// (and be deleted from, if DeleteSourceAfterStore)
	Source blobserver.Storage

	// MinZipSize is the minimum size of zip files to create.
	// If zero, DefaultMinZipSize is used.
	MinZipSize int64

	// Store specifies a function that writes the zip file
	// (encoded in the byte slice) to permanent storage
	// (e.g. Amazon Glacier) and notes somewhere (a database) that
	// it contains the listed blobs. The blobs are redundant with
	// the filenames in the zip file, which will be named by
	// their blobref string, with no extension.
	Store func(zip []byte, blobs []blob.SizedRef) error

	// DeleteSourceAfterStore, if true, deletes the blobs from Source
	// after Store returns success.
	// This should pretty much always be set true, otherwise subsequent
	// calls to Run/RunOnce will generate the same archives. Wrap
	// the Source in a "namespace" storage if you don't actually
	// want to delete things locally.
	DeleteSourceAfterStore bool
}

An Archiver specifies the parameters of the job that copies from one blobserver Storage (the Source) to long-term storage.

func (*Archiver) RunOnce

func (a *Archiver) RunOnce() error

RunOnce scans a.Source and conditionally creates a new zip. It returns ErrSourceTooSmall if there aren't enough blobs on Source.