archive

Documentation

Overview

Package archive defines an interface for storage engines that have no understanding of memorybox. It also provides higher level crud functions for interacting with any storage engine in a memorybox specific way.

Index

• func Delete(ctx context.Context, store Store, name string) error
• func Diff(ctx context.Context, source Store, dest Store) error
• func GetDataByPrefix(ctx context.Context, store Store, prefix string) (*file.File, error)
• func GetMetaByPrefix(ctx context.Context, store Store, prefix string) (*file.File, error)
• func Import(ctx context.Context, logger *Logger, store Store, concurrency int, set string, ...) error
• func Index(ctx context.Context, store Store, concurrency int) ([][]byte, error)
• func IndexUpdate(ctx context.Context, logger *Logger, store Store, concurrency int, ...) error
• func Put(ctx context.Context, store Store, f *file.File, set string) (*file.File, error)
• func Sync(ctx context.Context, logger *Logger, source Store, dest Store, mode string, ...) error
• type CheckItem
  • func (ci CheckItem) String() string
• type CheckOutput
  • func Check(ctx context.Context, store Store, concurrency int, mode string) (*CheckOutput, error)
  • func (co CheckOutput) String() string
• type Logger
• type Store

Functions

func Delete

func Delete(ctx context.Context, store Store, name string) error

Delete removes a datafile/metafile pair for any backing store.

func Diff

func Diff(ctx context.Context, source Store, dest Store) error

Diff shows the differences between two stores.

func GetDataByPrefix

func GetDataByPrefix(ctx context.Context, store Store, prefix string) (*file.File, error)

GetDataByPrefix retrieves a datafile from any backing store as long as there is only one match.

func GetMetaByPrefix

func GetMetaByPrefix(ctx context.Context, store Store, prefix string) (*file.File, error)

GetMetaByPrefix retrieves a metafile from any backing store as long as there is only one match.

func Import

func Import(ctx context.Context, logger *Logger, store Store, concurrency int, set string, data io.Reader) error

Import performs a mass put / annotation operation on any number of manifest files, formatted like so: ``` path/to/file.jpg {"title":"some file on my machine"} https://images.com/photo.jpg {"title":"some photo on the internet"} https://audio.com/audio.mp3 {"title":"some mp3 on the internet"} ``` Import will intelligently de-dupe manifests. It will also remove entries that already appear in the store (by checking every import line against every metafile `memorybox.import.source` key in the store).

func Index

func Index(ctx context.Context, store Store, concurrency int) ([][]byte, error)

Index concats all metafiles in the provided store, one per line.

func IndexUpdate

func IndexUpdate(ctx context.Context, logger *Logger, store Store, concurrency int, updates io.Reader) error

IndexUpdate reads a provided reader line by line where each line is expected to be the content of a metafile. The data within is persisted to the store

func Put

func Put(ctx context.Context, store Store, f *file.File, set string) (*file.File, error)

Put persists a datafile/metafile pair for any backing store and returns the meta information about the file.

func Sync

func Sync(ctx context.Context, logger *Logger, source Store, dest Store, mode string, concurrency int) error

Sync converges the content of two provided stores so they are identical.

Types

type CheckItem

type CheckItem struct {
	Name      string
	Count     int
	Signature string
	Source    string
}

func (CheckItem) String

func (ci CheckItem) String() string

type CheckOutput

type CheckOutput struct {
	Items   []CheckItem
	Details []string
}

func Check

func Check(ctx context.Context, store Store, concurrency int, mode string) (*CheckOutput, error)

func (CheckOutput) String

func (co CheckOutput) String() string

type Logger

type Logger struct {
	Stdout  *log.Logger
	Stderr  *log.Logger
	Verbose *log.Logger
}

Logger defines output streams for interacting with archives.

type Store

type Store interface {
	Get(context.Context, string) (*file.File, error)
	Put(context.Context, io.Reader, string, time.Time) error
	Delete(context.Context, string) error
	Search(context.Context, string) (file.List, error)
	Concat(context.Context, int, []string) ([][]byte, error)
	Stat(context.Context, string) (*file.File, error)
	String() string
}

Store defines a storage engine that can persist and retrieve content.