local

Documentation

Overview

Package local is a persistent local storage backend for Shade.

It stores files and chunks locally to disk. You may define full filepaths to store the files and chunks in the config, or via flag. If you define neither, the flags will choose sensible defaults for your operating system.

Index

• func NewClient(c drive.Config) (drive.Client, error)
• type Chunk
  • func (a Chunk) Less(bt btree.Item) bool
• type ChunkLister
  • func (c *ChunkLister) Err() error
  • func (c *ChunkLister) Next() bool
  • func (c *ChunkLister) Sha256() []byte
• type Drive
  • func (s *Drive) GetChunk(sha256sum []byte, f *shade.File) ([]byte, error)
  • func (s *Drive) GetConfig() drive.Config
  • func (s *Drive) GetFile(sha256sum []byte) ([]byte, error)
  • func (s *Drive) ListFiles() ([][]byte, error)
  • func (s *Drive) Local() bool
  • func (s *Drive) NewChunkLister() drive.ChunkLister
  • func (s *Drive) Persistent() bool
  • func (s *Drive) PutChunk(sha256sum []byte, data []byte, f *shade.File) error
  • func (s *Drive) PutFile(sha256sum, data []byte) error
  • func (s *Drive) ReleaseChunk(sha256sum []byte) error
  • func (s *Drive) ReleaseFile(sha256sum []byte) error
  • func (s *Drive) Warm(chunks [][]byte, f *shade.File)

Functions

func NewClient

func NewClient(c drive.Config) (drive.Client, error)

NewClient returns a fully initlized local client.

Types

type Chunk

type Chunk struct {
	// contains filtered or unexported fields
}

Chunk describes an object cached to the filesystem, in a way that the btree implementaiton can sort it. This allows garbage collection by mtime.

func (Chunk) Less

func (a Chunk) Less(bt btree.Item) bool

Less ultimately describes the order chunks are deleted in.

type ChunkLister

type ChunkLister struct {
	// contains filtered or unexported fields
}

ChunkLister allows iterating the chunks stored on disk.

func (*ChunkLister) Err

func (c *ChunkLister) Err() error

Err returns precisely no errors.

func (*ChunkLister) Next

func (c *ChunkLister) Next() bool

Next increments the pointer.

func (*ChunkLister) Sha256

func (c *ChunkLister) Sha256() []byte

Sha256 returns the chunk pointed to by the pointer.

type Drive

type Drive struct {
	sync.RWMutex // serializes accesses to the directories on local disk
	// contains filtered or unexported fields
}

Drive implements the drive.Client interface by storing Files and Chunks to the local filesystem. It treats the ChunkParentID and FileParentID as filepaths to the directory to store data in.

func (*Drive) GetChunk

func (s *Drive) GetChunk(sha256sum []byte, f *shade.File) ([]byte, error)

GetChunk retrieves a chunk with a given SHA-256 sum

func (*Drive) GetConfig

func (s *Drive) GetConfig() drive.Config

GetConfig returns the config used to initialize this client.

func (*Drive) GetFile

func (s *Drive) GetFile(sha256sum []byte) ([]byte, error)

GetFile retrieves a chunk with a given SHA-256 sum

func (*Drive) ListFiles

func (s *Drive) ListFiles() ([][]byte, error)

ListFiles retrieves all of the File objects known to the client. The return values are the sha256sum of the file object. The keys may be passed to GetChunk() to retrieve the corresponding shade.File.

func (*Drive) Local

func (s *Drive) Local() bool

Local returns whether the storage is local to this machine.

func (*Drive) NewChunkLister

func (s *Drive) NewChunkLister() drive.ChunkLister

NewChunkLister returns an iterator which lists the chunks stored on disk.

func (*Drive) Persistent

func (s *Drive) Persistent() bool

Persistent returns whether the storage is persistent across task restarts.

func (*Drive) PutChunk

func (s *Drive) PutChunk(sha256sum []byte, data []byte, f *shade.File) error

PutChunk writes a chunk to local disk

func (*Drive) PutFile

func (s *Drive) PutFile(sha256sum, data []byte) error

PutFile writes the metadata describing a new file. f should be marshalled JSON, and may be encrypted.

TODO(asjoyner): collapse the logic in PutFile and PutChunk into shared code.

func (*Drive) ReleaseChunk

func (s *Drive) ReleaseChunk(sha256sum []byte) error

ReleaseChunk deletes a chunk with a given SHA-256 sum

func (*Drive) ReleaseFile

func (s *Drive) ReleaseFile(sha256sum []byte) error

ReleaseFile deletes a file with a given SHA-256 sum

func (*Drive) Warm

func (s *Drive) Warm(chunks [][]byte, f *shade.File)

Warm is unnecessary for this client.