store

Documentation

Index

• Constants
• func NewClient(addr, label string) *client
• func NewRemoteChunkedFileSystem(client Client, chunkSize int64) *remoteChunkedFileSystem
• type CachedError
  • func (c *CachedError) Error() string
• type CachedErrorFileSystem
  • func (c *CachedErrorFileSystem) Open(ctx context.Context, path string) (http.File, error)
• type CachedFileSystem
  • func NewCachedFileSystem(src FileSystem, cache RWFileSystem) (*CachedFileSystem, <-chan error)
  • func (c *CachedFileSystem) Open(ctx context.Context, path string) (http.File, error)
  • func (c *CachedFileSystem) Wait() error
• type Client
  • func TraceClient(c Client, name string) Client
• type CloudStorageClient
  • func NewCloudStorageClient(bucket string) *CloudStorageClient
  • func (c *CloudStorageClient) Get(ctx context.Context, path string) (*File, error)
• type File
• type FileSystem
  • func ArtworkFileSystem(fs FileSystem) FileSystem
  • func FaviconFileSystem(fs FileSystem) FileSystem
  • func LogFileSystem(prefix string, fs FileSystem) FileSystem
  • func MultiFileSystem(fs ...FileSystem) FileSystem
  • func NewFileSystem(fs http.FileSystem, name string) FileSystem
  • func PathRewrite(fs FileSystem, trimPrefix, addPrefix string) FileSystem
  • func Trace(fs FileSystem, name string) FileSystem
• type RWFileSystem
  • func Dir(root string) RWFileSystem
  • func LogRWFileSystem(prefix string, fs RWFileSystem) RWFileSystem
• type RemoteFileSystem
  • func NewRemoteFileSystem(c Client) RemoteFileSystem
• type Request
• type Response
• type ResponseStatus
  • func (r ResponseStatus) String() string
• type S3Client
  • func NewS3Client(bucket string, auth aws.Auth, region aws.Region) *S3Client
  • func (c *S3Client) Get(ctx context.Context, path string) (*File, error)
• type Server
  • func NewServer(addr string) *Server
  • func (s *Server) Listen() error
  • func (s *Server) SetDefault(fs FileSystem)
  • func (s *Server) SetFileSystem(label string, fs FileSystem)
• type SizeReaderAt
  • func NewChunkedReaderAt(r io.ReadCloser, size, chunkSize int64) SizeReaderAt
  • func NewMultiReaderAt(parts ...SizeReaderAt) SizeReaderAt

Constants

const (
	StatusOK            ResponseStatus = "OK" // The response succeeded, and the file follows.
	StatusLabelNotFound                = "LF" // The label is invalid (does not exist).
	StatusPathError                    = "PE" // The path is errornous (could not be parsed).
	StatusInvalidPath                  = "IP" // The path is outside the root of the server.
	StatusNotFound                     = "NF" // The path is invalid (no file found).
	StatusFileError                    = "FE" // The path refers to a valid file, but there was a problem reading it.
	StatusDirectory                    = "ED" // The path refers to a directory, which cannot be transmitted.
)

All defined ResponseStatus values.

Functions

func NewClient

func NewClient(addr, label string) *client

NewClient initialises the default Client implementation with the given remote addr and filesystem label.

func NewRemoteChunkedFileSystem

func NewRemoteChunkedFileSystem(client Client, chunkSize int64) *remoteChunkedFileSystem

NewRemoteChunkedFileSystem creates an implementation of http.FileSystem which fetches files from the given Client, and allows access to chunks of the file contents as they are retrieved. See Open for more details.

Types

type CachedError

type CachedError struct {
	Err error
}

CachedError is an error returned by CachedErrorFileSystems when Open errors are cached rather than live.

func (*CachedError) Error

func (c *CachedError) Error() string

Error implements error.

type CachedErrorFileSystem

type CachedErrorFileSystem struct {
	FileSystem

	sync.RWMutex
	// contains filtered or unexported fields
}

CachedErrorFileSystem provides an error cache to prevent erroring FileSystem requests from being repeated. See open for more details.

func (*CachedErrorFileSystem) Open

func (c *CachedErrorFileSystem) Open(ctx context.Context, path string) (http.File, error)

Open implements FileSystem, and caches errors from the underlying FileSystem. The first time an error is encountered it is returned unchanged. Subsequent calls with an erroring path return a CachedError-wrapped version of the original error.

type CachedFileSystem

type CachedFileSystem struct {
	// contains filtered or unexported fields
}

CachedFileSystem is an implemetation of http.FileServer which caches the results of calls to src in a RWFileSystem.

func NewCachedFileSystem

func NewCachedFileSystem(src FileSystem, cache RWFileSystem) (*CachedFileSystem, <-chan error)

NewCachedFileSystem implements http.FileSystem and caches every request made to src in cache. The returned error channel passes back any errors which occur when files are being concurrently copied into the cache.

func (*CachedFileSystem) Open

func (c *CachedFileSystem) Open(ctx context.Context, path string) (http.File, error)

Open implements FileSystem. If the required file isn't in the cache then the file is opened from the src, and then concurrently copied into the cache (with errors passed back on the filesystem error channel).

func (*CachedFileSystem) Wait

func (c *CachedFileSystem) Wait() error

Wait implements RWFileSystem.

type Client

type Client interface {
	// Get reaches out to a remote server with a request for the given path.
	Get(ctx context.Context, path string) (*File, error)
}

Client is an interface which defines the Get method used to fetch files from remote hosts.

func TraceClient

func TraceClient(c Client, name string) Client

TraceClient creates a convenience method adding a tracing wrapper around a Client.

type CloudStorageClient

type CloudStorageClient struct {
	// contains filtered or unexported fields
}

CloudStorageClient implements Client and handles fetching Files from Google Cloud Storage buckets.

func NewCloudStorageClient

func NewCloudStorageClient(bucket string) *CloudStorageClient

NewCloudStorageClient creates a new Client implementation which will proxy filesystem calls to Google Cloud Storage bucket.

func (*CloudStorageClient) Get

func (c *CloudStorageClient) Get(ctx context.Context, path string) (*File, error)

type File

type File struct {
	io.ReadCloser
	Name    string
	ModTime time.Time
	Size    int64
}

File contains meta data for a remote file, and implements io.ReadCloser.

type FileSystem

type FileSystem interface {
	Open(ctx context.Context, path string) (http.File, error)
}

FileSystem is an interface which defines an open method similar to http.FileSystem, but which also includes a context parameter.

func ArtworkFileSystem

func ArtworkFileSystem(fs FileSystem) FileSystem

ArtworkFileSystem wraps a FileSystem, reworking file system operations to refer to artwork from the underlying file.

func FaviconFileSystem

func FaviconFileSystem(fs FileSystem) FileSystem

FaviconFileSystem wraps another FileSystem assumed to contain only images, which are then resized to 48px x 48px and returned in .ico format.

func LogFileSystem

func LogFileSystem(prefix string, fs FileSystem) FileSystem

LogFileSystem returns a wrapper around an http.FileSystem which logs calls to Open.

func MultiFileSystem

func MultiFileSystem(fs ...FileSystem) FileSystem

MultiFileSystem implements FileSystem and wraps an ordered list of FileSystem implementations. With each call to Open, the file systems are tried in turn until one returns without error. If all return errors, then we pass the result back to the caller.

func NewFileSystem

func NewFileSystem(fs http.FileSystem, name string) FileSystem

NewFileSystem creates a new FileSystem using an http.FileSystem as the underlying storage.

func PathRewrite

func PathRewrite(fs FileSystem, trimPrefix, addPrefix string) FileSystem

PathRewrite creates a FileSystem wrapper which will trim a prefix and add a prefix to all paths passed to Open on the underlying FileSystem.

func Trace

func Trace(fs FileSystem, name string) FileSystem

Trace is a convenience method for adding a tracing wrapper around a FileSystem.

type RWFileSystem

type RWFileSystem interface {
	FileSystem

	// Create a file with associated path, returns an io.WriteCloser.  Only when Close()
	// returns can it be assumed that the file has been written.
	Create(ctx context.Context, path string) (io.WriteCloser, error)

	// Wait blocks until any pending write calls have been completed.
	Wait() error
}

RWFileSystem is an interface which includes http.FileSystem and a Create method for creating files.

func Dir

func Dir(root string) RWFileSystem

Dir creates a new RWFileSystem with the specified root (similar to http.Dir)

func LogRWFileSystem

func LogRWFileSystem(prefix string, fs RWFileSystem) RWFileSystem

LogRWFileSystem returns a wrapper around a RWFileSystem which logs calls to Open and Create.

type RemoteFileSystem

type RemoteFileSystem interface {
	FileSystem

	// RemoteOpen returns a File which
	RemoteOpen(context.Context, string) (*File, error)
}

RemoteFileSystem is an extension of the http.FileSystem interface which includes the RemoteOpen method.

func NewRemoteFileSystem

func NewRemoteFileSystem(c Client) RemoteFileSystem

NewRemoteFileSystem creates a new file system using the given Client to handle file requests.

type Request

type Request struct {
	Path, Label string
}

Request is a type which represents an incoming request.

type Response

type Response struct {
	Status  ResponseStatus
	Size    int64 // The size of the returned output
	ModTime time.Time
	Name    string
}

Response is a type which represents a response to a Request.

type ResponseStatus

type ResponseStatus string

ResponseStatus is an enumeration of possible response statuses.

func (ResponseStatus) String

func (r ResponseStatus) String() string

Implements Stringer.

type S3Client

type S3Client struct {
	// contains filtered or unexported fields
}

S3Client implements Client and handles fetching Files from S3 buckets.

func NewS3Client

func NewS3Client(bucket string, auth aws.Auth, region aws.Region) *S3Client

NewS3Client creates a new Client implementation which will proxy filesystem calls to an S3 bucket using the given authentication and region information.

func (*S3Client) Get

func (c *S3Client) Get(ctx context.Context, path string) (*File, error)

Get implements Client.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server represents a store server, which implements a simple protocol for transfering files to the local system whilst piping them to the requesting client.

func NewServer

func NewServer(addr string) *Server

NewServer creates a new server listening on the given address.

func (*Server) Listen

func (s *Server) Listen() error

Listen starts listening on s.Addr. If there is an issue binding the listener, then an error is returned. Any errors which occur due to individual connections are logged.

func (*Server) SetDefault

func (s *Server) SetDefault(fs FileSystem)

SetDefault sets the default (empty-name) file system

func (*Server) SetFileSystem

func (s *Server) SetFileSystem(label string, fs FileSystem)

SetFileSystem sets the underlying file system to use for a label.

type SizeReaderAt

type SizeReaderAt interface {
	Size() int64
	io.ReaderAt
}

A SizeReaderAt is a ReaderAt with a Size method.

An io.SectionReader implements SizeReaderAt.

func NewChunkedReaderAt

func NewChunkedReaderAt(r io.ReadCloser, size, chunkSize int64) SizeReaderAt

NewChunkedReaderAt reads 'size' bytes of data from the reader, buffering into chunks of size 'chunkSize'. The returned SizeReaderAt allows access to chunks which have been downloaded in full, and will block on any partially downloaded chunks until they have completed.

func NewMultiReaderAt

func NewMultiReaderAt(parts ...SizeReaderAt) SizeReaderAt

NewMultiReaderAt is like io.MultiReader but produces a ReaderAt (and Size), instead of just a reader.