files

Documentation

Overview

Package files implements the blobserver interface by storing each blob in its own file in nested directories. Users don't use the "files" type directly; it's used by "localdisk" and in the future "sftp" and "webdav".

Index

• type ReadableFile
• type Storage
  • func NewStorage(fs VFS, root string) *Storage
  • func (ds *Storage) EnumerateBlobs(ctx context.Context, dest chan<- blob.SizedRef, after string, limit int) error
  • func (ds *Storage) Fetch(ctx context.Context, br blob.Ref) (io.ReadCloser, uint32, error)
  • func (ds *Storage) ReceiveBlob(ctx context.Context, blobRef blob.Ref, source io.Reader) (blob.SizedRef, error)
  • func (ds *Storage) RemoveBlobs(ctx context.Context, blobs []blob.Ref) error
  • func (ds *Storage) SetNewFileGate(g *syncutil.Gate)
  • func (ds *Storage) StatBlobs(ctx context.Context, blobs []blob.Ref, fn func(blob.SizedRef) error) error
  • func (ds *Storage) SubFetch(ctx context.Context, br blob.Ref, offset, length int64) (io.ReadCloser, error)
• type VFS
  • func OSFS() VFS
• type WritableFile

Types

type ReadableFile

type ReadableFile interface {
	io.Reader
	io.Seeker
	io.Closer
}

ReadableFile is the interface required by read-only files returned by VFS.Open.

type Storage

type Storage struct {
	// contains filtered or unexported fields
}

func NewStorage

func NewStorage(fs VFS, root string) *Storage

func (*Storage) EnumerateBlobs

func (ds *Storage) EnumerateBlobs(ctx context.Context, dest chan<- blob.SizedRef, after string, limit int) error

func (*Storage) Fetch

func (ds *Storage) Fetch(ctx context.Context, br blob.Ref) (io.ReadCloser, uint32, error)

func (*Storage) ReceiveBlob

func (ds *Storage) ReceiveBlob(ctx context.Context, blobRef blob.Ref, source io.Reader) (blob.SizedRef, error)

func (*Storage) RemoveBlobs

func (ds *Storage) RemoveBlobs(ctx context.Context, blobs []blob.Ref) error

func (*Storage) SetNewFileGate

func (ds *Storage) SetNewFileGate(g *syncutil.Gate)

SetNewFileGate sets a gate (counting semaphore) on the number of new files that may be opened for writing at a time.

func (*Storage) StatBlobs

func (ds *Storage) StatBlobs(ctx context.Context, blobs []blob.Ref, fn func(blob.SizedRef) error) error

func (*Storage) SubFetch

func (ds *Storage) SubFetch(ctx context.Context, br blob.Ref, offset, length int64) (io.ReadCloser, error)

type VFS

type VFS interface {
	Remove(string) error // files, not directories
	RemoveDir(string) error
	Stat(string) (os.FileInfo, error)
	Lstat(string) (os.FileInfo, error)
	Open(string) (ReadableFile, error)
	MkdirAll(path string, perm os.FileMode) error

	// Rename is a POSIX-style rename, overwriting newname if it exists.
	Rename(oldname, newname string) error

	// TempFile should behave like os.CreateTemp
	TempFile(dir, prefix string) (WritableFile, error)

	ReadDirNames(dir string) ([]string, error)
}

VFS describes the virtual filesystem needed by this package.

It is currently not possible to use this from other packages, but that will change.

func OSFS

func OSFS() VFS

OSFS returns an implementation of VFS interface using the host filesystem.

type WritableFile

type WritableFile interface {
	io.Writer
	io.Closer
	// Name returns the full path to the file.
	// It should behave like (*os.File).Name.
	Name() string
	// Sync fsyncs the file, like (*os.File).Sync.
	Sync() error
}

WritableFile is the interface required by files opened for Write from VFS.TempFile.