storage

Documentation

Index

• Constants
• func AcquireByteBuffer() *bytebufferpool.ByteBuffer
• func AcquireIndexBuffer() *bytebufferpool.ByteBuffer
• func ReleaseByteBuffer(b *bytebufferpool.ByteBuffer)
• func ReleaseIndexBuffer(b *bytebufferpool.ByteBuffer)
• type Allocator
• type Blob
  • func OpenBlob(path string, cfg *BlobConfig) (*Blob, error)
  • func (b *Blob) Allocate(size int64) (int64, error)
  • func (b *Blob) Close() error
  • func (b *Blob) Sync() (err error)
  • func (b *Blob) Truncate(size int64) (err error)
• type BlobBackend
• type BlobConfig
  • func (i *BlobConfig) GetInitialSize() int64
• type BlobHeader
  • func (h *BlobHeader) Append(b []byte) []byte
  • func (h BlobHeader) Put(buf []byte) int
  • func (h *BlobHeader) Read(buf []byte) error
• type StatBackend
• type TruncateSyncer

Constants

const (
	// ErrIsDirectory means that directory is found where file assumed.
	ErrIsDirectory sError = "Got directory, need file"
	// ErrBadHeader means that blob is unable to initialize due header corruption or wrong file format.
	ErrBadHeader sError = "Bad header magic bytes, file corrupted or in wrong format"
	// ErrBadHeaderCapacity means that decoded capacity from header is less than actual file size.
	ErrBadHeaderCapacity sError = "Capacity in header is less than actual file size, file can be corrupted"
	// ErrBadHeaderCRC means that header crc check failed.
	ErrBadHeaderCRC sError = "Header CRC missmatch"
)

const (
	// DefaultBlobSize is initial capacity for newly created blob.
	DefaultBlobSize = 1024
)

Functions

func AcquireByteBuffer

func AcquireByteBuffer() *bytebufferpool.ByteBuffer

AcquireByteBuffer returns an empty byte buffer from the pool.

Acquired byte buffer may be returned to the pool via ReleaseByteBuffer call. This reduces the number of memory allocations required for byte buffer management.

func AcquireIndexBuffer

func AcquireIndexBuffer() *bytebufferpool.ByteBuffer

AcquireIndexBuffer returns an empty byte buffer from the pool.

Acquired byte buffer may be returned to the pool via ReleaseByteBuffer call. This reduces the number of memory allocations required for byte buffer management.

func ReleaseByteBuffer

func ReleaseByteBuffer(b *bytebufferpool.ByteBuffer)

ReleaseByteBuffer returns byte buffer to the pool.

ByteBuffer.B mustn't be touched after returning it to the pool. Otherwise data races occur.

func ReleaseIndexBuffer

func ReleaseIndexBuffer(b *bytebufferpool.ByteBuffer)

ReleaseIndexBuffer returns byte buffer to the pool.

ByteBuffer.B mustn't be touched after returning it to the pool. Otherwise data races occur.

Types

type Allocator

type Allocator interface {
	Allocate(size int64) (offset int64, err error)
}

Allocator wraps Allocate method for allocating slices. Should be goroutine-safe.

type Blob

type Blob struct {
	sync.RWMutex
	Backend  BlobBackend
	Size     int64
	Capacity int64
	// contains filtered or unexported fields
}

Blob represents set of data slices on top of BlobBackend.

func OpenBlob

func OpenBlob(path string, cfg *BlobConfig) (*Blob, error)

OpenBlob opens or creates a Blob for the given path.

The returned Blob instance is goroutine-safe. The Blob must be closed after use, by calling Close method.

func (*Blob) Allocate

func (b *Blob) Allocate(size int64) (int64, error)

Allocate returns offset to atomically allocated slice of provided size and error if any. After allocation it is safe to call WriteAt(b, offset) with len(b) = size.

Example:

data := make([]byte, size)
offset, _ := b.Allocate(size)
b.WriteAt(data, offset)

func (*Blob) Close

func (b *Blob) Close() error

Close closes the Blob, rendering it unusable for changes. It returns an error, if any.

func (*Blob) Sync

func (b *Blob) Sync() (err error)

Sync commits the current state of blob.

func (*Blob) Truncate

func (b *Blob) Truncate(size int64) (err error)

Truncate changes the capacity of the blob.

type BlobBackend

type BlobBackend interface {
	io.ReaderAt
	io.WriterAt
	io.Closer
	TruncateSyncer
}

BlobBackend is the interface that groups basic methods for consistent storage.

type BlobConfig

type BlobConfig struct {
	InitialSize int64
}

BlobConfig is configuration for blob processing.

func (*BlobConfig) GetInitialSize

func (i *BlobConfig) GetInitialSize() int64

GetInitialSize returns initial size for the blob used upon creation.

type BlobHeader

type BlobHeader struct {
	Size     int64
	Capacity int64
}

BlobHeader contains info about Blob size and capacity.

func (*BlobHeader) Append

func (h *BlobHeader) Append(b []byte) []byte

Append encodes header to b and returns b.

func (BlobHeader) Put

func (h BlobHeader) Put(buf []byte) int

Put encodes BlobHeader into buf and returns the number of bytes written. If the buffer is too small, Put will panic.

func (*BlobHeader) Read

func (h *BlobHeader) Read(buf []byte) error

Read decodes BlobHeader from buf and returns ErrBadHeader if it fails.

type StatBackend

type StatBackend interface {
	Stat() (os.FileInfo, error)
}

StatBackend wraps Stat method for retrieving file stats.

type TruncateSyncer

type TruncateSyncer interface {
	// Sync commits the current contents of the file to stable storage.
	Sync() error
	// Truncate changes the size of the storage.
	Truncate(size int64) error
}

TruncateSyncer is the interface that groups Sync and Truncate methods.