blobstor

Documentation

Index

• Variables
• func IterateBinaryObjects(ctx context.Context, blz *BlobStor, ...) error
• type BlobStor
  • func New(opts ...Option) *BlobStor
  • func (b *BlobStor) Close() error
  • func (b *BlobStor) Compressor() *compression.Config
  • func (b *BlobStor) Delete(ctx context.Context, prm common.DeletePrm) (common.DeleteRes, error)
  • func (b *BlobStor) DumpInfo() Info
  • func (b *BlobStor) Exists(ctx context.Context, prm common.ExistsPrm) (common.ExistsRes, error)
  • func (b *BlobStor) Get(ctx context.Context, prm common.GetPrm) (res common.GetRes, err error)
  • func (b *BlobStor) GetRange(ctx context.Context, prm common.GetRangePrm) (res common.GetRangeRes, err error)
  • func (b *BlobStor) Init() error
  • func (b *BlobStor) Iterate(ctx context.Context, prm common.IteratePrm) (common.IterateRes, error)
  • func (b *BlobStor) Open(ctx context.Context, readOnly bool) error
  • func (b *BlobStor) Put(ctx context.Context, prm common.PutPrm) (common.PutRes, error)
  • func (b *BlobStor) SetLogger(l *logger.Logger)
  • func (b *BlobStor) SetMode(m mode.Mode) error
  • func (b *BlobStor) SetParentID(parentID string)
  • func (b *BlobStor) SetReportErrorFunc(f func(string, error))
• type Info
• type Metrics
• type Option
  • func WithCompressObjects(comp bool) Option
  • func WithLogger(l *logger.Logger) Option
  • func WithMetrics(m Metrics) Option
  • func WithStorages(st []SubStorage) Option
  • func WithUncompressableContentTypes(values []string) Option
• type SubStorage
• type SubStorageInfo

Variables

var ErrInitBlobovniczas = errors.New("failure on blobovnicza initialization stage")

ErrInitBlobovniczas is returned when blobovnicza initialization fails.

var ErrNoPlaceFound = logicerr.New("couldn't find a place to store an object")

ErrNoPlaceFound is returned when object can't be saved to any sub-storage component because of the policy.

Functions

func IterateBinaryObjects

func IterateBinaryObjects(ctx context.Context, blz *BlobStor, f func(addr oid.Address, data []byte, descriptor []byte) error) error

IterateBinaryObjects is a helper function which iterates over BlobStor and passes binary objects to f. Errors related to object reading and unmarshaling are logged and skipped.

Types

type BlobStor

type BlobStor struct {
	// contains filtered or unexported fields
}

BlobStor represents FrostFS local BLOB storage.

func New

func New(opts ...Option) *BlobStor

New creates, initializes and returns new BlobStor instance.

func (*BlobStor) Close

func (b *BlobStor) Close() error

Close releases all internal resources of BlobStor.

func (*BlobStor) Compressor

func (b *BlobStor) Compressor() *compression.Config

func (*BlobStor) Delete

func (b *BlobStor) Delete(ctx context.Context, prm common.DeletePrm) (common.DeleteRes, error)

func (*BlobStor) DumpInfo

func (b *BlobStor) DumpInfo() Info

DumpInfo returns information about blob stor.

func (*BlobStor) Exists

func (b *BlobStor) Exists(ctx context.Context, prm common.ExistsPrm) (common.ExistsRes, error)

Exists checks if the object is presented in BLOB storage.

Returns any error encountered that did not allow to completely check object existence.

func (*BlobStor) Get

func (b *BlobStor) Get(ctx context.Context, prm common.GetPrm) (res common.GetRes, err error)

Get reads the object from b. If the descriptor is present, only one sub-storage is tried, Otherwise, each sub-storage is tried in order.

func (*BlobStor) GetRange

func (b *BlobStor) GetRange(ctx context.Context, prm common.GetRangePrm) (res common.GetRangeRes, err error)

GetRange reads object payload data from b. If the descriptor is present, only one sub-storage is tried, Otherwise, each sub-storage is tried in order.

func (*BlobStor) Init

func (b *BlobStor) Init() error

Init initializes internal data structures and system resources.

If BlobStor is already initialized, no action is taken.

Returns wrapped ErrInitBlobovniczas on blobovnicza tree's initializaiton failure.

func (*BlobStor) Iterate

func (b *BlobStor) Iterate(ctx context.Context, prm common.IteratePrm) (common.IterateRes, error)

Iterate traverses the storage over the stored objects and calls the handler on each element.

Returns any error encountered that did not allow to completely iterate over the storage.

If handler returns an error, method wraps and returns it immediately.

func (*BlobStor) Open

func (b *BlobStor) Open(ctx context.Context, readOnly bool) error

Open opens BlobStor.

func (*BlobStor) Put

func (b *BlobStor) Put(ctx context.Context, prm common.PutPrm) (common.PutRes, error)

Put saves the object in BLOB storage.

If object is "big", BlobStor saves the object in shallow dir. Otherwise, BlobStor saves the object in blobonicza. In this case the identifier of blobovnicza is returned.

Returns any error encountered that did not allow to completely save the object.

func (*BlobStor) SetLogger

func (b *BlobStor) SetLogger(l *logger.Logger)

SetLogger sets logger. It is used after the shard ID was generated to use it in logs.

func (*BlobStor) SetMode

func (b *BlobStor) SetMode(m mode.Mode) error

SetMode sets the blobstor mode of operation.

func (*BlobStor) SetParentID

func (b *BlobStor) SetParentID(parentID string)

func (*BlobStor) SetReportErrorFunc

func (b *BlobStor) SetReportErrorFunc(f func(string, error))

SetReportErrorFunc allows to provide a function to be called on disk errors. This function MUST be called before Open.

type Info

type Info struct {
	SubStorages []SubStorageInfo
}

Info contains information about blobstor.

type Metrics

type Metrics interface {
	SetParentID(parentID string)
	SetMode(readOnly bool)
	Close()

	Delete(d time.Duration, success, withStorageID bool)
	Exists(d time.Duration, success, withStorageID bool)
	GetRange(d time.Duration, size int, success, withStorageID bool)
	Get(d time.Duration, size int, success, withStorageID bool)
	Iterate(d time.Duration, success bool)
	Put(d time.Duration, size int, success bool)
}

type Option

type Option func(*cfg)

Option represents BlobStor's constructor option.

func WithCompressObjects

func WithCompressObjects(comp bool) Option

WithCompressObjects returns option to toggle compression of the stored objects.

If true, Zstandard algorithm is used for data compression.

If compressor (decompressor) creation failed, the uncompressed option will be used, and the error is recorded in the provided log.

func WithLogger

func WithLogger(l *logger.Logger) Option

WithLogger returns option to specify BlobStor's logger.

func WithMetrics

func WithMetrics(m Metrics) Option

func WithStorages

func WithStorages(st []SubStorage) Option

WithStorages provides sub-blobstors.

func WithUncompressableContentTypes

func WithUncompressableContentTypes(values []string) Option

WithUncompressableContentTypes returns option to disable decompression for specific content types as seen by object.AttributeContentType attribute.

type SubStorage

type SubStorage struct {
	Storage common.Storage
	Policy  func(*objectSDK.Object, []byte) bool
}

SubStorage represents single storage component with some storage policy.

type SubStorageInfo

type SubStorageInfo struct {
	Type string
	Path string
}

SubStorageInfo contains information about blobstor storage component.