blobstor

Documentation

Index

• Variables
• type BlobStor
  • func New(opts ...Option) *BlobStor
  • func (b *BlobStor) Close() error
  • func (b *BlobStor) Delete(addr oid.Address, storageID []byte) error
  • func (b *BlobStor) DumpInfo() Info
  • func (b *BlobStor) Exists(addr oid.Address, storageID []byte) (bool, error)
  • func (b *BlobStor) Get(addr oid.Address, storageID []byte) (*objectSDK.Object, error)
  • func (b *BlobStor) GetBytes(addr oid.Address, subStorageID []byte) ([]byte, error)
  • func (b *BlobStor) GetRange(addr oid.Address, offset uint64, length uint64, storageID []byte) ([]byte, error)
  • func (b *BlobStor) Init() error
  • func (b *BlobStor) Iterate(objHandler func(addr oid.Address, data []byte, id []byte) error, ...) error
  • func (b *BlobStor) IterateBinaryObjects(f func(addr oid.Address, data []byte, descriptor []byte) error) error
  • func (b *BlobStor) NeedsCompression(obj *objectSDK.Object) bool
  • func (b *BlobStor) ObjectStatus(address oid.Address) (ObjectStatus, error)
  • func (b *BlobStor) Open(readOnly bool) error
  • func (b *BlobStor) Put(addr oid.Address, obj *objectSDK.Object, raw []byte) ([]byte, error)
  • func (b *BlobStor) SetLogger(l *zap.Logger)
  • func (b *BlobStor) SetMode(m mode.Mode) error
• type Info
• type ObjectStatus
• type ObjectSubstorageStatus
• type Option
  • func WithCompressObjects(comp bool) Option
  • func WithLogger(l *zap.Logger) Option
  • func WithStorages(st []SubStorage) Option
  • func WithUncompressableContentTypes(values []string) Option
• type SubStorage
• type SubStorageInfo

Variables

var ErrNoPlaceFound = logicerr.New("couldn't find a place to store an object")

ErrNoPlaceFound is returned when object can't be saved to any sub-storage component because of the policy.

Types

type BlobStor

type BlobStor struct {
	// contains filtered or unexported fields
}

BlobStor represents NeoFS local BLOB storage.

func New

func New(opts ...Option) *BlobStor

New creates, initializes and returns new BlobStor instance.

func (*BlobStor) Close

func (b *BlobStor) Close() error

Close releases all internal resources of BlobStor.

func (*BlobStor) Delete

func (b *BlobStor) Delete(addr oid.Address, storageID []byte) error

func (*BlobStor) DumpInfo

func (b *BlobStor) DumpInfo() Info

DumpInfo returns information about blob stor.

func (*BlobStor) Exists

func (b *BlobStor) Exists(addr oid.Address, storageID []byte) (bool, error)

Exists checks if the object is presented in BLOB storage.

Returns any error encountered that did not allow to completely check object existence.

func (*BlobStor) Get

func (b *BlobStor) Get(addr oid.Address, storageID []byte) (*objectSDK.Object, error)

Get reads the object from b. If the descriptor is present, only one sub-storage is tried, Otherwise, each sub-storage is tried in order.

func (*BlobStor) GetBytes

func (b *BlobStor) GetBytes(addr oid.Address, subStorageID []byte) ([]byte, error)

GetBytes reads object from the BlobStor by address into memory buffer in a canonical NeoFS binary format. Returns apistatus.ObjectNotFound if object is missing.

func (*BlobStor) GetRange

func (b *BlobStor) GetRange(addr oid.Address, offset uint64, length uint64, storageID []byte) ([]byte, error)

GetRange reads object payload data from b. If the descriptor is present, only one sub-storage is tried, Otherwise, each sub-storage is tried in order.

func (*BlobStor) Init

func (b *BlobStor) Init() error

Init initializes internal data structures and system resources.

If BlobStor is already initialized, no action is taken.

func (*BlobStor) Iterate

func (b *BlobStor) Iterate(objHandler func(addr oid.Address, data []byte, id []byte) error, errorHandler func(addr oid.Address, err error) error) error

Iterate traverses the storage over the stored objects and calls the objHandler on each element. errorHandler is called if specified and ignoreErrors is true.

Returns any error encountered that did not allow to completely iterate over the storage.

If handler returns an error, method wraps and returns it immediately.

func (*BlobStor) IterateBinaryObjects

func (b *BlobStor) IterateBinaryObjects(f func(addr oid.Address, data []byte, descriptor []byte) error) error

IterateBinaryObjects is a helper method which iterates over BlobStor and passes binary objects to f. Errors related to object reading and unmarshaling are logged and skipped.

func (*BlobStor) NeedsCompression

func (b *BlobStor) NeedsCompression(obj *objectSDK.Object) bool

NeedsCompression returns true if the object should be compressed. For an object to be compressed 2 conditions must hold: 1. Compression is enabled in settings. 2. Object MIME Content-Type is allowed for compression.

func (*BlobStor) ObjectStatus

func (b *BlobStor) ObjectStatus(address oid.Address) (ObjectStatus, error)

ObjectStatus returns the status of the object in the Blob storage. It contains status of the object in all blob substorages.

func (*BlobStor) Open

func (b *BlobStor) Open(readOnly bool) error

Open opens BlobStor.

func (*BlobStor) Put

func (b *BlobStor) Put(addr oid.Address, obj *objectSDK.Object, raw []byte) ([]byte, error)

Put saves the object in BLOB storage. raw can be nil, in which case obj is serialized internally.

If object is "big", BlobStor saves the object in shallow dir. Otherwise, BlobStor saves the object in peapod.

Returns storage ID that saved the object given or an error if any.

func (*BlobStor) SetLogger

func (b *BlobStor) SetLogger(l *zap.Logger)

SetLogger sets logger. It is used after the shard ID was generated to use it in logs.

func (*BlobStor) SetMode

func (b *BlobStor) SetMode(m mode.Mode) error

SetMode sets the blobstor mode of operation.

type Info

type Info struct {
	SubStorages []SubStorageInfo
}

Info contains information about blobstor.

type ObjectStatus

type ObjectStatus struct {
	Substorages []ObjectSubstorageStatus
}

ObjectStatus represents the status of the object in the Blob storage.

type ObjectSubstorageStatus

type ObjectSubstorageStatus struct {
	Type  string
	Path  string
	Error error
}

ObjectSubstorageStatus represents the status of the object in the Blob storage, containing the type and path of the storage and an error if it occurred.

type Option

type Option func(*cfg)

Option represents BlobStor's constructor option.

func WithCompressObjects

func WithCompressObjects(comp bool) Option

WithCompressObjects returns option to toggle compression of the stored objects.

If true, Zstandard algorithm is used for data compression.

If compressor (decompressor) creation failed, the uncompressed option will be used, and the error is recorded in the provided log.

func WithLogger

func WithLogger(l *zap.Logger) Option

WithLogger returns option to specify BlobStor's logger.

func WithStorages

func WithStorages(st []SubStorage) Option

WithStorages provides sub-blobstors.

func WithUncompressableContentTypes

func WithUncompressableContentTypes(values []string) Option

WithUncompressableContentTypes returns option to disable decompression for specific content types as seen by object.AttributeContentType attribute.

type SubStorage

type SubStorage struct {
	Storage common.Storage
	Policy  func(*objectSDK.Object, []byte) bool
}

SubStorage represents single storage component with some storage policy.

type SubStorageInfo

type SubStorageInfo struct {
	Type string
	Path string
}

SubStorageInfo contains information about blobstor storage component.