shard

package
v0.27.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 3, 2021 License: GPL-3.0 Imports: 17 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrEndOfListing = meta.ErrEndOfListing

ErrEndOfListing is returned from object listing with cursor when storage can't return any more objects after provided cursor. Use nil cursor object to start listing again.

Functions

func ContainerSize added in v0.15.0 

func ContainerSize(s *Shard, cid *cid.ID) (uint64, error)

func ListContainers added in v0.15.0 

func ListContainers(s *Shard) ([]*cid.ID, error)

Types

type ContainerSizePrm added in v0.15.0 

type ContainerSizePrm struct {
	// contains filtered or unexported fields
}

func (*ContainerSizePrm) WithContainerID added in v0.15.0 

func (p *ContainerSizePrm) WithContainerID(cid *cid.ID) *ContainerSizePrm

type ContainerSizeRes added in v0.15.0 

type ContainerSizeRes struct {
	// contains filtered or unexported fields
}

func (*ContainerSizeRes) Size added in v0.15.0 

func (r *ContainerSizeRes) Size() uint64

type Cursor added in v0.27.0 

type Cursor = meta.Cursor

Cursor is a type for continuous object listing.

func ListWithCursor added in v0.27.0 

func ListWithCursor(s *Shard, count uint32, cursor *Cursor) ([]*object.Address, *Cursor, error)

ListWithCursor lists physical objects available in shard starting from cursor. Includes regular, tombstone and storage group objects. Does not include inhumed objects. Use cursor value from response for consecutive requests.

Returns ErrEndOfListing if there are no more objects to return or count parameter set to zero.

type DeletePrm 

type DeletePrm struct {
	// contains filtered or unexported fields
}

DeletePrm groups the parameters of Delete operation.

func (*DeletePrm) WithAddresses 

func (p *DeletePrm) WithAddresses(addr ...*objectSDK.Address) *DeletePrm

WithAddresses is a Delete option to set the addresses of the objects to delete.

Option is required.

type DeleteRes 

type DeleteRes struct{}

DeleteRes groups resulting values of Delete operation.

type Event added in v0.16.0 

type Event interface {
	// contains filtered or unexported methods
}

Event represents class of external events.

func EventNewEpoch added in v0.16.0 

func EventNewEpoch(e uint64) Event

EventNewEpoch returns new epoch event.

type ExistsPrm 

type ExistsPrm struct {
	// contains filtered or unexported fields
}

ExistsPrm groups the parameters of Exists operation.

func (*ExistsPrm) WithAddress 

func (p *ExistsPrm) WithAddress(addr *object.Address) *ExistsPrm

WithAddress is an Exists option to set object checked for existence.

type ExistsRes 

type ExistsRes struct {
	// contains filtered or unexported fields
}

ExistsRes groups resulting values of Exists operation.

func (*ExistsRes) Exists 

func (p *ExistsRes) Exists() bool

Exists returns the fact that the object is in the shard.

type ExpiredObjectsCallback added in v0.16.0 

type ExpiredObjectsCallback func(context.Context, []*object.Address)

ExpiredObjectsCallback is a callback handling list of expired objects.

type GetPrm 

type GetPrm struct {
	// contains filtered or unexported fields
}

GetPrm groups the parameters of Get operation.

func (*GetPrm) WithAddress 

func (p *GetPrm) WithAddress(addr *objectSDK.Address) *GetPrm

WithAddress is a Get option to set the address of the requested object.

Option is required.

type GetRes 

type GetRes struct {
	// contains filtered or unexported fields
}

GetRes groups resulting values of Get operation.

func (*GetRes) Object 

func (r *GetRes) Object() *object.Object

Object returns the requested object.

type HeadPrm 

type HeadPrm struct {
	// contains filtered or unexported fields
}

HeadPrm groups the parameters of Head operation.

func (*HeadPrm) WithAddress 

func (p *HeadPrm) WithAddress(addr *objectSDK.Address) *HeadPrm

WithAddress is a Head option to set the address of the requested object.

Option is required.

func (*HeadPrm) WithRaw 

func (p *HeadPrm) WithRaw(raw bool) *HeadPrm

WithRaw is a Head option to set raw flag value. If flag is unset, then Head returns header of virtual object, otherwise it returns SplitInfo of virtual object.

type HeadRes 

type HeadRes struct {
	// contains filtered or unexported fields
}

HeadRes groups resulting values of Head operation.

func (*HeadRes) Object 

func (r *HeadRes) Object() *object.Object

Object returns the requested object header.

type ID 

type ID []byte

ID represents Shard identifier.

Each shard should have the unique ID within a single instance of local storage.

func NewIDFromBytes 

func NewIDFromBytes(v []byte) *ID

NewIDFromBytes constructs ID from byte slice.

func (ID) String 

func (id ID) String() string

type Info 

type Info struct {
	// Identifier of the shard.
	ID *ID

	// Information about the metabase.
	MetaBaseInfo meta.Info

	// Information about the BLOB storage.
	BlobStorInfo blobstor.Info

	// Information about the Write Cache.
	WriteCacheInfo blobstor.Info

	// Weight parameters of the shard.
	WeightValues WeightValues
}

Info groups the information about Shard.

type InhumePrm 

type InhumePrm struct {
	// contains filtered or unexported fields
}

InhumePrm encapsulates parameters for inhume operation.

func (*InhumePrm) MarkAsGarbage added in v0.16.0 

func (p *InhumePrm) MarkAsGarbage(addr ...*objectSDK.Address) *InhumePrm

MarkAsGarbage marks object to be physically removed from shard.

Should not be called along with WithTarget.

func (*InhumePrm) WithTarget 

func (p *InhumePrm) WithTarget(tombstone *objectSDK.Address, addrs ...*objectSDK.Address) *InhumePrm

WithTarget sets list of objects that should be inhumed and tombstone address as the reason for inhume operation.

tombstone should not be nil, addr should not be empty. Should not be called along with MarkAsGarbage.

type InhumeRes 

type InhumeRes struct{}

InhumeRes encapsulates results of inhume operation.

type ListContainersPrm added in v0.15.0 

type ListContainersPrm struct{}

type ListContainersRes added in v0.15.0 

type ListContainersRes struct {
	// contains filtered or unexported fields
}

func (*ListContainersRes) Containers added in v0.15.0 

func (r *ListContainersRes) Containers() []*cid.ID

type ListWithCursorPrm added in v0.27.0 

type ListWithCursorPrm struct {
	// contains filtered or unexported fields
}

ListWithCursorPrm contains parameters for ListWithCursor operation.

func (*ListWithCursorPrm) WithCount added in v0.27.0 

func (p *ListWithCursorPrm) WithCount(count uint32) *ListWithCursorPrm

WithCount sets maximum amount of addresses that ListWithCursor should return.

func (*ListWithCursorPrm) WithCursor added in v0.27.0 

func (p *ListWithCursorPrm) WithCursor(cursor *Cursor) *ListWithCursorPrm

WithCursor sets cursor for ListWithCursor operation. For initial request, ignore this param or use nil value. For consecutive requests, use value from ListWithCursorRes.

type ListWithCursorRes added in v0.27.0 

type ListWithCursorRes struct {
	// contains filtered or unexported fields
}

ListWithCursorRes contains values returned from ListWithCursor operation.

func (ListWithCursorRes) AddressList added in v0.27.0 

func (r ListWithCursorRes) AddressList() []*object.Address

AddressList returns addresses selected by ListWithCursor operation.

func (ListWithCursorRes) Cursor added in v0.27.0 

func (r ListWithCursorRes) Cursor() *Cursor

Cursor returns cursor for consecutive listing requests.

type Mode 

type Mode uint32

Mode represents enumeration of Shard work modes. 

const (

	// ModeActive is a Mode value for active shard.
	ModeActive Mode

	// ModeInactive is a Mode value for inactive shard.
	ModeInactive

	// ModeReadOnly is a Mode value for read-only shard.
	ModeReadOnly

	// ModeFault is a Mode value for faulty shard.
	ModeFault

	// ModeEvacuate is a Mode value for evacuating shard.
	ModeEvacuate
)

func (Mode) String 

func (m Mode) String() string

type Option 

type Option func(*cfg)

Option represents Shard's constructor option.

func WithBlobStorOptions 

func WithBlobStorOptions(opts ...blobstor.Option) Option

WithBlobStorOptions returns option to set internal BlobStor options.

func WithExpiredObjectsCallback added in v0.16.0 

func WithExpiredObjectsCallback(cb ExpiredObjectsCallback) Option

WithExpiredObjectsCallback returns option to specify callback of the expired tombstones handler.

func WithGCEventChannelInitializer added in v0.16.0 

func WithGCEventChannelInitializer(chInit func() <-chan Event) Option

WithGCEventChannelInitializer returns option to set set initializer of GC event channel.

func WithGCRemoverSleepInterval added in v0.16.0 

func WithGCRemoverSleepInterval(dur time.Duration) Option

WithGCRemoverSleepInterval returns option to specify sleep interval between object remover executions.

func WithGCWorkerPoolInitializer added in v0.16.0 

func WithGCWorkerPoolInitializer(wpInit func(int) util.WorkerPool) Option

WithGCWorkerPoolInitializer returns option to set initializer of worker pool with specified worker number.

func WithID 

func WithID(id *ID) Option

WithID returns option to set shard identifier.

func WithLogger 

func WithLogger(l *logger.Logger) Option

WithLogger returns option to set Shard's logger.

func WithMetaBaseOptions 

func WithMetaBaseOptions(opts ...meta.Option) Option

WithMetaBaseOptions returns option to set internal metabase options.

func WithRefillMetabase added in v0.25.0 

func WithRefillMetabase(v bool) Option

WithRefillMetabase returns option to set flag to refill the Metabase on Shard's initialization step.

func WithRemoverBatchSize added in v0.16.0 

func WithRemoverBatchSize(sz int) Option

WithRemoverBatchSize returns option to set batch size of single removal operation.

func WithWriteCache 

func WithWriteCache(use bool) Option

WithWriteCache returns option to toggle write cache usage.

func WithWriteCacheOptions 

func WithWriteCacheOptions(opts ...writecache.Option) Option

WithWriteCacheOptions returns option to set internal write cache options.

type PutPrm 

type PutPrm struct {
	// contains filtered or unexported fields
}

PutPrm groups the parameters of Put operation.

func (*PutPrm) WithObject 

func (p *PutPrm) WithObject(obj *object.Object) *PutPrm

WithObject is a Put option to set object to save.

type PutRes 

type PutRes struct{}

PutRes groups resulting values of Put operation.

type RngPrm 

type RngPrm struct {
	// contains filtered or unexported fields
}

RngPrm groups the parameters of GetRange operation.

func (*RngPrm) WithAddress 

func (p *RngPrm) WithAddress(addr *objectSDK.Address) *RngPrm

WithAddress is a Rng option to set the address of the requested object.

Option is required.

func (*RngPrm) WithRange 

func (p *RngPrm) WithRange(off uint64, ln uint64) *RngPrm

WithRange is a GetRange option to set range of requested payload data.

type RngRes 

type RngRes struct {
	// contains filtered or unexported fields
}

RngRes groups resulting values of GetRange operation.

func (*RngRes) Object 

func (r *RngRes) Object() *object.Object

Object returns the requested object part.

Instance payload contains the requested range of the original object.

type SelectPrm 

type SelectPrm struct {
	// contains filtered or unexported fields
}

SelectPrm groups the parameters of Select operation.

func (*SelectPrm) WithContainerID 

func (p *SelectPrm) WithContainerID(cid *cid.ID) *SelectPrm

WithContainerID is a Select option to set the container id to search in.

func (*SelectPrm) WithFilters 

func (p *SelectPrm) WithFilters(fs objectSDK.SearchFilters) *SelectPrm

WithFilters is a Select option to set the object filters.

type SelectRes 

type SelectRes struct {
	// contains filtered or unexported fields
}

SelectRes groups resulting values of Select operation.

func (*SelectRes) AddressList 

func (r *SelectRes) AddressList() []*objectSDK.Address

AddressList returns list of addresses of the selected objects.

type Shard 

type Shard struct {
	// contains filtered or unexported fields
}

Shard represents single shard of NeoFS Local Storage Engine.

func New 

func New(opts ...Option) *Shard

New creates, initializes and returns new Shard instance.

func (*Shard) Close 

func (s *Shard) Close() error

Close releases all Shard's components.

func (*Shard) ContainerSize added in v0.15.0 

func (s *Shard) ContainerSize(prm *ContainerSizePrm) (*ContainerSizeRes, error)

func (*Shard) Delete 

func (s *Shard) Delete(prm *DeletePrm) (*DeleteRes, error)

Delete removes data from the shard's writeCache, metaBase and blobStor.

func (*Shard) DumpInfo 

func (s *Shard) DumpInfo() Info

DumpInfo returns information about the Shard.

func (*Shard) Exists 

func (s *Shard) Exists(prm *ExistsPrm) (*ExistsRes, error)

Exists checks if object is presented in shard.

Returns any error encountered that does not allow to unambiguously determine the presence of an object.

func (*Shard) Get 

func (s *Shard) Get(prm *GetPrm) (*GetRes, error)

Get reads an object from shard.

Returns any error encountered that did not allow to completely read the object part.

Returns object.ErrNotFound if requested object is missing in shard.

func (*Shard) GetRange 

func (s *Shard) GetRange(prm *RngPrm) (*RngRes, error)

GetRange reads part of an object from shard.

Returns any error encountered that did not allow to completely read the object part.

Returns ErrRangeOutOfBounds if requested object range is out of bounds.

func (*Shard) HandleExpiredTombstones added in v0.16.0 

func (s *Shard) HandleExpiredTombstones(tss map[string]struct{})

HandleExpiredTombstones marks to be removed all objects that are protected by tombstones with string addresses from tss. If successful, marks tombstones themselves as garbage.

Does not modify tss.

func (*Shard) Head 

func (s *Shard) Head(prm *HeadPrm) (*HeadRes, error)

Head reads header of the object from the shard.

Returns any error encountered.

func (*Shard) ID 

func (s *Shard) ID() *ID

ID returns Shard identifier.

func (*Shard) Inhume 

func (s *Shard) Inhume(prm *InhumePrm) (*InhumeRes, error)

Inhume calls metabase. Inhume method to mark object as removed. It won't be removed physically from blobStor and metabase until `Delete` operation.

func (*Shard) Init 

func (s *Shard) Init() error

Init initializes all Shard's components.

func (*Shard) List 

func (s *Shard) List() (*SelectRes, error)

List returns all objects physically stored in the Shard.

func (*Shard) ListContainers added in v0.15.0 

func (s *Shard) ListContainers(_ *ListContainersPrm) (*ListContainersRes, error)

func (*Shard) ListWithCursor added in v0.27.0 

func (s *Shard) ListWithCursor(prm *ListWithCursorPrm) (*ListWithCursorRes, error)

ListWithCursor lists physical objects available in shard starting from cursor. Includes regular, tombstone and storage group objects. Does not include inhumed objects. Use cursor value from response for consecutive requests.

Returns ErrEndOfListing if there are no more objects to return or count parameter set to zero.

func (*Shard) Open 

func (s *Shard) Open() error

Open opens all Shard's components.

func (*Shard) Put 

func (s *Shard) Put(prm *PutPrm) (*PutRes, error)

Put saves the object in shard.

Returns any error encountered that did not allow to completely save the object.

func (*Shard) Select 

func (s *Shard) Select(prm *SelectPrm) (*SelectRes, error)

Select selects the objects from shard that match select parameters.

Returns any error encountered that did not allow to completely select the objects.

func (*Shard) SetMode 

func (s *Shard) SetMode(m Mode) error

SetMode sets mode of the shard.

Returns any error encountered that did not allow to set shard mode.

func (*Shard) ToMoveIt 

func (s *Shard) ToMoveIt(prm *ToMoveItPrm) (*ToMoveItRes, error)

ToMoveIt calls metabase.ToMoveIt method to mark object as relocatable to another shard.

func (*Shard) WeightValues 

func (s *Shard) WeightValues() WeightValues

WeightValues returns current weight values of the Shard.

type ToMoveItPrm 

type ToMoveItPrm struct {
	// contains filtered or unexported fields
}

ToMoveItPrm encapsulates parameters for ToMoveIt operation.

func (*ToMoveItPrm) WithAddress 

func (p *ToMoveItPrm) WithAddress(addr *objectSDK.Address) *ToMoveItPrm

WithAddress sets object address that should be marked to move into another shard.

type ToMoveItRes 

type ToMoveItRes struct{}

ToMoveItRes encapsulates results of ToMoveIt operation.

type WeightValues 

type WeightValues struct {
	// Amount of free disk space. Measured in kilobytes.
	FreeSpace uint64
}

WeightValues groups values of Shard weight parameters.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.