persist

package
v1.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 20, 2021 License: Apache-2.0 Imports: 12 Imported by: 29

Documentation

Overview

Package persist is a generated GoMock package.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CommitLogFile added in v0.7.0 

type CommitLogFile struct {
	FilePath string
	Index    int64
}

CommitLogFile represents a commit log file and its associated metadata.

type CommitLogFiles added in v0.7.0 

type CommitLogFiles []CommitLogFile

CommitLogFiles represents a slice of commitlog files.

func (CommitLogFiles) Contains added in v0.7.0 

func (c CommitLogFiles) Contains(path string) bool

Contains returns a boolean indicating whether the CommitLogFiles slice contains the provided CommitlogFile based on its path.

type DataCloser 

type DataCloser func() error

DataCloser is a function that performs cleanup after persisting the data blocks for a (shard, blockStart) combination.

type DataFn 

type DataFn func(metadata Metadata, segment ts.Segment, checksum uint32) error

DataFn is a function that persists a m3db segment for a given ID.

type DataPrepareOptions 

type DataPrepareOptions struct {
	NamespaceMetadata namespace.Metadata
	BlockStart        xtime.UnixNano
	Shard             uint32
	// This volume index is only used when preparing for a flush fileset type.
	// When opening a snapshot, the new volume index is determined by looking
	// at what files exist on disk.
	VolumeIndex    int
	FileSetType    FileSetType
	DeleteIfExists bool
	// Snapshot options are applicable to snapshots (index yes, data yes)
	Snapshot DataPrepareSnapshotOptions
}

DataPrepareOptions is the options struct for the DataFlush's Prepare method. nolint: maligned

type DataPrepareSnapshotOptions 

type DataPrepareSnapshotOptions struct {
	SnapshotTime xtime.UnixNano
	SnapshotID   uuid.UUID
}

DataPrepareSnapshotOptions is the options struct for the Prepare method that contains information specific to read/writing snapshot files.

type DeferCloser added in v0.15.3 

type DeferCloser func() (DataCloser, error)

DeferCloser returns a DataCloser that persists the data checkpoint file when called.

type FileSetContentType 

type FileSetContentType int

FileSetContentType is an enum that indicates what the contents of files a fileset contains 

const (
	// FileSetDataContentType indicates that the fileset files contents is time series data
	FileSetDataContentType FileSetContentType = iota
	// FileSetIndexContentType indicates that the fileset files contain time series index metadata
	FileSetIndexContentType
)

func (FileSetContentType) String 

func (f FileSetContentType) String() string

type FileSetType 

type FileSetType int

FileSetType is an enum that indicates what type of files a fileset contains 

const (
	// FileSetFlushType indicates that the fileset files contain a complete flush
	FileSetFlushType FileSetType = iota
	// FileSetSnapshotType indicates that the fileset files contain a snapshot
	FileSetSnapshotType
)

func (FileSetType) String 

func (f FileSetType) String() string

type FlushPreparer added in v0.7.0 

type FlushPreparer interface {
	Preparer

	// DoneFlush marks the data flush as complete.
	DoneFlush() error
}

FlushPreparer is a persist flush cycle, each shard and block start permutation needs to explicitly be prepared.

type IDAndEncodedTags added in v1.0.1 

type IDAndEncodedTags struct {
	ID          ident.BytesID
	EncodedTags ts.EncodedTags
}

IDAndEncodedTags contains a series ID and encoded tags.

type IndexCloser 

type IndexCloser func() ([]segment.Segment, error)

IndexCloser is a function that performs cleanup after persisting the index data block for a (namespace, blockStart) combination and returns the corresponding immutable Segment.

type IndexFlush 

type IndexFlush interface {
	// Prepare prepares writing data for a given ns/blockStart, returning a
	// PreparedIndexPersist object and any error encountered during
	// preparation if any.
	PrepareIndex(opts IndexPrepareOptions) (PreparedIndexPersist, error)

	// DoneIndex marks the index flush as complete.
	DoneIndex() error
}

IndexFlush is a persist flush cycle, each namespace, block combination needs to explicitly be prepared.

type IndexFn 

type IndexFn func(segment.Builder) error

IndexFn is a function that persists a m3ninx MutableSegment.

type IndexPrepareOptions 

type IndexPrepareOptions struct {
	NamespaceMetadata namespace.Metadata
	BlockStart        xtime.UnixNano
	FileSetType       FileSetType
	Shards            map[uint32]struct{}
	IndexVolumeType   idxpersist.IndexVolumeType
	VolumeIndex       int
}

IndexPrepareOptions is the options struct for the IndexFlush's Prepare method. nolint: maligned

type Manager 

type Manager interface {
	// StartFlushPersist begins a data flush for a set of shards.
	StartFlushPersist() (FlushPreparer, error)

	// StartSnapshotPersist begins a snapshot for a set of shards.
	StartSnapshotPersist(snapshotID uuid.UUID) (SnapshotPreparer, error)

	// StartIndexPersist begins a flush for index data.
	StartIndexPersist() (IndexFlush, error)

	Close()
}

Manager manages the internals of persisting data onto storage layer.

type Metadata added in v0.15.4 

type Metadata struct {
	// contains filtered or unexported fields
}

Metadata is metadata for a time series, it can have several underlying sources.

func NewMetadata added in v0.15.4 

func NewMetadata(metadata doc.Metadata) Metadata

NewMetadata returns a new metadata struct from series metadata. Note: because doc.Metadata has no pools for finalization we do not take MetadataOptions here, in future if we have pools or some other shared options that Metadata needs we will add it to this constructor as well.

func NewMetadataFromIDAndTagIterator added in v0.15.4 

func NewMetadataFromIDAndTagIterator(
	id ident.ID,
	tagsIter ident.TagIterator,
	opts MetadataOptions,
) Metadata

NewMetadataFromIDAndTagIterator returns a new metadata struct from explicit ID and tag iterator.

func NewMetadataFromIDAndTags added in v0.15.4 

func NewMetadataFromIDAndTags(
	id ident.ID,
	tags ident.Tags,
	opts MetadataOptions,
) Metadata

NewMetadataFromIDAndTags returns a new metadata struct from explicit ID and tags.

func (Metadata) BytesID added in v0.15.4 

func (m Metadata) BytesID() []byte

BytesID returns the bytes ID of the series.

func (Metadata) Finalize added in v0.15.4 

func (m Metadata) Finalize()

Finalize will finalize any resources that requested to be finalized.

func (Metadata) ResetOrReturnProvidedTagIterator added in v0.15.4 

func (m Metadata) ResetOrReturnProvidedTagIterator(
	reusableTagsIterator ident.TagsIterator,
) (ident.TagIterator, error)

ResetOrReturnProvidedTagIterator returns a tag iterator for the series, returning a direct ref to a provided tag iterator or using the reusable tag iterator provided by the callsite if it needs to iterate over tags or fields.

type MetadataOptions added in v0.15.4 

type MetadataOptions struct {
	FinalizeID          bool
	FinalizeTags        bool
	FinalizeTagIterator bool
}

MetadataOptions is options to use when creating metadata.

type MockFlushPreparer added in v0.7.0 

type MockFlushPreparer struct {
	// contains filtered or unexported fields
}

MockFlushPreparer is a mock of FlushPreparer interface.

func NewMockFlushPreparer added in v0.7.0 

func NewMockFlushPreparer(ctrl *gomock.Controller) *MockFlushPreparer

NewMockFlushPreparer creates a new mock instance.

func (*MockFlushPreparer) DoneFlush added in v0.7.0 

func (m *MockFlushPreparer) DoneFlush() error

DoneFlush mocks base method.

func (*MockFlushPreparer) EXPECT added in v0.7.0 

func (m *MockFlushPreparer) EXPECT() *MockFlushPreparerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockFlushPreparer) PrepareData added in v0.7.0 

func (m *MockFlushPreparer) PrepareData(opts DataPrepareOptions) (PreparedDataPersist, error)

PrepareData mocks base method.

type MockFlushPreparerMockRecorder added in v0.7.0 

type MockFlushPreparerMockRecorder struct {
	// contains filtered or unexported fields
}

MockFlushPreparerMockRecorder is the mock recorder for MockFlushPreparer.

func (*MockFlushPreparerMockRecorder) DoneFlush added in v0.7.0 

func (mr *MockFlushPreparerMockRecorder) DoneFlush() *gomock.Call

DoneFlush indicates an expected call of DoneFlush.

func (*MockFlushPreparerMockRecorder) PrepareData added in v0.7.0 

func (mr *MockFlushPreparerMockRecorder) PrepareData(opts interface{}) *gomock.Call

PrepareData indicates an expected call of PrepareData.

type MockIndexFlush 

type MockIndexFlush struct {
	// contains filtered or unexported fields
}

MockIndexFlush is a mock of IndexFlush interface.

func NewMockIndexFlush 

func NewMockIndexFlush(ctrl *gomock.Controller) *MockIndexFlush

NewMockIndexFlush creates a new mock instance.

func (*MockIndexFlush) DoneIndex 

func (m *MockIndexFlush) DoneIndex() error

DoneIndex mocks base method.

func (*MockIndexFlush) EXPECT 

func (m *MockIndexFlush) EXPECT() *MockIndexFlushMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockIndexFlush) PrepareIndex 

func (m *MockIndexFlush) PrepareIndex(opts IndexPrepareOptions) (PreparedIndexPersist, error)

PrepareIndex mocks base method.

type MockIndexFlushMockRecorder 

type MockIndexFlushMockRecorder struct {
	// contains filtered or unexported fields
}

MockIndexFlushMockRecorder is the mock recorder for MockIndexFlush.

func (*MockIndexFlushMockRecorder) DoneIndex 

func (mr *MockIndexFlushMockRecorder) DoneIndex() *gomock.Call

DoneIndex indicates an expected call of DoneIndex.

func (*MockIndexFlushMockRecorder) PrepareIndex 

func (mr *MockIndexFlushMockRecorder) PrepareIndex(opts interface{}) *gomock.Call

PrepareIndex indicates an expected call of PrepareIndex.

type MockManager 

type MockManager struct {
	// contains filtered or unexported fields
}

MockManager is a mock of Manager interface.

func NewMockManager 

func NewMockManager(ctrl *gomock.Controller) *MockManager

NewMockManager creates a new mock instance.

func (*MockManager) Close added in v0.15.0 

func (m *MockManager) Close()

Close mocks base method.

func (*MockManager) EXPECT 

func (m *MockManager) EXPECT() *MockManagerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockManager) StartFlushPersist added in v0.7.0 

func (m *MockManager) StartFlushPersist() (FlushPreparer, error)

StartFlushPersist mocks base method.

func (*MockManager) StartIndexPersist 

func (m *MockManager) StartIndexPersist() (IndexFlush, error)

StartIndexPersist mocks base method.

func (*MockManager) StartSnapshotPersist added in v0.7.0 

func (m *MockManager) StartSnapshotPersist(snapshotID uuid.UUID) (SnapshotPreparer, error)

StartSnapshotPersist mocks base method.

type MockManagerMockRecorder 

type MockManagerMockRecorder struct {
	// contains filtered or unexported fields
}

MockManagerMockRecorder is the mock recorder for MockManager.

func (*MockManagerMockRecorder) Close added in v0.15.0 

func (mr *MockManagerMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close.

func (*MockManagerMockRecorder) StartFlushPersist added in v0.7.0 

func (mr *MockManagerMockRecorder) StartFlushPersist() *gomock.Call

StartFlushPersist indicates an expected call of StartFlushPersist.

func (*MockManagerMockRecorder) StartIndexPersist 

func (mr *MockManagerMockRecorder) StartIndexPersist() *gomock.Call

StartIndexPersist indicates an expected call of StartIndexPersist.

func (*MockManagerMockRecorder) StartSnapshotPersist added in v0.7.0 

func (mr *MockManagerMockRecorder) StartSnapshotPersist(snapshotID interface{}) *gomock.Call

StartSnapshotPersist indicates an expected call of StartSnapshotPersist.

type MockOnFlushSeries added in v0.15.0 

type MockOnFlushSeries struct {
	// contains filtered or unexported fields
}

MockOnFlushSeries is a mock of OnFlushSeries interface.

func NewMockOnFlushSeries added in v0.15.0 

func NewMockOnFlushSeries(ctrl *gomock.Controller) *MockOnFlushSeries

NewMockOnFlushSeries creates a new mock instance.

func (*MockOnFlushSeries) CheckpointAndMaybeCompact added in v1.0.1 

func (m *MockOnFlushSeries) CheckpointAndMaybeCompact() error

CheckpointAndMaybeCompact mocks base method.

func (*MockOnFlushSeries) EXPECT added in v0.15.0 

func (m *MockOnFlushSeries) EXPECT() *MockOnFlushSeriesMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockOnFlushSeries) OnFlushNewSeries added in v0.15.0 

func (m *MockOnFlushSeries) OnFlushNewSeries(arg0 OnFlushNewSeriesEvent) error

OnFlushNewSeries mocks base method.

type MockOnFlushSeriesMockRecorder added in v0.15.0 

type MockOnFlushSeriesMockRecorder struct {
	// contains filtered or unexported fields
}

MockOnFlushSeriesMockRecorder is the mock recorder for MockOnFlushSeries.

func (*MockOnFlushSeriesMockRecorder) CheckpointAndMaybeCompact added in v1.0.1 

func (mr *MockOnFlushSeriesMockRecorder) CheckpointAndMaybeCompact() *gomock.Call

CheckpointAndMaybeCompact indicates an expected call of CheckpointAndMaybeCompact.

func (*MockOnFlushSeriesMockRecorder) OnFlushNewSeries added in v0.15.0 

func (mr *MockOnFlushSeriesMockRecorder) OnFlushNewSeries(arg0 interface{}) *gomock.Call

OnFlushNewSeries indicates an expected call of OnFlushNewSeries.

type MockPreparer added in v0.7.0 

type MockPreparer struct {
	// contains filtered or unexported fields
}

MockPreparer is a mock of Preparer interface.

func NewMockPreparer added in v0.7.0 

func NewMockPreparer(ctrl *gomock.Controller) *MockPreparer

NewMockPreparer creates a new mock instance.

func (*MockPreparer) EXPECT added in v0.7.0 

func (m *MockPreparer) EXPECT() *MockPreparerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockPreparer) PrepareData added in v0.7.0 

func (m *MockPreparer) PrepareData(opts DataPrepareOptions) (PreparedDataPersist, error)

PrepareData mocks base method.

type MockPreparerMockRecorder added in v0.7.0 

type MockPreparerMockRecorder struct {
	// contains filtered or unexported fields
}

MockPreparerMockRecorder is the mock recorder for MockPreparer.

func (*MockPreparerMockRecorder) PrepareData added in v0.7.0 

func (mr *MockPreparerMockRecorder) PrepareData(opts interface{}) *gomock.Call

PrepareData indicates an expected call of PrepareData.

type MockSnapshotPreparer added in v0.7.0 

type MockSnapshotPreparer struct {
	// contains filtered or unexported fields
}

MockSnapshotPreparer is a mock of SnapshotPreparer interface.

func NewMockSnapshotPreparer added in v0.7.0 

func NewMockSnapshotPreparer(ctrl *gomock.Controller) *MockSnapshotPreparer

NewMockSnapshotPreparer creates a new mock instance.

func (*MockSnapshotPreparer) DoneSnapshot added in v0.7.0 

func (m *MockSnapshotPreparer) DoneSnapshot(snapshotUUID uuid.UUID, commitLogIdentifier CommitLogFile) error

DoneSnapshot mocks base method.

func (*MockSnapshotPreparer) EXPECT added in v0.7.0 

func (m *MockSnapshotPreparer) EXPECT() *MockSnapshotPreparerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockSnapshotPreparer) PrepareData added in v0.7.0 

func (m *MockSnapshotPreparer) PrepareData(opts DataPrepareOptions) (PreparedDataPersist, error)

PrepareData mocks base method.

type MockSnapshotPreparerMockRecorder added in v0.7.0 

type MockSnapshotPreparerMockRecorder struct {
	// contains filtered or unexported fields
}

MockSnapshotPreparerMockRecorder is the mock recorder for MockSnapshotPreparer.

func (*MockSnapshotPreparerMockRecorder) DoneSnapshot added in v0.7.0 

func (mr *MockSnapshotPreparerMockRecorder) DoneSnapshot(snapshotUUID, commitLogIdentifier interface{}) *gomock.Call

DoneSnapshot indicates an expected call of DoneSnapshot.

func (*MockSnapshotPreparerMockRecorder) PrepareData added in v0.7.0 

func (mr *MockSnapshotPreparerMockRecorder) PrepareData(opts interface{}) *gomock.Call

PrepareData indicates an expected call of PrepareData.

type NoOpColdFlushNamespace added in v0.15.0 

type NoOpColdFlushNamespace struct{}

NoOpColdFlushNamespace is a no-op impl of OnFlushSeries.

func (*NoOpColdFlushNamespace) Abort added in v1.2.0 

func (n *NoOpColdFlushNamespace) Abort() error

Abort is a no-op.

func (*NoOpColdFlushNamespace) CheckpointAndMaybeCompact added in v1.0.1 

func (n *NoOpColdFlushNamespace) CheckpointAndMaybeCompact() error

CheckpointAndMaybeCompact is a no-op.

func (*NoOpColdFlushNamespace) Done added in v0.15.0 

func (n *NoOpColdFlushNamespace) Done() error

Done is a no-op.

func (*NoOpColdFlushNamespace) OnFlushNewSeries added in v0.15.0 

func (n *NoOpColdFlushNamespace) OnFlushNewSeries(event OnFlushNewSeriesEvent) error

OnFlushNewSeries is a no-op.

type OnFlushNewSeriesEvent added in v0.15.0 

type OnFlushNewSeriesEvent struct {
	Shard          uint32
	BlockStart     xtime.UnixNano
	FirstWrite     xtime.UnixNano
	SeriesMetadata SeriesMetadata
}

OnFlushNewSeriesEvent is the fields related to a flush of a new series.

type OnFlushSeries added in v0.15.0 

type OnFlushSeries interface {
	OnFlushNewSeries(OnFlushNewSeriesEvent) error

	// CheckpointAndMaybeCompact checks to see if we're at maximum cardinality
	// for any index segments we're currently building and compact if we are.
	CheckpointAndMaybeCompact() error
}

OnFlushSeries performs work on a per series level. Also exposes a checkpoint fn for maybe compacting multiple index segments based on size.

type PreparedDataPersist 

type PreparedDataPersist struct {
	Persist    DataFn
	Close      DataCloser
	DeferClose DeferCloser
}

PreparedDataPersist is an object that wraps holds a persist function and a closer.

type PreparedIndexPersist 

type PreparedIndexPersist struct {
	Persist IndexFn
	Close   IndexCloser
}

PreparedIndexPersist is an object that wraps holds a persist function and a closer.

type Preparer added in v0.7.0 

type Preparer interface {
	// Prepare prepares writing data for a given (shard, blockStart) combination,
	// returning a PreparedDataPersist object and any error encountered during
	// preparation if any.
	PrepareData(opts DataPrepareOptions) (PreparedDataPersist, error)
}

Preparer can generate a PreparedDataPersist object for writing data for a given (shard, blockstart) combination.

type SeriesMetadata added in v1.0.1 

type SeriesMetadata struct {
	Document         doc.Metadata
	IDAndEncodedTags IDAndEncodedTags
	Type             SeriesMetadataType
	LifeTime         SeriesMetadataLifeTime
}

SeriesMetadata captures different representations of series metadata and the ownership status of the underlying memory.

type SeriesMetadataLifeTime added in v1.0.1 

type SeriesMetadataLifeTime uint8

SeriesMetadataLifeTime describes the memory life time type. 

const (
	// SeriesLifeTimeLong means the underlying memory's life time is long lived and exceeds
	// the execution duration of the series metadata receiver.
	SeriesLifeTimeLong SeriesMetadataLifeTime = iota
	// SeriesLifeTimeShort means that the underlying memory is only valid for the duration
	// of the OnFlushNewSeries call. Must clone the underlying bytes in order to extend the life time.
	SeriesLifeTimeShort
)

type SeriesMetadataType added in v1.0.1 

type SeriesMetadataType uint8

SeriesMetadataType describes the type of series metadata. 

const (
	// SeriesDocumentType means the metadata is in doc.Metadata form.
	SeriesDocumentType SeriesMetadataType = iota
	// SeriesIDAndEncodedTagsType means the metadata is in IDAndEncodedTags form.
	SeriesIDAndEncodedTagsType
)

type SnapshotPreparer added in v0.7.0 

type SnapshotPreparer interface {
	Preparer

	// DoneSnapshot marks the snapshot as complete.
	DoneSnapshot(snapshotUUID uuid.UUID, commitLogIdentifier CommitLogFile) error
}

SnapshotPreparer is a persist snapshot cycle, each shard and block start permutation needs to explicitly be prepared.

Source Files

View all Source files

Directories

Path Synopsis
fs
Package fs is a generated GoMock package.
 Package fs is a generated GoMock package.
clone
commitlog
Package commitlog is a generated GoMock package.
 Package commitlog is a generated GoMock package.
migration
msgpack

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.