fs

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 21, 2021 License: Apache-2.0 Imports: 60 Imported by: 29

Documentation

Overview

Package fs is a generated GoMock package.

Index

Constants

View Source 
const (
	// CheckpointFileSizeBytes is the expected size of a valid checkpoint file.
	CheckpointFileSizeBytes = 4
)

Variables

View Source 
var (
	// ErrCheckpointFileNotFound returned when the checkpoint file doesn't exist
	ErrCheckpointFileNotFound = errors.New("checkpoint file does not exist")
)
View Source 
var (

	// ErrOutOfRetentionClaim returned when reserving a claim that is
	// out of retention.
	ErrOutOfRetentionClaim = errors.New("out of retention index volume claim")
)

Functions

func CommitLogFilePath added in v0.7.0 

func CommitLogFilePath(prefix string, index int) string

CommitLogFilePath returns the path for a commitlog file.

func CommitLogsDirPath 

func CommitLogsDirPath(prefix string) string

CommitLogsDirPath returns the path to commit logs.

func CompleteCheckpointFileExists added in v0.4.8 

func CompleteCheckpointFileExists(filePath string) (bool, error)

CompleteCheckpointFileExists returns whether a checkpoint file exists, and if so, is it complete.

func DataDirPath 

func DataDirPath(prefix string) string

DataDirPath returns the path to the data directory belonging to a db

func DataFileSetExists added in v0.11.0 

func DataFileSetExists(
	filePathPrefix string,
	namespace ident.ID,
	shard uint32,
	blockStart time.Time,
	volume int,
) (bool, error)

DataFileSetExists determines whether data fileset files exist for the given namespace, shard, block start, and volume.

func DataFileSetsBefore 

func DataFileSetsBefore(filePathPrefix string, namespace ident.ID, shard uint32, t time.Time) ([]string, error)

DataFileSetsBefore returns all the flush data fileset paths whose timestamps are earlier than a given time.

func DeleteDirectories 

func DeleteDirectories(dirPaths []string) error

DeleteDirectories delets a set of directories and its contents, returning all of the errors encountered during the deletion process.

func DeleteFileSetAt 

func DeleteFileSetAt(filePathPrefix string, namespace ident.ID, shard uint32, blockStart time.Time, volume int) error

DeleteFileSetAt deletes a FileSetFile for a given namespace/shard/blockStart/volume combination if it exists.

func DeleteFiles 

func DeleteFiles(filePaths []string) error

DeleteFiles delete a set of files, returning all the errors encountered during the deletion process.

func DeleteInactiveDirectories 

func DeleteInactiveDirectories(parentDirectoryPath string, activeDirectories []string) error

DeleteInactiveDirectories deletes any directories that are not currently active, as defined by the inputed active directories within the parent directory

func FileExists 

func FileExists(filePath string) (bool, error)

FileExists returns whether a file at the given path exists.

func FilesBefore 

func FilesBefore(files []string, t time.Time) ([]string, error)

FilesBefore filters the list of files down to those whose name indicate they are before a given time period. Mutates the provided slice.

func IndexDataDirPath added in v0.15.0 

func IndexDataDirPath(prefix string) string

IndexDataDirPath returns the path to the index data directory belonging to a db

func IndexFileSetsBefore 

func IndexFileSetsBefore(filePathPrefix string, namespace ident.ID, t time.Time) ([]string, error)

IndexFileSetsBefore returns all the flush index fileset paths whose timestamps are earlier than a given time.

func NamespaceDataDirPath 

func NamespaceDataDirPath(prefix string, namespace ident.ID) string

NamespaceDataDirPath returns the path to the data directory for a given namespace.

func NamespaceIndexDataDirPath 

func NamespaceIndexDataDirPath(prefix string, namespace ident.ID) string

NamespaceIndexDataDirPath returns the path to the data directory for a given namespace.

func NamespaceIndexSnapshotDirPath 

func NamespaceIndexSnapshotDirPath(prefix string, namespace ident.ID) string

NamespaceIndexSnapshotDirPath returns the path to the data directory for a given namespace.

func NamespaceSnapshotsDirPath 

func NamespaceSnapshotsDirPath(prefix string, namespace ident.ID) string

NamespaceSnapshotsDirPath returns the path to the snapshots directory for a given namespace.

func NewPersistManager 

func NewPersistManager(opts Options) (persist.Manager, error)

NewPersistManager creates a new filesystem persist manager

func NextIndexFileSetVolumeIndex 

func NextIndexFileSetVolumeIndex(filePathPrefix string, namespace ident.ID, blockStart time.Time) (int, error)

NextIndexFileSetVolumeIndex returns the next index file set index for a given namespace/blockStart combination.

func NextIndexSnapshotFileIndex 

func NextIndexSnapshotFileIndex(filePathPrefix string, namespace ident.ID, blockStart time.Time) (int, error)

NextIndexSnapshotFileIndex returns the next snapshot file index for a given namespace/shard/blockStart combination.

func NextSnapshotFileSetVolumeIndex 

func NextSnapshotFileSetVolumeIndex(filePathPrefix string, namespace ident.ID, shard uint32, blockStart time.Time) (int, error)

NextSnapshotFileSetVolumeIndex returns the next snapshot file set index for a given namespace/shard/blockStart combination.

func NextSnapshotMetadataFileIndex added in v0.4.8 

func NextSnapshotMetadataFileIndex(opts Options) (int64, error)

NextSnapshotMetadataFileIndex returns the next snapshot metadata file index.

func OpenWritable 

func OpenWritable(filePath string, perm os.FileMode) (*os.File, error)

OpenWritable opens a file for writing and truncating as necessary.

func ResetIndexClaimsManagersUnsafe added in v1.0.0 

func ResetIndexClaimsManagersUnsafe()

ResetIndexClaimsManagersUnsafe should only be used from tests or integration tests, it resets the count of index claim managers to allow new claim managers to be created. By default this is restricted to just once instantiation since otherwise concurrency issues can be skipped without realizing.

func ShardDataDirPath 

func ShardDataDirPath(prefix string, namespace ident.ID, shard uint32) string

ShardDataDirPath returns the path to the data directory for a given shard.

func ShardSnapshotsDirPath 

func ShardSnapshotsDirPath(prefix string, namespace ident.ID, shard uint32) string

ShardSnapshotsDirPath returns the path to the snapshots directory for a given shard.

func SnapshotDirPath 

func SnapshotDirPath(prefix string) string

SnapshotDirPath returns the path to the snapshot directory belong to a db

func SnapshotFileSetExistsAt 

func SnapshotFileSetExistsAt(
	prefix string,
	namespace ident.ID,
	snapshotID uuid.UUID,
	shard uint32,
	blockStart time.Time,
) (bool, error)

SnapshotFileSetExistsAt determines whether snapshot fileset files exist for the given namespace, shard, and block start time.

func SnapshotTimeAndID added in v0.5.0 

func SnapshotTimeAndID(
	filePathPrefix string, id FileSetFileIdentifier) (time.Time, uuid.UUID, error)

SnapshotTimeAndID returns the metadata for the snapshot.

func SnapshotsDirPath added in v0.4.8 

func SnapshotsDirPath(prefix string) string

SnapshotsDirPath returns the path to the snapshots directory.

func SortedCommitLogFiles 

func SortedCommitLogFiles(commitLogsDir string) ([]string, error)

SortedCommitLogFiles returns all the commit log files in the commit logs directory.

func SortedSnapshotMetadataFiles added in v0.4.8 

func SortedSnapshotMetadataFiles(opts Options) (
	[]SnapshotMetadata, []SnapshotMetadataErrorWithPaths, error)

SortedSnapshotMetadataFiles returns a slice of all the SnapshotMetadata files that are on disk, as well as any files that it encountered errors for (corrupt, missing checkpoints, etc) which facilitates cleanup of corrupt files. []SnapshotMetadata will be sorted by index (i.e the chronological order in which the snapshots were taken), but []SnapshotMetadataErrorWithPaths will not be in any particular order.

func TimeAndIndexFromCommitlogFilename 

func TimeAndIndexFromCommitlogFilename(fname string) (time.Time, int, error)

TimeAndIndexFromCommitlogFilename extracts the block start and index from file name for a commitlog.

func TimeAndVolumeIndexFromDataFileSetFilename added in v0.11.0 

func TimeAndVolumeIndexFromDataFileSetFilename(fname string) (time.Time, int, error)

TimeAndVolumeIndexFromDataFileSetFilename extracts the block start and volume index from a data fileset file name that may or may not have an index. If the file name does not include an index, unindexedFilesetIndex is returned as the volume index.

func TimeAndVolumeIndexFromFileSetFilename 

func TimeAndVolumeIndexFromFileSetFilename(fname string) (time.Time, int, error)

TimeAndVolumeIndexFromFileSetFilename extracts the block start and volume index from an index file name.

func TimeFromFileName 

func TimeFromFileName(fname string) (time.Time, error)

TimeFromFileName extracts the block start time from file name.

Types

type BlockRetrieverOptions 

type BlockRetrieverOptions interface {
	// Validate validates the options.
	Validate() error

	// SetRetrieveRequestPool sets the retrieve request pool.
	SetRetrieveRequestPool(value RetrieveRequestPool) BlockRetrieverOptions

	// RetrieveRequestPool returns the retrieve request pool.
	RetrieveRequestPool() RetrieveRequestPool

	// SetBytesPool sets the bytes pool.
	SetBytesPool(value pool.CheckedBytesPool) BlockRetrieverOptions

	// BytesPool returns the bytes pool.
	BytesPool() pool.CheckedBytesPool

	// SetFetchConcurrency sets the fetch concurrency.
	SetFetchConcurrency(value int) BlockRetrieverOptions

	// FetchConcurrency returns the fetch concurrency.
	FetchConcurrency() int

	// SetCacheBlocksOnRetrieve sets whether to cache blocks after retrieval at a global level.
	SetCacheBlocksOnRetrieve(value bool) BlockRetrieverOptions

	// CacheBlocksOnRetrieve returns whether to cache blocks after retrieval at a global level.
	CacheBlocksOnRetrieve() bool

	// SetIdentifierPool sets the identifierPool.
	SetIdentifierPool(value ident.Pool) BlockRetrieverOptions

	// IdentifierPool returns the identifierPool.
	IdentifierPool() ident.Pool

	// SetBlockLeaseManager sets the block leaser.
	SetBlockLeaseManager(leaseMgr block.LeaseManager) BlockRetrieverOptions

	// BlockLeaseManager returns the block leaser.
	BlockLeaseManager() block.LeaseManager

	// SetQueryLimits sets query limits.
	SetQueryLimits(value limits.QueryLimits) BlockRetrieverOptions

	// QueryLimits returns the query limits.
	QueryLimits() limits.QueryLimits
}

BlockRetrieverOptions represents the options for block retrieval.

func NewBlockRetrieverOptions 

func NewBlockRetrieverOptions() BlockRetrieverOptions

NewBlockRetrieverOptions creates a new set of block retriever options

type ConcurrentDataFileSetSeeker 

type ConcurrentDataFileSetSeeker interface {
	io.Closer

	// SeekByID is the same as in DataFileSetSeeker.
	SeekByID(id ident.ID, resources ReusableSeekerResources) (data checked.Bytes, err error)

	// SeekByIndexEntry is the same as in DataFileSetSeeker.
	SeekByIndexEntry(entry IndexEntry, resources ReusableSeekerResources) (checked.Bytes, error)

	// SeekIndexEntry is the same as in DataFileSetSeeker.
	SeekIndexEntry(id ident.ID, resources ReusableSeekerResources) (IndexEntry, error)

	// SeekWideEntry is the same as in DataFileSetSeeker.
	SeekWideEntry(
		id ident.ID,
		filter schema.WideEntryFilter,
		resources ReusableSeekerResources,
	) (xio.WideEntry, error)

	// ConcurrentIDBloomFilter is the same as in DataFileSetSeeker.
	ConcurrentIDBloomFilter() *ManagedConcurrentBloomFilter
}

ConcurrentDataFileSetSeeker is a limited interface that is returned when ConcurrentClone() is called on DataFileSetSeeker. A seeker is essentially a wrapper around file descriptors around a set of files, allowing for interaction with them. We can ask a seeker for a specific time series, which will then be streamed out from the according data file. The clones can be used together concurrently and share underlying resources. Clones are no longer usable once the original has been closed.

type DataBlockRetriever 

type DataBlockRetriever interface {
	io.Closer
	block.DatabaseBlockRetriever

	// Open the block retriever to retrieve from a namespace.
	Open(
		md namespace.Metadata,
		shardSet sharding.ShardSet,
	) error
}

DataBlockRetriever provides a block retriever for TSDB file sets.

func NewBlockRetriever 

func NewBlockRetriever(
	opts BlockRetrieverOptions,
	fsOpts Options,
) (DataBlockRetriever, error)

NewBlockRetriever returns a new block retriever for TSDB file sets.

type DataEntryProcessor added in v1.0.1 

type DataEntryProcessor interface {
	// SetEntriesCount sets the number of entries to be processed.
	SetEntriesCount(int)
	// ProcessEntry processes a single StreamedDataEntry.
	ProcessEntry(StreamedDataEntry) error
}

DataEntryProcessor processes StreamedDataEntries.

type DataFileSetReader 

type DataFileSetReader interface {
	io.Closer

	// Open opens the files for the given shard and version for reading
	Open(opts DataReaderOpenOptions) error

	// Status returns the status of the reader
	Status() DataFileSetReaderStatus

	// Read returns the next id, tags, data, checksum tuple or error, will return io.EOF at end of volume.
	// Use either Read or ReadMetadata to progress through a volume, but not both.
	// Note: make sure to finalize the ID, close the Tags and finalize the Data when done with
	// them so they can be returned to their respective pools.
	Read() (id ident.ID, tags ident.TagIterator, data checked.Bytes, checksum uint32, err error)

	// StreamingRead returns the next unpooled id, encodedTags, data, checksum
	// values ordered by id, or error, will return io.EOF at end of volume.
	// Can only by used when DataReaderOpenOptions.StreamingEnabled is true.
	// Note: the returned data gets invalidated on the next call to StreamingRead.
	StreamingRead() (StreamedDataEntry, error)

	// ReadMetadata returns the next id and metadata or error, will return io.EOF at end of volume.
	// Use either Read or ReadMetadata to progress through a volume, but not both.
	// Note: make sure to finalize the ID, and close the Tags when done with them so they can
	// be returned to their respective pools.
	ReadMetadata() (id ident.ID, tags ident.TagIterator, length int, checksum uint32, err error)

	// ReadBloomFilter returns the bloom filter stored on disk in a container object that is safe
	// for concurrent use and has a Close() method for releasing resources when done.
	ReadBloomFilter() (*ManagedConcurrentBloomFilter, error)

	// Validate validates both the metadata and data and returns an error if either is corrupted.
	Validate() error

	// ValidateMetadata validates the data and returns an error if the data is corrupted.
	ValidateMetadata() error

	// ValidateData validates the data and returns an error if the data is corrupted.
	ValidateData() error

	// Range returns the time range associated with data in the volume.
	Range() xtime.Range

	// Entries returns the count of entries in the volume.
	Entries() int

	// EntriesRead returns the position read into the volume.
	EntriesRead() int

	// MetadataRead returns the position of metadata read into the volume.
	MetadataRead() int

	// StreamingEnabled returns true if the reader is opened in streaming mode.
	StreamingEnabled() bool
}

DataFileSetReader provides an unsynchronized reader for a TSDB file set.

func NewReader 

func NewReader(
	bytesPool pool.CheckedBytesPool,
	opts Options,
) (DataFileSetReader, error)

NewReader returns a new reader and expects all files to exist. Will read the index info in full on call to Open. The bytesPool can be passed as nil if callers would prefer just dynamically allocated IDs and data.

type DataFileSetReaderStatus 

type DataFileSetReaderStatus struct {
	Namespace  ident.ID
	BlockStart time.Time
	Shard      uint32
	Volume     int
	Open       bool
	BlockSize  time.Duration
}

DataFileSetReaderStatus describes the status of a file set reader.

type DataFileSetSeeker 

type DataFileSetSeeker interface {
	io.Closer

	// Open opens the files for the given shard and version for reading.
	Open(
		namespace ident.ID,
		shard uint32,
		start time.Time,
		volume int,
		resources ReusableSeekerResources,
	) error

	// SeekByID returns the data for specified ID provided the index was loaded upon open. An
	// error will be returned if the index was not loaded or ID cannot be found.
	SeekByID(id ident.ID, resources ReusableSeekerResources) (data checked.Bytes, err error)

	// SeekByIndexEntry is similar to Seek, but uses an IndexEntry instead of
	// looking it up on its own. Useful in cases where you've already obtained an
	// entry and don't want to waste resources looking it up again.
	SeekByIndexEntry(entry IndexEntry, resources ReusableSeekerResources) (checked.Bytes, error)

	// SeekIndexEntry returns the IndexEntry for the specified ID. This can be useful
	// ahead of issuing a number of seek requests so that the seek requests can be
	// made in order. The returned IndexEntry can also be passed to SeekByIndexEntry
	// to prevent duplicate index lookups.
	SeekIndexEntry(id ident.ID, resources ReusableSeekerResources) (IndexEntry, error)

	// SeekWideEntry seeks in a manner similar to SeekIndexEntry, but
	// instead yields a wide entry checksum of the series.
	SeekWideEntry(
		id ident.ID,
		filter schema.WideEntryFilter,
		resources ReusableSeekerResources,
	) (xio.WideEntry, error)

	// Range returns the time range associated with data in the volume
	Range() xtime.Range

	// ConcurrentIDBloomFilter returns a concurrency-safe bloom filter that can
	// be used to quickly disqualify ID's that definitely do not exist. I.E if the
	// Test() method returns true, the ID may exist on disk, but if it returns
	// false, it definitely does not.
	ConcurrentIDBloomFilter() *ManagedConcurrentBloomFilter

	// ConcurrentClone clones a seeker, creating a copy that uses the same underlying resources
	// (mmaps), but that is capable of seeking independently. The original can continue
	// to be used after the clones are closed, but the clones cannot be used after the
	// original is closed.
	ConcurrentClone() (ConcurrentDataFileSetSeeker, error)
}

DataFileSetSeeker provides an out of order reader for a TSDB file set.

func NewSeeker 

func NewSeeker(
	filePathPrefix string,
	dataBufferSize int,
	infoBufferSize int,
	bytesPool pool.CheckedBytesPool,
	keepUnreadBuf bool,
	opts Options,
) DataFileSetSeeker

NewSeeker returns a new seeker.

type DataFileSetSeekerManager 

type DataFileSetSeekerManager interface {
	io.Closer

	// Open opens the seekers for a given namespace.
	Open(
		md namespace.Metadata,
		shardSet sharding.ShardSet,
	) error

	// CacheShardIndices will pre-parse the indexes for given shards
	// to improve times when seeking to a block.
	CacheShardIndices(shards []uint32) error

	// AssignShardSet assigns current per ns shardset.
	AssignShardSet(shardSet sharding.ShardSet)

	// Borrow returns an open seeker for a given shard, block start time, and
	// volume.
	Borrow(shard uint32, start time.Time) (ConcurrentDataFileSetSeeker, error)

	// Return returns (closes) an open seeker for a given shard, block start
	// time, and volume.
	Return(shard uint32, start time.Time, seeker ConcurrentDataFileSetSeeker) error

	// Test checks if an ID exists in a concurrent ID bloom filter for a
	// given shard, block, start time and volume.
	Test(id ident.ID, shard uint32, start time.Time) (bool, error)
}

DataFileSetSeekerManager provides management of seekers for a TSDB namespace.

func NewSeekerManager 

func NewSeekerManager(
	bytesPool pool.CheckedBytesPool,
	opts Options,
	blockRetrieverOpts BlockRetrieverOptions,
) DataFileSetSeekerManager

NewSeekerManager returns a new TSDB file set seeker manager.

type DataFileSetWriter 

type DataFileSetWriter interface {
	io.Closer

	// Open opens the files for writing data to the given shard in the given namespace.
	// This method is not thread-safe, so its the callers responsibilities that they never
	// try and write two snapshot files for the same block start at the same time or their
	// will be a race in determining the snapshot file's index.
	Open(opts DataWriterOpenOptions) error

	// Write will write the id and data pair and returns an error on a write error. Callers
	// must not call this method with a given ID more than once.
	Write(metadata persist.Metadata, data checked.Bytes, checksum uint32) error

	// WriteAll will write the id and all byte slices and returns an error on a write error.
	// Callers must not call this method with a given ID more than once.
	WriteAll(metadata persist.Metadata, data []checked.Bytes, checksum uint32) error

	// DeferClose returns a DataCloser that defers writing of a checkpoint file.
	DeferClose() (persist.DataCloser, error)
}

DataFileSetWriter provides an unsynchronized writer for a TSDB file set.

func NewWriter 

func NewWriter(opts Options) (DataFileSetWriter, error)

NewWriter returns a new writer with options.

type DataReaderOpenOptions 

type DataReaderOpenOptions struct {
	// Identifier allows to identify a FileSetFile.
	Identifier FileSetFileIdentifier
	// FileSetType is the file set type.
	FileSetType persist.FileSetType
	// StreamingEnabled enables using streaming methods, such as DataFileSetReader.StreamingRead.
	StreamingEnabled bool
	// NB(bodu): This option can inform the reader to optimize for reading
	// only metadata by not sorting index entries. Setting this option will
	// throw an error if a regular `Read()` is attempted.
	OptimizedReadMetadataOnly bool
}

DataReaderOpenOptions is options struct for the reader open method.

type DataWriterOpenOptions 

type DataWriterOpenOptions struct {
	FileSetType        persist.FileSetType
	FileSetContentType persist.FileSetContentType
	Identifier         FileSetFileIdentifier
	BlockSize          time.Duration
	// Only used when writing snapshot files
	Snapshot DataWriterSnapshotOptions
}

DataWriterOpenOptions is the options struct for the Open method on the DataFileSetWriter.

type DataWriterSnapshotOptions 

type DataWriterSnapshotOptions struct {
	SnapshotTime time.Time
	SnapshotID   []byte
}

DataWriterSnapshotOptions is the options struct for Open method on the DataFileSetWriter that contains information specific to writing snapshot files.

type FileSetFile 

type FileSetFile struct {
	ID                FileSetFileIdentifier
	AbsoluteFilePaths []string

	CachedSnapshotTime              time.Time
	CachedSnapshotID                uuid.UUID
	CachedHasCompleteCheckpointFile LazyEvalBool
	// contains filtered or unexported fields
}

FileSetFile represents a set of FileSet files for a given block start

func FileSetAt 

func FileSetAt(filePathPrefix string, namespace ident.ID, shard uint32, blockStart time.Time, volume int) (FileSetFile, bool, error)

FileSetAt returns a FileSetFile for the given namespace/shard/blockStart/volume combination if it exists.

func NewFileSetFile 

func NewFileSetFile(id FileSetFileIdentifier, filePathPrefix string) FileSetFile

NewFileSetFile creates a new FileSet file

func (*FileSetFile) HasCompleteCheckpointFile added in v0.8.4 

func (f *FileSetFile) HasCompleteCheckpointFile() bool

HasCompleteCheckpointFile returns a bool indicating whether the given set of fileset files has a checkpoint file.

func (*FileSetFile) InfoFilePath added in v0.15.0 

func (f *FileSetFile) InfoFilePath() (string, bool)

InfoFilePath returns the info file path of a filesetfile (if found).

func (FileSetFile) IsZero added in v0.4.0 

func (f FileSetFile) IsZero() bool

IsZero returns whether the FileSetFile is a zero value.

func (*FileSetFile) SnapshotFilepath added in v0.15.0 

func (f *FileSetFile) SnapshotFilepath() (string, bool)

SnapshotFilepath returns the info file path of a filesetfile (if found).

func (*FileSetFile) SnapshotTimeAndID added in v0.5.0 

func (f *FileSetFile) SnapshotTimeAndID() (time.Time, uuid.UUID, error)

SnapshotTimeAndID returns the snapshot time and id for the given FileSetFile. Value is meaningless if the the FileSetFile is a flush instead of a snapshot.

type FileSetFileIdentifier 

type FileSetFileIdentifier struct {
	FileSetContentType persist.FileSetContentType
	Namespace          ident.ID
	BlockStart         time.Time
	// Only required for data content files
	Shard uint32
	// Required for snapshot files (index yes, data yes) and flush files (index yes, data yes)
	VolumeIndex int
}

FileSetFileIdentifier contains all the information required to identify a FileSetFile.

func NewFileSetFileIdentifier added in v0.15.14 

func NewFileSetFileIdentifier(
	namespace ident.ID,
	blockStart time.Time,
	shard uint32,
	volumeIndex int,
) FileSetFileIdentifier

NewFileSetFileIdentifier creates a new FileSetFileIdentifier.

type FileSetFilesSlice 

type FileSetFilesSlice []FileSetFile

FileSetFilesSlice is a slice of FileSetFile

func DataFiles added in v0.11.0 

func DataFiles(filePathPrefix string, namespace ident.ID, shard uint32) (FileSetFilesSlice, error)

DataFiles returns a slice of all the names for all the fileset files for a given namespace and shard combination.

func IndexFileSetsAt 

func IndexFileSetsAt(filePathPrefix string, namespace ident.ID, blockStart time.Time) (FileSetFilesSlice, error)

IndexFileSetsAt returns all FileSetFile(s) for the given namespace/blockStart combination. NB: It returns all complete Volumes found on disk.

func IndexSnapshotFiles 

func IndexSnapshotFiles(filePathPrefix string, namespace ident.ID) (FileSetFilesSlice, error)

IndexSnapshotFiles returns a slice of all the names for all the index fileset files for a given namespace.

func SnapshotFiles 

func SnapshotFiles(filePathPrefix string, namespace ident.ID, shard uint32) (FileSetFilesSlice, error)

SnapshotFiles returns a slice of all the names for all the snapshot files for a given namespace and shard combination.

func (FileSetFilesSlice) Filepaths 

func (f FileSetFilesSlice) Filepaths() []string

Filepaths flattens a slice of FileSetFiles to a single slice of filepaths. All paths returned are absolute.

func (FileSetFilesSlice) LatestVolumeForBlock 

func (f FileSetFilesSlice) LatestVolumeForBlock(blockStart time.Time) (FileSetFile, bool)

LatestVolumeForBlock returns the latest (highest index) FileSetFile in the slice for a given block start that has a complete checkpoint file.

func (FileSetFilesSlice) VolumeExistsForBlock added in v0.11.0 

func (f FileSetFilesSlice) VolumeExistsForBlock(blockStart time.Time, volume int) bool

VolumeExistsForBlock returns whether there is a valid FileSetFile for the given block start and volume index.

type ForEachRemainingFn added in v0.10.0 

type ForEachRemainingFn func(seriesMetadata doc.Metadata, data block.FetchBlockResult) error

ForEachRemainingFn is the function that is run on each of the remaining series of the merge target that did not intersect with the fileset.

type IndexClaimsManager added in v1.0.0 

type IndexClaimsManager interface {
	ClaimNextIndexFileSetVolumeIndex(
		md namespace.Metadata,
		blockStart time.Time,
	) (int, error)
}

IndexClaimsManager manages concurrent claims to volume indices per ns and block start. This allows multiple threads to safely increment the volume index.

func NewIndexClaimsManager added in v1.0.0 

func NewIndexClaimsManager(opts Options) (IndexClaimsManager, error)

NewIndexClaimsManager returns an instance of the index claim manager. This manages concurrent claims for volume indices per ns and block start. NB(bodu): There should be only a single shared index claim manager among all threads writing index data filesets.

type IndexEntry 

type IndexEntry struct {
	Size         uint32
	DataChecksum uint32
	Offset       int64
	EncodedTags  checked.Bytes
}

IndexEntry is an entry from the index file which can be passed to SeekUsingIndexEntry to seek to the data for that entry.

type IndexFileSetReader 

type IndexFileSetReader interface {
	idxpersist.IndexFileSetReader
	io.Closer

	// Open the index file set reader.
	Open(opts IndexReaderOpenOptions) (IndexReaderOpenResult, error)

	// Validate returns whether all checksums were matched as expected,
	// it must be called after reading all the segment file sets otherwise
	// it returns an error.
	Validate() error
}

IndexFileSetReader is an index file set reader.

func NewIndexReader 

func NewIndexReader(opts Options) (IndexFileSetReader, error)

NewIndexReader returns a new index reader with options.

type IndexFileSetWriter 

type IndexFileSetWriter interface {
	idxpersist.IndexFileSetWriter
	io.Closer

	// Open the index file set writer.
	Open(opts IndexWriterOpenOptions) error
}

IndexFileSetWriter is a index file set writer.

func NewIndexWriter 

func NewIndexWriter(opts Options) (IndexFileSetWriter, error)

NewIndexWriter returns a new index writer with options.

type IndexReaderOpenOptions 

type IndexReaderOpenOptions struct {
	Identifier  FileSetFileIdentifier
	FileSetType persist.FileSetType
}

IndexReaderOpenOptions is the index file set reader open options.

type IndexReaderOpenResult 

type IndexReaderOpenResult struct {
	Shards map[uint32]struct{}
}

IndexReaderOpenResult describes the results of opening a index file set volume.

type IndexSegmentFile 

type IndexSegmentFile interface {
	idxpersist.IndexSegmentFileSet
}

IndexSegmentFile is a file in an index segment file set.

type IndexSegmentFileSet 

type IndexSegmentFileSet interface {
	idxpersist.IndexSegmentFileSet
}

IndexSegmentFileSet is an index segment file set.

type IndexSegmentFileSetWriter 

type IndexSegmentFileSetWriter interface {
	idxpersist.IndexSegmentFileSetWriter
}

IndexSegmentFileSetWriter is an index segment file set writer.

type IndexWriterOpenOptions 

type IndexWriterOpenOptions struct {
	Identifier  FileSetFileIdentifier
	BlockSize   time.Duration
	FileSetType persist.FileSetType
	Shards      map[uint32]struct{}
	// Only used when writing snapshot files
	Snapshot        IndexWriterSnapshotOptions
	IndexVolumeType idxpersist.IndexVolumeType
}

IndexWriterOpenOptions is a set of options when opening an index file set writer.

type IndexWriterSnapshotOptions 

type IndexWriterSnapshotOptions struct {
	SnapshotTime time.Time
}

IndexWriterSnapshotOptions is a set of options for writing an index file set snapshot.

type Inspection 

type Inspection struct {
	// SortedCommitLogFiles contains all commitlog filenames that existed
	// before the node began accepting writes.
	SortedCommitLogFiles []string
}

Inspection contains the outcome of a filesystem inspection.

func InspectFilesystem 

func InspectFilesystem(fsOpts Options) (Inspection, error)

InspectFilesystem scans the filesystem and generates a Inspection which the commitlog bootstrapper needs to avoid reading commitlog files that were written *after* the process has already started. I.E in order to distinguish between files that were already on disk before the process started, and those that were written by the process itself once it started accepting writes (but before bootstrapping had complete) we export a function which can be called during node startup.

func (Inspection) CommitLogFilesSet 

func (f Inspection) CommitLogFilesSet() map[string]struct{}

CommitLogFilesSet generates a set of unique commitlog files.

type LazyEvalBool added in v0.8.4 

type LazyEvalBool uint8

LazyEvalBool is a boolean that is lazily evaluated. 

const (
	// EvalNone indicates the boolean has not been evaluated.
	EvalNone LazyEvalBool = iota
	// EvalTrue indicates the boolean has been evaluated to true.
	EvalTrue
	// EvalFalse indicates the boolean has been evaluated to false.
	EvalFalse
)

type ManagedConcurrentBloomFilter 

type ManagedConcurrentBloomFilter struct {
	// contains filtered or unexported fields
}

ManagedConcurrentBloomFilter is a container object that implements lifecycle management on-top of a BloomFilter. I.E it wraps a bloom filter such that all resources are released when the Close() method is called. It's also safe for concurrent access

func (*ManagedConcurrentBloomFilter) Close 

func (bf *ManagedConcurrentBloomFilter) Close() error

Close closes the bloom filter, releasing any held resources

func (*ManagedConcurrentBloomFilter) K 

func (bf *ManagedConcurrentBloomFilter) K() uint

K returns the number of hash functions in the bloom filter

func (*ManagedConcurrentBloomFilter) M 

func (bf *ManagedConcurrentBloomFilter) M() uint

M returns the number of elements in the bloom filter

func (*ManagedConcurrentBloomFilter) Test 

func (bf *ManagedConcurrentBloomFilter) Test(value []byte) bool

Test tests whether a value is in the bloom filter

type MergeWith added in v0.10.0 

type MergeWith interface {
	// Read returns the data for the given block start and series ID, whether
	// any data was found, and the error encountered (if any).
	Read(
		ctx context.Context,
		seriesID ident.ID,
		blockStart xtime.UnixNano,
		nsCtx namespace.Context,
	) ([]xio.BlockReader, bool, error)

	// ForEachRemaining loops through each seriesID/blockStart combination that
	// was not already handled by a call to Read().
	ForEachRemaining(
		ctx context.Context,
		blockStart xtime.UnixNano,
		fn ForEachRemainingFn,
		nsCtx namespace.Context,
	) error
}

MergeWith is an interface that the fs merger uses to merge data with.

func NewNoopMergeWith added in v0.15.10 

func NewNoopMergeWith() MergeWith

NewNoopMergeWith creates a new noopMergeWith object. NoopMergeWith can be used in conjunction with the merge.Merger to generate copies of filesets.

type Merger added in v0.10.0 

type Merger interface {
	// Merge merges the specified fileset file with a merge target.
	Merge(
		fileID FileSetFileIdentifier,
		mergeWith MergeWith,
		nextVolumeIndex int,
		flushPreparer persist.FlushPreparer,
		nsCtx namespace.Context,
		onFlush persist.OnFlushSeries,
	) (persist.DataCloser, error)

	// MergeAndCleanup merges the specified fileset file with a merge target and
	// removes the previous version of the fileset. This should only be called
	// within the bootstrapper. Any other file deletions outside of the
	// bootstrapper should be handled by the CleanupManager.
	MergeAndCleanup(
		fileID FileSetFileIdentifier,
		mergeWith MergeWith,
		nextVolumeIndex int,
		flushPreparer persist.FlushPreparer,
		nsCtx namespace.Context,
		onFlush persist.OnFlushSeries,
		isBootstrapped bool,
	) error
}

Merger is in charge of merging filesets with some target MergeWith interface.

func NewMerger added in v0.10.0 

func NewMerger(
	reader DataFileSetReader,
	blockAllocSize int,
	srPool xio.SegmentReaderPool,
	multiIterPool encoding.MultiReaderIteratorPool,
	identPool ident.Pool,
	encoderPool encoding.EncoderPool,
	contextPool context.Pool,
	filePathPrefix string,
	nsOpts namespace.Options,
) Merger

NewMerger returns a new Merger. This implementation is in charge of merging the data from an existing fileset with a merge target. If data for a series at a timestamp exists both on disk and the merge target, data from the merge target will be used. This merged data is then persisted.

Note that the merger does not know how or where this merged data is persisted since it just uses the flushPreparer that is passed in. Further, it does not signal to the database of the existence of the newly persisted data, nor does it clean up the original fileset.

type MockConcurrentDataFileSetSeeker added in v0.8.0 

type MockConcurrentDataFileSetSeeker struct {
	// contains filtered or unexported fields
}

MockConcurrentDataFileSetSeeker is a mock of ConcurrentDataFileSetSeeker interface

func NewMockConcurrentDataFileSetSeeker added in v0.8.0 

func NewMockConcurrentDataFileSetSeeker(ctrl *gomock.Controller) *MockConcurrentDataFileSetSeeker

NewMockConcurrentDataFileSetSeeker creates a new mock instance

func (*MockConcurrentDataFileSetSeeker) Close added in v0.8.0 

func (m *MockConcurrentDataFileSetSeeker) Close() error

Close mocks base method

func (*MockConcurrentDataFileSetSeeker) ConcurrentIDBloomFilter added in v0.8.0 

func (m *MockConcurrentDataFileSetSeeker) ConcurrentIDBloomFilter() *ManagedConcurrentBloomFilter

ConcurrentIDBloomFilter mocks base method

func (*MockConcurrentDataFileSetSeeker) EXPECT added in v0.8.0 

func (m *MockConcurrentDataFileSetSeeker) EXPECT() *MockConcurrentDataFileSetSeekerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockConcurrentDataFileSetSeeker) SeekByID added in v0.8.0 

func (m *MockConcurrentDataFileSetSeeker) SeekByID(arg0 ident.ID, arg1 ReusableSeekerResources) (checked.Bytes, error)

SeekByID mocks base method

func (*MockConcurrentDataFileSetSeeker) SeekByIndexEntry added in v0.8.0 

func (m *MockConcurrentDataFileSetSeeker) SeekByIndexEntry(arg0 IndexEntry, arg1 ReusableSeekerResources) (checked.Bytes, error)

SeekByIndexEntry mocks base method

func (*MockConcurrentDataFileSetSeeker) SeekIndexEntry added in v0.8.0 

func (m *MockConcurrentDataFileSetSeeker) SeekIndexEntry(arg0 ident.ID, arg1 ReusableSeekerResources) (IndexEntry, error)

SeekIndexEntry mocks base method

func (*MockConcurrentDataFileSetSeeker) SeekWideEntry added in v1.0.1 

func (m *MockConcurrentDataFileSetSeeker) SeekWideEntry(arg0 ident.ID, arg1 schema.WideEntryFilter, arg2 ReusableSeekerResources) (xio.WideEntry, error)

SeekWideEntry mocks base method

type MockConcurrentDataFileSetSeekerMockRecorder added in v0.8.0 

type MockConcurrentDataFileSetSeekerMockRecorder struct {
	// contains filtered or unexported fields
}

MockConcurrentDataFileSetSeekerMockRecorder is the mock recorder for MockConcurrentDataFileSetSeeker

func (*MockConcurrentDataFileSetSeekerMockRecorder) Close added in v0.8.0 

func (mr *MockConcurrentDataFileSetSeekerMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockConcurrentDataFileSetSeekerMockRecorder) ConcurrentIDBloomFilter added in v0.8.0 

func (mr *MockConcurrentDataFileSetSeekerMockRecorder) ConcurrentIDBloomFilter() *gomock.Call

ConcurrentIDBloomFilter indicates an expected call of ConcurrentIDBloomFilter

func (*MockConcurrentDataFileSetSeekerMockRecorder) SeekByID added in v0.8.0 

func (mr *MockConcurrentDataFileSetSeekerMockRecorder) SeekByID(arg0, arg1 interface{}) *gomock.Call

SeekByID indicates an expected call of SeekByID

func (*MockConcurrentDataFileSetSeekerMockRecorder) SeekByIndexEntry added in v0.8.0 

func (mr *MockConcurrentDataFileSetSeekerMockRecorder) SeekByIndexEntry(arg0, arg1 interface{}) *gomock.Call

SeekByIndexEntry indicates an expected call of SeekByIndexEntry

func (*MockConcurrentDataFileSetSeekerMockRecorder) SeekIndexEntry added in v0.8.0 

func (mr *MockConcurrentDataFileSetSeekerMockRecorder) SeekIndexEntry(arg0, arg1 interface{}) *gomock.Call

SeekIndexEntry indicates an expected call of SeekIndexEntry

func (*MockConcurrentDataFileSetSeekerMockRecorder) SeekWideEntry added in v1.0.1 

func (mr *MockConcurrentDataFileSetSeekerMockRecorder) SeekWideEntry(arg0, arg1, arg2 interface{}) *gomock.Call

SeekWideEntry indicates an expected call of SeekWideEntry

type MockDataEntryProcessor added in v1.0.1 

type MockDataEntryProcessor struct {
	// contains filtered or unexported fields
}

MockDataEntryProcessor is a mock of DataEntryProcessor interface

func NewMockDataEntryProcessor added in v1.0.1 

func NewMockDataEntryProcessor(ctrl *gomock.Controller) *MockDataEntryProcessor

NewMockDataEntryProcessor creates a new mock instance

func (*MockDataEntryProcessor) EXPECT added in v1.0.1 

func (m *MockDataEntryProcessor) EXPECT() *MockDataEntryProcessorMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockDataEntryProcessor) ProcessEntry added in v1.0.1 

func (m *MockDataEntryProcessor) ProcessEntry(arg0 StreamedDataEntry) error

ProcessEntry mocks base method

func (*MockDataEntryProcessor) SetEntriesCount added in v1.0.1 

func (m *MockDataEntryProcessor) SetEntriesCount(arg0 int)

SetEntriesCount mocks base method

type MockDataEntryProcessorMockRecorder added in v1.0.1 

type MockDataEntryProcessorMockRecorder struct {
	// contains filtered or unexported fields
}

MockDataEntryProcessorMockRecorder is the mock recorder for MockDataEntryProcessor

func (*MockDataEntryProcessorMockRecorder) ProcessEntry added in v1.0.1 

func (mr *MockDataEntryProcessorMockRecorder) ProcessEntry(arg0 interface{}) *gomock.Call

ProcessEntry indicates an expected call of ProcessEntry

func (*MockDataEntryProcessorMockRecorder) SetEntriesCount added in v1.0.1 

func (mr *MockDataEntryProcessorMockRecorder) SetEntriesCount(arg0 interface{}) *gomock.Call

SetEntriesCount indicates an expected call of SetEntriesCount

type MockDataFileSetReader 

type MockDataFileSetReader struct {
	// contains filtered or unexported fields
}

MockDataFileSetReader is a mock of DataFileSetReader interface

func NewMockDataFileSetReader 

func NewMockDataFileSetReader(ctrl *gomock.Controller) *MockDataFileSetReader

NewMockDataFileSetReader creates a new mock instance

func (*MockDataFileSetReader) Close 

func (m *MockDataFileSetReader) Close() error

Close mocks base method

func (*MockDataFileSetReader) EXPECT 

func (m *MockDataFileSetReader) EXPECT() *MockDataFileSetReaderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockDataFileSetReader) Entries 

func (m *MockDataFileSetReader) Entries() int

Entries mocks base method

func (*MockDataFileSetReader) EntriesRead 

func (m *MockDataFileSetReader) EntriesRead() int

EntriesRead mocks base method

func (*MockDataFileSetReader) MetadataRead 

func (m *MockDataFileSetReader) MetadataRead() int

MetadataRead mocks base method

func (*MockDataFileSetReader) Open 

func (m *MockDataFileSetReader) Open(arg0 DataReaderOpenOptions) error

Open mocks base method

func (*MockDataFileSetReader) Range 

func (m *MockDataFileSetReader) Range() time0.Range

Range mocks base method

func (*MockDataFileSetReader) Read 

func (m *MockDataFileSetReader) Read() (ident.ID, ident.TagIterator, checked.Bytes, uint32, error)

Read mocks base method

func (*MockDataFileSetReader) ReadBloomFilter 

func (m *MockDataFileSetReader) ReadBloomFilter() (*ManagedConcurrentBloomFilter, error)

ReadBloomFilter mocks base method

func (*MockDataFileSetReader) ReadMetadata 

func (m *MockDataFileSetReader) ReadMetadata() (ident.ID, ident.TagIterator, int, uint32, error)

ReadMetadata mocks base method

func (*MockDataFileSetReader) Status 

func (m *MockDataFileSetReader) Status() DataFileSetReaderStatus

Status mocks base method

func (*MockDataFileSetReader) StreamingEnabled added in v0.15.15 

func (m *MockDataFileSetReader) StreamingEnabled() bool

StreamingEnabled mocks base method

func (*MockDataFileSetReader) StreamingRead added in v0.15.15 

func (m *MockDataFileSetReader) StreamingRead() (StreamedDataEntry, error)

StreamingRead mocks base method

func (*MockDataFileSetReader) Validate 

func (m *MockDataFileSetReader) Validate() error

Validate mocks base method

func (*MockDataFileSetReader) ValidateData 

func (m *MockDataFileSetReader) ValidateData() error

ValidateData mocks base method

func (*MockDataFileSetReader) ValidateMetadata 

func (m *MockDataFileSetReader) ValidateMetadata() error

ValidateMetadata mocks base method

type MockDataFileSetReaderMockRecorder 

type MockDataFileSetReaderMockRecorder struct {
	// contains filtered or unexported fields
}

MockDataFileSetReaderMockRecorder is the mock recorder for MockDataFileSetReader

func (*MockDataFileSetReaderMockRecorder) Close 

func (mr *MockDataFileSetReaderMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockDataFileSetReaderMockRecorder) Entries 

func (mr *MockDataFileSetReaderMockRecorder) Entries() *gomock.Call

Entries indicates an expected call of Entries

func (*MockDataFileSetReaderMockRecorder) EntriesRead 

func (mr *MockDataFileSetReaderMockRecorder) EntriesRead() *gomock.Call

EntriesRead indicates an expected call of EntriesRead

func (*MockDataFileSetReaderMockRecorder) MetadataRead 

func (mr *MockDataFileSetReaderMockRecorder) MetadataRead() *gomock.Call

MetadataRead indicates an expected call of MetadataRead

func (*MockDataFileSetReaderMockRecorder) Open 

func (mr *MockDataFileSetReaderMockRecorder) Open(arg0 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockDataFileSetReaderMockRecorder) Range 

func (mr *MockDataFileSetReaderMockRecorder) Range() *gomock.Call

Range indicates an expected call of Range

func (*MockDataFileSetReaderMockRecorder) Read 

func (mr *MockDataFileSetReaderMockRecorder) Read() *gomock.Call

Read indicates an expected call of Read

func (*MockDataFileSetReaderMockRecorder) ReadBloomFilter 

func (mr *MockDataFileSetReaderMockRecorder) ReadBloomFilter() *gomock.Call

ReadBloomFilter indicates an expected call of ReadBloomFilter

func (*MockDataFileSetReaderMockRecorder) ReadMetadata 

func (mr *MockDataFileSetReaderMockRecorder) ReadMetadata() *gomock.Call

ReadMetadata indicates an expected call of ReadMetadata

func (*MockDataFileSetReaderMockRecorder) Status 

func (mr *MockDataFileSetReaderMockRecorder) Status() *gomock.Call

Status indicates an expected call of Status

func (*MockDataFileSetReaderMockRecorder) StreamingEnabled added in v0.15.15 

func (mr *MockDataFileSetReaderMockRecorder) StreamingEnabled() *gomock.Call

StreamingEnabled indicates an expected call of StreamingEnabled

func (*MockDataFileSetReaderMockRecorder) StreamingRead added in v0.15.15 

func (mr *MockDataFileSetReaderMockRecorder) StreamingRead() *gomock.Call

StreamingRead indicates an expected call of StreamingRead

func (*MockDataFileSetReaderMockRecorder) Validate 

func (mr *MockDataFileSetReaderMockRecorder) Validate() *gomock.Call

Validate indicates an expected call of Validate

func (*MockDataFileSetReaderMockRecorder) ValidateData 

func (mr *MockDataFileSetReaderMockRecorder) ValidateData() *gomock.Call

ValidateData indicates an expected call of ValidateData

func (*MockDataFileSetReaderMockRecorder) ValidateMetadata 

func (mr *MockDataFileSetReaderMockRecorder) ValidateMetadata() *gomock.Call

ValidateMetadata indicates an expected call of ValidateMetadata

type MockDataFileSetSeeker 

type MockDataFileSetSeeker struct {
	// contains filtered or unexported fields
}

MockDataFileSetSeeker is a mock of DataFileSetSeeker interface

func NewMockDataFileSetSeeker 

func NewMockDataFileSetSeeker(ctrl *gomock.Controller) *MockDataFileSetSeeker

NewMockDataFileSetSeeker creates a new mock instance

func (*MockDataFileSetSeeker) Close 

func (m *MockDataFileSetSeeker) Close() error

Close mocks base method

func (*MockDataFileSetSeeker) ConcurrentClone 

func (m *MockDataFileSetSeeker) ConcurrentClone() (ConcurrentDataFileSetSeeker, error)

ConcurrentClone mocks base method

func (*MockDataFileSetSeeker) ConcurrentIDBloomFilter 

func (m *MockDataFileSetSeeker) ConcurrentIDBloomFilter() *ManagedConcurrentBloomFilter

ConcurrentIDBloomFilter mocks base method

func (*MockDataFileSetSeeker) EXPECT 

func (m *MockDataFileSetSeeker) EXPECT() *MockDataFileSetSeekerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockDataFileSetSeeker) Open 

func (m *MockDataFileSetSeeker) Open(arg0 ident.ID, arg1 uint32, arg2 time.Time, arg3 int, arg4 ReusableSeekerResources) error

Open mocks base method

func (*MockDataFileSetSeeker) Range 

func (m *MockDataFileSetSeeker) Range() time0.Range

Range mocks base method

func (*MockDataFileSetSeeker) SeekByID 

func (m *MockDataFileSetSeeker) SeekByID(arg0 ident.ID, arg1 ReusableSeekerResources) (checked.Bytes, error)

SeekByID mocks base method

func (*MockDataFileSetSeeker) SeekByIndexEntry 

func (m *MockDataFileSetSeeker) SeekByIndexEntry(arg0 IndexEntry, arg1 ReusableSeekerResources) (checked.Bytes, error)

SeekByIndexEntry mocks base method

func (*MockDataFileSetSeeker) SeekIndexEntry 

func (m *MockDataFileSetSeeker) SeekIndexEntry(arg0 ident.ID, arg1 ReusableSeekerResources) (IndexEntry, error)

SeekIndexEntry mocks base method

func (*MockDataFileSetSeeker) SeekWideEntry added in v1.0.1 

func (m *MockDataFileSetSeeker) SeekWideEntry(arg0 ident.ID, arg1 schema.WideEntryFilter, arg2 ReusableSeekerResources) (xio.WideEntry, error)

SeekWideEntry mocks base method

type MockDataFileSetSeekerManager added in v0.8.0 

type MockDataFileSetSeekerManager struct {
	// contains filtered or unexported fields
}

MockDataFileSetSeekerManager is a mock of DataFileSetSeekerManager interface

func NewMockDataFileSetSeekerManager added in v0.8.0 

func NewMockDataFileSetSeekerManager(ctrl *gomock.Controller) *MockDataFileSetSeekerManager

NewMockDataFileSetSeekerManager creates a new mock instance

func (*MockDataFileSetSeekerManager) AssignShardSet added in v0.15.0 

func (m *MockDataFileSetSeekerManager) AssignShardSet(arg0 sharding.ShardSet)

AssignShardSet mocks base method

func (*MockDataFileSetSeekerManager) Borrow added in v0.8.0 

func (m *MockDataFileSetSeekerManager) Borrow(arg0 uint32, arg1 time.Time) (ConcurrentDataFileSetSeeker, error)

Borrow mocks base method

func (*MockDataFileSetSeekerManager) CacheShardIndices added in v0.8.0 

func (m *MockDataFileSetSeekerManager) CacheShardIndices(arg0 []uint32) error

CacheShardIndices mocks base method

func (*MockDataFileSetSeekerManager) Close added in v0.8.0 

func (m *MockDataFileSetSeekerManager) Close() error

Close mocks base method

func (*MockDataFileSetSeekerManager) EXPECT added in v0.8.0 

func (m *MockDataFileSetSeekerManager) EXPECT() *MockDataFileSetSeekerManagerMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockDataFileSetSeekerManager) Open added in v0.8.0 

func (m *MockDataFileSetSeekerManager) Open(arg0 namespace.Metadata, arg1 sharding.ShardSet) error

Open mocks base method

func (*MockDataFileSetSeekerManager) Return added in v0.8.0 

func (m *MockDataFileSetSeekerManager) Return(arg0 uint32, arg1 time.Time, arg2 ConcurrentDataFileSetSeeker) error

Return mocks base method

func (*MockDataFileSetSeekerManager) Test added in v0.11.0 

func (m *MockDataFileSetSeekerManager) Test(arg0 ident.ID, arg1 uint32, arg2 time.Time) (bool, error)

Test mocks base method

type MockDataFileSetSeekerManagerMockRecorder added in v0.8.0 

type MockDataFileSetSeekerManagerMockRecorder struct {
	// contains filtered or unexported fields
}

MockDataFileSetSeekerManagerMockRecorder is the mock recorder for MockDataFileSetSeekerManager

func (*MockDataFileSetSeekerManagerMockRecorder) AssignShardSet added in v0.15.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) AssignShardSet(arg0 interface{}) *gomock.Call

AssignShardSet indicates an expected call of AssignShardSet

func (*MockDataFileSetSeekerManagerMockRecorder) Borrow added in v0.8.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) Borrow(arg0, arg1 interface{}) *gomock.Call

Borrow indicates an expected call of Borrow

func (*MockDataFileSetSeekerManagerMockRecorder) CacheShardIndices added in v0.8.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) CacheShardIndices(arg0 interface{}) *gomock.Call

CacheShardIndices indicates an expected call of CacheShardIndices

func (*MockDataFileSetSeekerManagerMockRecorder) Close added in v0.8.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockDataFileSetSeekerManagerMockRecorder) Open added in v0.8.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) Open(arg0, arg1 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockDataFileSetSeekerManagerMockRecorder) Return added in v0.8.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) Return(arg0, arg1, arg2 interface{}) *gomock.Call

Return indicates an expected call of Return

func (*MockDataFileSetSeekerManagerMockRecorder) Test added in v0.11.0 

func (mr *MockDataFileSetSeekerManagerMockRecorder) Test(arg0, arg1, arg2 interface{}) *gomock.Call

Test indicates an expected call of Test

type MockDataFileSetSeekerMockRecorder 

type MockDataFileSetSeekerMockRecorder struct {
	// contains filtered or unexported fields
}

MockDataFileSetSeekerMockRecorder is the mock recorder for MockDataFileSetSeeker

func (*MockDataFileSetSeekerMockRecorder) Close 

func (mr *MockDataFileSetSeekerMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockDataFileSetSeekerMockRecorder) ConcurrentClone 

func (mr *MockDataFileSetSeekerMockRecorder) ConcurrentClone() *gomock.Call

ConcurrentClone indicates an expected call of ConcurrentClone

func (*MockDataFileSetSeekerMockRecorder) ConcurrentIDBloomFilter 

func (mr *MockDataFileSetSeekerMockRecorder) ConcurrentIDBloomFilter() *gomock.Call

ConcurrentIDBloomFilter indicates an expected call of ConcurrentIDBloomFilter

func (*MockDataFileSetSeekerMockRecorder) Open 

func (mr *MockDataFileSetSeekerMockRecorder) Open(arg0, arg1, arg2, arg3, arg4 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockDataFileSetSeekerMockRecorder) Range 

func (mr *MockDataFileSetSeekerMockRecorder) Range() *gomock.Call

Range indicates an expected call of Range

func (*MockDataFileSetSeekerMockRecorder) SeekByID 

func (mr *MockDataFileSetSeekerMockRecorder) SeekByID(arg0, arg1 interface{}) *gomock.Call

SeekByID indicates an expected call of SeekByID

func (*MockDataFileSetSeekerMockRecorder) SeekByIndexEntry 

func (mr *MockDataFileSetSeekerMockRecorder) SeekByIndexEntry(arg0, arg1 interface{}) *gomock.Call

SeekByIndexEntry indicates an expected call of SeekByIndexEntry

func (*MockDataFileSetSeekerMockRecorder) SeekIndexEntry 

func (mr *MockDataFileSetSeekerMockRecorder) SeekIndexEntry(arg0, arg1 interface{}) *gomock.Call

SeekIndexEntry indicates an expected call of SeekIndexEntry

func (*MockDataFileSetSeekerMockRecorder) SeekWideEntry added in v1.0.1 

func (mr *MockDataFileSetSeekerMockRecorder) SeekWideEntry(arg0, arg1, arg2 interface{}) *gomock.Call

SeekWideEntry indicates an expected call of SeekWideEntry

type MockDataFileSetWriter 

type MockDataFileSetWriter struct {
	// contains filtered or unexported fields
}

MockDataFileSetWriter is a mock of DataFileSetWriter interface

func NewMockDataFileSetWriter 

func NewMockDataFileSetWriter(ctrl *gomock.Controller) *MockDataFileSetWriter

NewMockDataFileSetWriter creates a new mock instance

func (*MockDataFileSetWriter) Close 

func (m *MockDataFileSetWriter) Close() error

Close mocks base method

func (*MockDataFileSetWriter) DeferClose added in v0.15.3 

func (m *MockDataFileSetWriter) DeferClose() (persist.DataCloser, error)

DeferClose mocks base method

func (*MockDataFileSetWriter) EXPECT 

func (m *MockDataFileSetWriter) EXPECT() *MockDataFileSetWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockDataFileSetWriter) Open 

func (m *MockDataFileSetWriter) Open(arg0 DataWriterOpenOptions) error

Open mocks base method

func (*MockDataFileSetWriter) Write 

func (m *MockDataFileSetWriter) Write(arg0 persist.Metadata, arg1 checked.Bytes, arg2 uint32) error

Write mocks base method

func (*MockDataFileSetWriter) WriteAll 

func (m *MockDataFileSetWriter) WriteAll(arg0 persist.Metadata, arg1 []checked.Bytes, arg2 uint32) error

WriteAll mocks base method

type MockDataFileSetWriterMockRecorder 

type MockDataFileSetWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockDataFileSetWriterMockRecorder is the mock recorder for MockDataFileSetWriter

func (*MockDataFileSetWriterMockRecorder) Close 

func (mr *MockDataFileSetWriterMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockDataFileSetWriterMockRecorder) DeferClose added in v0.15.3 

func (mr *MockDataFileSetWriterMockRecorder) DeferClose() *gomock.Call

DeferClose indicates an expected call of DeferClose

func (*MockDataFileSetWriterMockRecorder) Open 

func (mr *MockDataFileSetWriterMockRecorder) Open(arg0 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockDataFileSetWriterMockRecorder) Write 

func (mr *MockDataFileSetWriterMockRecorder) Write(arg0, arg1, arg2 interface{}) *gomock.Call

Write indicates an expected call of Write

func (*MockDataFileSetWriterMockRecorder) WriteAll 

func (mr *MockDataFileSetWriterMockRecorder) WriteAll(arg0, arg1, arg2 interface{}) *gomock.Call

WriteAll indicates an expected call of WriteAll

type MockIndexFileSetReader 

type MockIndexFileSetReader struct {
	// contains filtered or unexported fields
}

MockIndexFileSetReader is a mock of IndexFileSetReader interface

func NewMockIndexFileSetReader 

func NewMockIndexFileSetReader(ctrl *gomock.Controller) *MockIndexFileSetReader

NewMockIndexFileSetReader creates a new mock instance

func (*MockIndexFileSetReader) Close 

func (m *MockIndexFileSetReader) Close() error

Close mocks base method

func (*MockIndexFileSetReader) EXPECT 

func (m *MockIndexFileSetReader) EXPECT() *MockIndexFileSetReaderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockIndexFileSetReader) IndexVolumeType added in v0.15.0 

func (m *MockIndexFileSetReader) IndexVolumeType() persist0.IndexVolumeType

IndexVolumeType mocks base method

func (*MockIndexFileSetReader) Open 

func (m *MockIndexFileSetReader) Open(arg0 IndexReaderOpenOptions) (IndexReaderOpenResult, error)

Open mocks base method

func (*MockIndexFileSetReader) ReadSegmentFileSet 

func (m *MockIndexFileSetReader) ReadSegmentFileSet() (persist0.IndexSegmentFileSet, error)

ReadSegmentFileSet mocks base method

func (*MockIndexFileSetReader) SegmentFileSets 

func (m *MockIndexFileSetReader) SegmentFileSets() int

SegmentFileSets mocks base method

func (*MockIndexFileSetReader) Validate 

func (m *MockIndexFileSetReader) Validate() error

Validate mocks base method

type MockIndexFileSetReaderMockRecorder 

type MockIndexFileSetReaderMockRecorder struct {
	// contains filtered or unexported fields
}

MockIndexFileSetReaderMockRecorder is the mock recorder for MockIndexFileSetReader

func (*MockIndexFileSetReaderMockRecorder) Close 

func (mr *MockIndexFileSetReaderMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockIndexFileSetReaderMockRecorder) IndexVolumeType added in v0.15.0 

func (mr *MockIndexFileSetReaderMockRecorder) IndexVolumeType() *gomock.Call

IndexVolumeType indicates an expected call of IndexVolumeType

func (*MockIndexFileSetReaderMockRecorder) Open 

func (mr *MockIndexFileSetReaderMockRecorder) Open(arg0 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockIndexFileSetReaderMockRecorder) ReadSegmentFileSet 

func (mr *MockIndexFileSetReaderMockRecorder) ReadSegmentFileSet() *gomock.Call

ReadSegmentFileSet indicates an expected call of ReadSegmentFileSet

func (*MockIndexFileSetReaderMockRecorder) SegmentFileSets 

func (mr *MockIndexFileSetReaderMockRecorder) SegmentFileSets() *gomock.Call

SegmentFileSets indicates an expected call of SegmentFileSets

func (*MockIndexFileSetReaderMockRecorder) Validate 

func (mr *MockIndexFileSetReaderMockRecorder) Validate() *gomock.Call

Validate indicates an expected call of Validate

type MockIndexFileSetWriter 

type MockIndexFileSetWriter struct {
	// contains filtered or unexported fields
}

MockIndexFileSetWriter is a mock of IndexFileSetWriter interface

func NewMockIndexFileSetWriter 

func NewMockIndexFileSetWriter(ctrl *gomock.Controller) *MockIndexFileSetWriter

NewMockIndexFileSetWriter creates a new mock instance

func (*MockIndexFileSetWriter) Close 

func (m *MockIndexFileSetWriter) Close() error

Close mocks base method

func (*MockIndexFileSetWriter) EXPECT 

func (m *MockIndexFileSetWriter) EXPECT() *MockIndexFileSetWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockIndexFileSetWriter) Open 

func (m *MockIndexFileSetWriter) Open(arg0 IndexWriterOpenOptions) error

Open mocks base method

func (*MockIndexFileSetWriter) WriteSegmentFileSet 

func (m *MockIndexFileSetWriter) WriteSegmentFileSet(arg0 persist0.IndexSegmentFileSetWriter) error

WriteSegmentFileSet mocks base method

type MockIndexFileSetWriterMockRecorder 

type MockIndexFileSetWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockIndexFileSetWriterMockRecorder is the mock recorder for MockIndexFileSetWriter

func (*MockIndexFileSetWriterMockRecorder) Close 

func (mr *MockIndexFileSetWriterMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockIndexFileSetWriterMockRecorder) Open 

func (mr *MockIndexFileSetWriterMockRecorder) Open(arg0 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockIndexFileSetWriterMockRecorder) WriteSegmentFileSet 

func (mr *MockIndexFileSetWriterMockRecorder) WriteSegmentFileSet(arg0 interface{}) *gomock.Call

WriteSegmentFileSet indicates an expected call of WriteSegmentFileSet

type MockIndexSegmentFile 

type MockIndexSegmentFile struct {
	// contains filtered or unexported fields
}

MockIndexSegmentFile is a mock of IndexSegmentFile interface

func NewMockIndexSegmentFile 

func NewMockIndexSegmentFile(ctrl *gomock.Controller) *MockIndexSegmentFile

NewMockIndexSegmentFile creates a new mock instance

func (*MockIndexSegmentFile) EXPECT 

func (m *MockIndexSegmentFile) EXPECT() *MockIndexSegmentFileMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockIndexSegmentFile) Files 

func (m *MockIndexSegmentFile) Files() []persist0.IndexSegmentFile

Files mocks base method

func (*MockIndexSegmentFile) MajorVersion 

func (m *MockIndexSegmentFile) MajorVersion() int

MajorVersion mocks base method

func (*MockIndexSegmentFile) MinorVersion 

func (m *MockIndexSegmentFile) MinorVersion() int

MinorVersion mocks base method

func (*MockIndexSegmentFile) SegmentMetadata 

func (m *MockIndexSegmentFile) SegmentMetadata() []byte

SegmentMetadata mocks base method

func (*MockIndexSegmentFile) SegmentType 

func (m *MockIndexSegmentFile) SegmentType() persist0.IndexSegmentType

SegmentType mocks base method

type MockIndexSegmentFileMockRecorder 

type MockIndexSegmentFileMockRecorder struct {
	// contains filtered or unexported fields
}

MockIndexSegmentFileMockRecorder is the mock recorder for MockIndexSegmentFile

func (*MockIndexSegmentFileMockRecorder) Files 

func (mr *MockIndexSegmentFileMockRecorder) Files() *gomock.Call

Files indicates an expected call of Files

func (*MockIndexSegmentFileMockRecorder) MajorVersion 

func (mr *MockIndexSegmentFileMockRecorder) MajorVersion() *gomock.Call

MajorVersion indicates an expected call of MajorVersion

func (*MockIndexSegmentFileMockRecorder) MinorVersion 

func (mr *MockIndexSegmentFileMockRecorder) MinorVersion() *gomock.Call

MinorVersion indicates an expected call of MinorVersion

func (*MockIndexSegmentFileMockRecorder) SegmentMetadata 

func (mr *MockIndexSegmentFileMockRecorder) SegmentMetadata() *gomock.Call

SegmentMetadata indicates an expected call of SegmentMetadata

func (*MockIndexSegmentFileMockRecorder) SegmentType 

func (mr *MockIndexSegmentFileMockRecorder) SegmentType() *gomock.Call

SegmentType indicates an expected call of SegmentType

type MockIndexSegmentFileSet 

type MockIndexSegmentFileSet struct {
	// contains filtered or unexported fields
}

MockIndexSegmentFileSet is a mock of IndexSegmentFileSet interface

func NewMockIndexSegmentFileSet 

func NewMockIndexSegmentFileSet(ctrl *gomock.Controller) *MockIndexSegmentFileSet

NewMockIndexSegmentFileSet creates a new mock instance

func (*MockIndexSegmentFileSet) EXPECT 

func (m *MockIndexSegmentFileSet) EXPECT() *MockIndexSegmentFileSetMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockIndexSegmentFileSet) Files 

func (m *MockIndexSegmentFileSet) Files() []persist0.IndexSegmentFile

Files mocks base method

func (*MockIndexSegmentFileSet) MajorVersion 

func (m *MockIndexSegmentFileSet) MajorVersion() int

MajorVersion mocks base method

func (*MockIndexSegmentFileSet) MinorVersion 

func (m *MockIndexSegmentFileSet) MinorVersion() int

MinorVersion mocks base method

func (*MockIndexSegmentFileSet) SegmentMetadata 

func (m *MockIndexSegmentFileSet) SegmentMetadata() []byte

SegmentMetadata mocks base method

func (*MockIndexSegmentFileSet) SegmentType 

func (m *MockIndexSegmentFileSet) SegmentType() persist0.IndexSegmentType

SegmentType mocks base method

type MockIndexSegmentFileSetMockRecorder 

type MockIndexSegmentFileSetMockRecorder struct {
	// contains filtered or unexported fields
}

MockIndexSegmentFileSetMockRecorder is the mock recorder for MockIndexSegmentFileSet

func (*MockIndexSegmentFileSetMockRecorder) Files 

func (mr *MockIndexSegmentFileSetMockRecorder) Files() *gomock.Call

Files indicates an expected call of Files

func (*MockIndexSegmentFileSetMockRecorder) MajorVersion 

func (mr *MockIndexSegmentFileSetMockRecorder) MajorVersion() *gomock.Call

MajorVersion indicates an expected call of MajorVersion

func (*MockIndexSegmentFileSetMockRecorder) MinorVersion 

func (mr *MockIndexSegmentFileSetMockRecorder) MinorVersion() *gomock.Call

MinorVersion indicates an expected call of MinorVersion

func (*MockIndexSegmentFileSetMockRecorder) SegmentMetadata 

func (mr *MockIndexSegmentFileSetMockRecorder) SegmentMetadata() *gomock.Call

SegmentMetadata indicates an expected call of SegmentMetadata

func (*MockIndexSegmentFileSetMockRecorder) SegmentType 

func (mr *MockIndexSegmentFileSetMockRecorder) SegmentType() *gomock.Call

SegmentType indicates an expected call of SegmentType

type MockIndexSegmentFileSetWriter 

type MockIndexSegmentFileSetWriter struct {
	// contains filtered or unexported fields
}

MockIndexSegmentFileSetWriter is a mock of IndexSegmentFileSetWriter interface

func NewMockIndexSegmentFileSetWriter 

func NewMockIndexSegmentFileSetWriter(ctrl *gomock.Controller) *MockIndexSegmentFileSetWriter

NewMockIndexSegmentFileSetWriter creates a new mock instance

func (*MockIndexSegmentFileSetWriter) EXPECT 

func (m *MockIndexSegmentFileSetWriter) EXPECT() *MockIndexSegmentFileSetWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockIndexSegmentFileSetWriter) Files 

func (m *MockIndexSegmentFileSetWriter) Files() []persist0.IndexSegmentFileType

Files mocks base method

func (*MockIndexSegmentFileSetWriter) MajorVersion 

func (m *MockIndexSegmentFileSetWriter) MajorVersion() int

MajorVersion mocks base method

func (*MockIndexSegmentFileSetWriter) MinorVersion 

func (m *MockIndexSegmentFileSetWriter) MinorVersion() int

MinorVersion mocks base method

func (*MockIndexSegmentFileSetWriter) SegmentMetadata 

func (m *MockIndexSegmentFileSetWriter) SegmentMetadata() []byte

SegmentMetadata mocks base method

func (*MockIndexSegmentFileSetWriter) SegmentType 

func (m *MockIndexSegmentFileSetWriter) SegmentType() persist0.IndexSegmentType

SegmentType mocks base method

func (*MockIndexSegmentFileSetWriter) WriteFile 

func (m *MockIndexSegmentFileSetWriter) WriteFile(arg0 persist0.IndexSegmentFileType, arg1 io.Writer) error

WriteFile mocks base method

type MockIndexSegmentFileSetWriterMockRecorder 

type MockIndexSegmentFileSetWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockIndexSegmentFileSetWriterMockRecorder is the mock recorder for MockIndexSegmentFileSetWriter

func (*MockIndexSegmentFileSetWriterMockRecorder) Files 

func (mr *MockIndexSegmentFileSetWriterMockRecorder) Files() *gomock.Call

Files indicates an expected call of Files

func (*MockIndexSegmentFileSetWriterMockRecorder) MajorVersion 

func (mr *MockIndexSegmentFileSetWriterMockRecorder) MajorVersion() *gomock.Call

MajorVersion indicates an expected call of MajorVersion

func (*MockIndexSegmentFileSetWriterMockRecorder) MinorVersion 

func (mr *MockIndexSegmentFileSetWriterMockRecorder) MinorVersion() *gomock.Call

MinorVersion indicates an expected call of MinorVersion

func (*MockIndexSegmentFileSetWriterMockRecorder) SegmentMetadata 

func (mr *MockIndexSegmentFileSetWriterMockRecorder) SegmentMetadata() *gomock.Call

SegmentMetadata indicates an expected call of SegmentMetadata

func (*MockIndexSegmentFileSetWriterMockRecorder) SegmentType 

func (mr *MockIndexSegmentFileSetWriterMockRecorder) SegmentType() *gomock.Call

SegmentType indicates an expected call of SegmentType

func (*MockIndexSegmentFileSetWriterMockRecorder) WriteFile 

func (mr *MockIndexSegmentFileSetWriterMockRecorder) WriteFile(arg0, arg1 interface{}) *gomock.Call

WriteFile indicates an expected call of WriteFile

type MockMergeWith added in v0.10.0 

type MockMergeWith struct {
	// contains filtered or unexported fields
}

MockMergeWith is a mock of MergeWith interface

func NewMockMergeWith added in v0.10.0 

func NewMockMergeWith(ctrl *gomock.Controller) *MockMergeWith

NewMockMergeWith creates a new mock instance

func (*MockMergeWith) EXPECT added in v0.10.0 

func (m *MockMergeWith) EXPECT() *MockMergeWithMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockMergeWith) ForEachRemaining added in v0.10.0 

func (m *MockMergeWith) ForEachRemaining(arg0 context.Context, arg1 time0.UnixNano, arg2 ForEachRemainingFn, arg3 namespace.Context) error

ForEachRemaining mocks base method

func (*MockMergeWith) Read added in v0.10.0 

func (m *MockMergeWith) Read(arg0 context.Context, arg1 ident.ID, arg2 time0.UnixNano, arg3 namespace.Context) ([]xio.BlockReader, bool, error)

Read mocks base method

type MockMergeWithMockRecorder added in v0.10.0 

type MockMergeWithMockRecorder struct {
	// contains filtered or unexported fields
}

MockMergeWithMockRecorder is the mock recorder for MockMergeWith

func (*MockMergeWithMockRecorder) ForEachRemaining added in v0.10.0 

func (mr *MockMergeWithMockRecorder) ForEachRemaining(arg0, arg1, arg2, arg3 interface{}) *gomock.Call

ForEachRemaining indicates an expected call of ForEachRemaining

func (*MockMergeWithMockRecorder) Read added in v0.10.0 

func (mr *MockMergeWithMockRecorder) Read(arg0, arg1, arg2, arg3 interface{}) *gomock.Call

Read indicates an expected call of Read

type MockSnapshotMetadataFileWriter added in v0.7.0 

type MockSnapshotMetadataFileWriter struct {
	// contains filtered or unexported fields
}

MockSnapshotMetadataFileWriter is a mock of SnapshotMetadataFileWriter interface

func NewMockSnapshotMetadataFileWriter added in v0.7.0 

func NewMockSnapshotMetadataFileWriter(ctrl *gomock.Controller) *MockSnapshotMetadataFileWriter

NewMockSnapshotMetadataFileWriter creates a new mock instance

func (*MockSnapshotMetadataFileWriter) EXPECT added in v0.7.0 

func (m *MockSnapshotMetadataFileWriter) EXPECT() *MockSnapshotMetadataFileWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockSnapshotMetadataFileWriter) Write added in v0.7.0 

func (m *MockSnapshotMetadataFileWriter) Write(arg0 SnapshotMetadataWriteArgs) error

Write mocks base method

type MockSnapshotMetadataFileWriterMockRecorder added in v0.7.0 

type MockSnapshotMetadataFileWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockSnapshotMetadataFileWriterMockRecorder is the mock recorder for MockSnapshotMetadataFileWriter

func (*MockSnapshotMetadataFileWriterMockRecorder) Write added in v0.7.0 

func (mr *MockSnapshotMetadataFileWriterMockRecorder) Write(arg0 interface{}) *gomock.Call

Write indicates an expected call of Write

type MockStreamingWriter added in v1.0.0 

type MockStreamingWriter struct {
	// contains filtered or unexported fields
}

MockStreamingWriter is a mock of StreamingWriter interface

func NewMockStreamingWriter added in v1.0.0 

func NewMockStreamingWriter(ctrl *gomock.Controller) *MockStreamingWriter

NewMockStreamingWriter creates a new mock instance

func (*MockStreamingWriter) Abort added in v1.0.0 

func (m *MockStreamingWriter) Abort() error

Abort mocks base method

func (*MockStreamingWriter) Close added in v1.0.0 

func (m *MockStreamingWriter) Close() error

Close mocks base method

func (*MockStreamingWriter) EXPECT added in v1.0.0 

func (m *MockStreamingWriter) EXPECT() *MockStreamingWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockStreamingWriter) Open added in v1.0.0 

func (m *MockStreamingWriter) Open(arg0 StreamingWriterOpenOptions) error

Open mocks base method

func (*MockStreamingWriter) WriteAll added in v1.0.0 

func (m *MockStreamingWriter) WriteAll(arg0 ident.BytesID, arg1 ts.EncodedTags, arg2 [][]byte, arg3 uint32) error

WriteAll mocks base method

type MockStreamingWriterMockRecorder added in v1.0.0 

type MockStreamingWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockStreamingWriterMockRecorder is the mock recorder for MockStreamingWriter

func (*MockStreamingWriterMockRecorder) Abort added in v1.0.0 

func (mr *MockStreamingWriterMockRecorder) Abort() *gomock.Call

Abort indicates an expected call of Abort

func (*MockStreamingWriterMockRecorder) Close added in v1.0.0 

func (mr *MockStreamingWriterMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockStreamingWriterMockRecorder) Open added in v1.0.0 

func (mr *MockStreamingWriterMockRecorder) Open(arg0 interface{}) *gomock.Call

Open indicates an expected call of Open

func (*MockStreamingWriterMockRecorder) WriteAll added in v1.0.0 

func (mr *MockStreamingWriterMockRecorder) WriteAll(arg0, arg1, arg2, arg3 interface{}) *gomock.Call

WriteAll indicates an expected call of WriteAll

type NewMergerFn added in v0.10.0 

type NewMergerFn func(
	reader DataFileSetReader,
	blockAllocSize int,
	srPool xio.SegmentReaderPool,
	multiIterPool encoding.MultiReaderIteratorPool,
	identPool ident.Pool,
	encoderPool encoding.EncoderPool,
	contextPool context.Pool,
	filePathPrefix string,
	nsOpts namespace.Options,
) Merger

NewMergerFn is the function to call to get a new Merger.

type NewReaderFn added in v1.0.1 

type NewReaderFn func(bytesPool pool.CheckedBytesPool, opts Options) (DataFileSetReader, error)

NewReaderFn creates a new DataFileSetReader.

type Options 

type Options interface {
	// Validate will validate the options and return an error if not valid.
	Validate() error

	// SetClockOptions sets the clock options.
	SetClockOptions(value clock.Options) Options

	// ClockOptions returns the clock options.
	ClockOptions() clock.Options

	// SetInstrumentOptions sets the instrumentation options.
	SetInstrumentOptions(value instrument.Options) Options

	// InstrumentOptions returns the instrumentation options.
	InstrumentOptions() instrument.Options

	// SetRuntimeOptionsManager sets the runtime options manager.
	SetRuntimeOptionsManager(value runtime.OptionsManager) Options

	// RuntimeOptionsManager returns the runtime options manager.
	RuntimeOptionsManager() runtime.OptionsManager

	// SetDecodingOptions sets the decoding options.
	SetDecodingOptions(value msgpack.DecodingOptions) Options

	// DecodingOptions returns the decoding options.
	DecodingOptions() msgpack.DecodingOptions

	// SetFilePathPrefix sets the file path prefix for sharded TSDB files.
	SetFilePathPrefix(value string) Options

	// FilePathPrefix returns the file path prefix for sharded TSDB files.
	FilePathPrefix() string

	// SetNewFileMode sets the new file mode.
	SetNewFileMode(value os.FileMode) Options

	// NewFileMode returns the new file mode.
	NewFileMode() os.FileMode

	// SetNewDirectoryMode sets the new directory mode.
	SetNewDirectoryMode(value os.FileMode) Options

	// NewDirectoryMode returns the new directory mode.
	NewDirectoryMode() os.FileMode

	// SetIndexSummariesPercent size sets the percent of index summaries to write.
	SetIndexSummariesPercent(value float64) Options

	// IndexSummariesPercent size returns the percent of index summaries to write.
	IndexSummariesPercent() float64

	// SetIndexBloomFilterFalsePositivePercent size sets the percent of false positive
	// rate to use for the index bloom filter size and k hashes estimation.
	SetIndexBloomFilterFalsePositivePercent(value float64) Options

	// IndexBloomFilterFalsePositivePercent size returns the percent of false positive
	// rate to use for the index bloom filter size and k hashes estimation.
	IndexBloomFilterFalsePositivePercent() float64

	// SetForceIndexSummariesMmapMemory sets whether the summaries files will be mmap'd
	// as an anonymous region, or as a file.
	SetForceIndexSummariesMmapMemory(value bool) Options

	// ForceIndexSummariesMmapMemory returns whether the summaries files will be mmap'd
	// as an anonymous region, or as a file.
	ForceIndexSummariesMmapMemory() bool

	// SetForceBloomFilterMmapMemory sets whether the bloom filters will be mmap'd.
	// as an anonymous region, or as a file.
	SetForceBloomFilterMmapMemory(value bool) Options

	// ForceBloomFilterMmapMemory returns whether the bloom filters will be mmap'd.
	// as an anonymous region, or as a file.
	ForceBloomFilterMmapMemory() bool

	// SetWriterBufferSize sets the buffer size for writing TSDB files.
	SetWriterBufferSize(value int) Options

	// WriterBufferSize returns the buffer size for writing TSDB files.
	WriterBufferSize() int

	// SetInfoReaderBufferSize sets the buffer size for reading TSDB info,
	// digest and checkpoint files.
	SetInfoReaderBufferSize(value int) Options

	// InfoReaderBufferSize returns the buffer size for reading TSDB info,
	// digest and checkpoint files.
	InfoReaderBufferSize() int

	// SetDataReaderBufferSize sets the buffer size for reading TSDB data and index files.
	SetDataReaderBufferSize(value int) Options

	// DataReaderBufferSize returns the buffer size for reading TSDB data and index files.
	DataReaderBufferSize() int

	// SetSeekReaderBufferSize size sets the buffer size for seeking TSDB files.
	SetSeekReaderBufferSize(value int) Options

	// SeekReaderBufferSize size returns the buffer size for seeking TSDB files.
	SeekReaderBufferSize() int

	// SetMmapEnableHugeTLB sets whether mmap huge pages are enabled when running on linux.
	SetMmapEnableHugeTLB(value bool) Options

	// MmapEnableHugeTLB returns whether mmap huge pages are enabled when running on linux.
	MmapEnableHugeTLB() bool

	// SetMmapHugeTLBThreshold sets the threshold when to use mmap huge pages for mmap'd files on linux.
	SetMmapHugeTLBThreshold(value int64) Options

	// MmapHugeTLBThreshold returns the threshold when to use mmap huge pages for mmap'd files on linux.
	MmapHugeTLBThreshold() int64

	// SetTagEncoderPool sets the tag encoder pool.
	SetTagEncoderPool(value serialize.TagEncoderPool) Options

	// TagEncoderPool returns the tag encoder pool.
	TagEncoderPool() serialize.TagEncoderPool

	// SetTagDecoderPool sets the tag decoder pool.
	SetTagDecoderPool(value serialize.TagDecoderPool) Options

	// TagDecoderPool returns the tag decoder pool.
	TagDecoderPool() serialize.TagDecoderPool

	// SetFSTOptions sets the fst options.
	SetFSTOptions(value fst.Options) Options

	// FSTOptions returns the fst options.
	FSTOptions() fst.Options

	// SetFStWriterOptions sets the fst writer options.
	SetFSTWriterOptions(value fst.WriterOptions) Options

	// FSTWriterOptions returns the fst writer options.
	FSTWriterOptions() fst.WriterOptions

	// SetMmapReporter sets the mmap reporter.
	SetMmapReporter(value mmap.Reporter) Options

	// MmapReporter returns the mmap reporter.
	MmapReporter() mmap.Reporter

	// SetIndexReaderAutovalidateIndexSegments sets the index reader to
	// autovalidate index segments data integrity on file open.
	SetIndexReaderAutovalidateIndexSegments(value bool) Options

	// IndexReaderAutovalidateIndexSegments returns the index reader to
	// autovalidate index segments data integrity on file open.
	IndexReaderAutovalidateIndexSegments() bool

	// SetEncodingOptions sets the encoder options used by the encoder.
	SetEncodingOptions(value msgpack.LegacyEncodingOptions) Options

	// EncodingOptions returns the encoder options used by the encoder.
	EncodingOptions() msgpack.LegacyEncodingOptions
}

Options represents the options for filesystem persistence.

func NewOptions 

func NewOptions() Options

NewOptions creates a new set of fs options

type ReadIndexInfoFileResult 

type ReadIndexInfoFileResult struct {
	ID                FileSetFileIdentifier
	Info              index.IndexVolumeInfo
	AbsoluteFilePaths []string
	Err               ReadInfoFileResultError
}

ReadIndexInfoFileResult is the result of reading an info file

func ReadIndexInfoFiles 

func ReadIndexInfoFiles(
	filePathPrefix string,
	namespace ident.ID,
	readerBufferSize int,
) []ReadIndexInfoFileResult

ReadIndexInfoFiles reads all the valid index info entries. Even if ReadIndexInfoFiles returns an error, there may be some valid entries in the returned slice.

type ReadIndexSegmentsOptions 

type ReadIndexSegmentsOptions struct {
	// ReaderOptions is the index file set reader options.
	ReaderOptions IndexReaderOpenOptions

	// FilesystemOptions is the filesystem options which is
	// required for reading index segments.
	FilesystemOptions Options
	// contains filtered or unexported fields
}

ReadIndexSegmentsOptions is a set of options used when reading index segments.

type ReadIndexSegmentsResult added in v1.0.0 

type ReadIndexSegmentsResult struct {
	Segments  []segment.Segment
	Validated bool
}

ReadIndexSegmentsResult is the result of a call to ReadIndexSegments.

func ReadIndexSegments 

func ReadIndexSegments(
	opts ReadIndexSegmentsOptions,
) (ReadIndexSegmentsResult, error)

ReadIndexSegments will read a set of segments.

type ReadInfoFileResult 

type ReadInfoFileResult struct {
	Info schema.IndexInfo
	Err  ReadInfoFileResultError
}

ReadInfoFileResult is the result of reading an info file

func ReadInfoFiles 

func ReadInfoFiles(
	filePathPrefix string,
	namespace ident.ID,
	shard uint32,
	readerBufferSize int,
	decodingOpts msgpack.DecodingOptions,
	fileSetType persist.FileSetType,
) []ReadInfoFileResult

ReadInfoFiles reads all the valid info entries. Even if ReadInfoFiles returns an error, there may be some valid entries in the returned slice.

type ReadInfoFileResultError 

type ReadInfoFileResultError interface {
	Error() error
	Filepath() string
}

ReadInfoFileResultError is the interface for obtaining information about an error that occurred trying to read an info file

type ReaderOpenOptionsMatcher 

type ReaderOpenOptionsMatcher struct {
	ID          FileSetFileIdentifier
	FileSetType persist.FileSetType
}

ReaderOpenOptionsMatcher is a matcher for the DataReaderOpenOptions struct

func (ReaderOpenOptionsMatcher) Matches 

func (m ReaderOpenOptionsMatcher) Matches(x interface{}) bool

Matches determine whether m matches a DataWriterOpenOptions

func (ReaderOpenOptionsMatcher) String 

func (m ReaderOpenOptionsMatcher) String() string

type RetrievableDataBlockSegmentReader 

type RetrievableDataBlockSegmentReader interface {
	xio.SegmentReader
}

RetrievableDataBlockSegmentReader is a retrievable block reader.

type RetrieveRequestPool added in v0.15.0 

type RetrieveRequestPool interface {
	// Init initializes the request pool.
	Init()
	// Get gets a retrieve request.
	Get() *retrieveRequest
	// Put returns a retrieve request to the pool.
	Put(req *retrieveRequest)
}

RetrieveRequestPool is the retrieve request pool.

func NewRetrieveRequestPool added in v0.15.0 

func NewRetrieveRequestPool(
	segmentReaderPool xio.SegmentReaderPool,
	opts pool.ObjectPoolOptions,
) RetrieveRequestPool

NewRetrieveRequestPool returns a new retrieve request pool.

type ReusableSeekerResources added in v0.8.0 

type ReusableSeekerResources struct {
	// contains filtered or unexported fields
}

ReusableSeekerResources is a collection of reusable resources that the seeker requires for seeking. It can be pooled by callers using the seeker so that expensive resources don't need to be maintained for each seeker, especially when only a few are generally being used at a time due to the FetchConcurrency.

func NewReusableSeekerResources added in v0.8.0 

func NewReusableSeekerResources(opts Options) ReusableSeekerResources

NewReusableSeekerResources creates a new ReusableSeekerResources.

type Segments added in v0.15.0 

type Segments interface {
	ShardTimeRanges() result.ShardTimeRanges
	VolumeType() idxpersist.IndexVolumeType
	VolumeIndex() int
	AbsoluteFilePaths() []string
	BlockStart() time.Time
}

Segments represents on index segments on disk for an index volume.

func NewSegments added in v0.15.0 

func NewSegments(
	info index.IndexVolumeInfo,
	volumeIndex int,
	absoluteFilepaths []string,
) Segments

NewSegments returns an on disk segments for an index info file.

type SnapshotMetadata added in v0.4.8 

type SnapshotMetadata struct {
	ID                  SnapshotMetadataIdentifier
	CommitlogIdentifier persist.CommitLogFile
	MetadataFilePath    string
	CheckpointFilePath  string
}

SnapshotMetadata represents a SnapshotMetadata file, along with its checkpoint file, as well as all the information contained within the metadata file and paths to the physical files on disk.

func (SnapshotMetadata) AbsoluteFilePaths added in v0.15.0 

func (s SnapshotMetadata) AbsoluteFilePaths() []string

AbsoluteFilePaths returns a slice of all the absolute filepaths associated with a snapshot metadata.

type SnapshotMetadataErrorWithPaths added in v0.4.8 

type SnapshotMetadataErrorWithPaths struct {
	Error              error
	MetadataFilePath   string
	CheckpointFilePath string
}

SnapshotMetadataErrorWithPaths contains an error that occurred while trying to read a snapshot metadata file, as well as paths for the metadata file path and the checkpoint file path so that they can be cleaned up. The checkpoint file may not exist if only the metadata file was written out (due to sudden node failure) or if the metadata file name was structured incorrectly (should never happen.)

type SnapshotMetadataFileReader added in v0.7.0 

type SnapshotMetadataFileReader interface {
	Read(id SnapshotMetadataIdentifier) (SnapshotMetadata, error)
}

SnapshotMetadataFileReader reads snapshot metadata files.

type SnapshotMetadataFileWriter added in v0.7.0 

type SnapshotMetadataFileWriter interface {
	Write(args SnapshotMetadataWriteArgs) error
}

SnapshotMetadataFileWriter writes out snapshot metadata files.

type SnapshotMetadataIdentifier added in v0.4.8 

type SnapshotMetadataIdentifier struct {
	Index int64
	UUID  uuid.UUID
}

SnapshotMetadataIdentifier is an identifier for a snapshot metadata file

type SnapshotMetadataReader added in v0.4.8 

type SnapshotMetadataReader struct {
	// contains filtered or unexported fields
}

SnapshotMetadataReader is a reader for SnapshotMetadata.

func NewSnapshotMetadataReader added in v0.4.8 

func NewSnapshotMetadataReader(opts Options) *SnapshotMetadataReader

NewSnapshotMetadataReader returns a new SnapshotMetadataReader.

func (*SnapshotMetadataReader) Read added in v0.4.8 

func (w *SnapshotMetadataReader) Read(id SnapshotMetadataIdentifier) (SnapshotMetadata, error)

type SnapshotMetadataWriteArgs added in v0.4.8 

type SnapshotMetadataWriteArgs struct {
	ID                  SnapshotMetadataIdentifier
	CommitlogIdentifier persist.CommitLogFile
}

SnapshotMetadataWriteArgs are the arguments for SnapshotMetadataWriter.Write.

type SnapshotMetadataWriter added in v0.4.8 

type SnapshotMetadataWriter struct {
	// contains filtered or unexported fields
}

SnapshotMetadataWriter is a writer for SnapshotMetadata.

func NewSnapshotMetadataWriter added in v0.4.8 

func NewSnapshotMetadataWriter(opts Options) *SnapshotMetadataWriter

NewSnapshotMetadataWriter constructs a new snapshot metadata writer.

func (*SnapshotMetadataWriter) Write added in v0.4.8 

func (w *SnapshotMetadataWriter) Write(args SnapshotMetadataWriteArgs) (finalErr error)

type StreamedDataEntry added in v1.0.1 

type StreamedDataEntry struct {
	ID           ident.BytesID
	EncodedTags  ts.EncodedTags
	Data         []byte
	DataChecksum uint32
}

StreamedDataEntry contains the data of single entry returned by streaming method(s). The underlying data slices are reused and invalidated on every read.

type StreamingWriter added in v0.15.15 

type StreamingWriter interface {
	io.Closer

	// Open opens the files for writing data to the given shard in the given namespace.
	Open(opts StreamingWriterOpenOptions) error

	// WriteAll will write the id and all byte slices and returns an error on a write error.
	// Callers should call this method with strictly lexicographically increasing ID values.
	WriteAll(id ident.BytesID, encodedTags ts.EncodedTags, data [][]byte, dataChecksum uint32) error

	// Abort closes the file descriptors without writing out a checkpoint file.
	Abort() error
}

StreamingWriter writes into data fileset without intermediate buffering. Writes must be lexicographically ordered by the id.

func NewStreamingWriter added in v0.15.15 

func NewStreamingWriter(opts Options) (StreamingWriter, error)

NewStreamingWriter creates a new streaming writer that writes into the data fileset without buffering.

type StreamingWriterOpenOptions added in v1.0.0 

type StreamingWriterOpenOptions struct {
	NamespaceID ident.ID
	ShardID     uint32
	BlockStart  time.Time
	BlockSize   time.Duration
	VolumeIndex int

	// PlannedRecordsCount is an estimate of the number of series to be written.
	// Must be greater than 0.
	PlannedRecordsCount uint
}

StreamingWriterOpenOptions in the options for the StreamingWriter.

Source Files

View all Source files

Directories

Path Synopsis
Package commitlog is a generated GoMock package.
 Package commitlog is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.