storage

package
v0.0.0-...-d500d3c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 17, 2019 License: MIT Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DefaultRetentionInterval   = 1 * time.Hour
	DefaultValidateKeys        = false
	DefaultTraceLoggingEnabled = false

	DefaultSeriesFileDirectoryName = "_series"
	DefaultIndexDirectoryName      = "index"
	DefaultWALDirectoryName        = "wal"
	DefaultEngineDirectoryName     = "data"
)

Variables

View Source 
var ErrEngineClosed = errors.New("engine is closed")

ErrEngineClosed is returned when a caller attempts to use the engine while it's closed.

View Source 
var ErrServiceClosed = errors.New("service is currently closed")

ErrServiceClosed is returned when the service is unavailable.

Functions

This section is empty.

Types

type BucketDeleter 

type BucketDeleter interface {
	DeleteBucket(platform.ID, platform.ID) error
}

BucketDeleter defines the behaviour of deleting a bucket.

type BucketFinder 

type BucketFinder interface {
	FindBuckets(context.Context, platform.BucketFilter, ...platform.FindOptions) ([]*platform.Bucket, int, error)
}

A BucketFinder is responsible for providing access to buckets via a filter.

type BucketService 

type BucketService struct {
	// contains filtered or unexported fields
}

BucketService wraps an existing platform.BucketService implementation.

BucketService ensures that when a bucket is deleted, all stored data associated with the bucket is either removed, or marked to be removed via a future compaction.

func NewBucketService 

func NewBucketService(s platform.BucketService, engine BucketDeleter) *BucketService

NewBucketService returns a new BucketService for the provided BucketDeleter, which typically will be an Engine.

func (*BucketService) CreateBucket 

func (s *BucketService) CreateBucket(ctx context.Context, b *platform.Bucket) error

CreateBucket creates a new bucket and sets b.ID with the new identifier.

func (*BucketService) DeleteBucket 

func (s *BucketService) DeleteBucket(ctx context.Context, bucketID platform.ID) error

DeleteBucket removes a bucket by ID.

func (*BucketService) FindBucket 

func (s *BucketService) FindBucket(ctx context.Context, filter platform.BucketFilter) (*platform.Bucket, error)

FindBucket returns the first bucket that matches filter.

func (*BucketService) FindBucketByID 

func (s *BucketService) FindBucketByID(ctx context.Context, id platform.ID) (*platform.Bucket, error)

FindBucketByID returns a single bucket by ID.

func (*BucketService) FindBuckets 

func (s *BucketService) FindBuckets(ctx context.Context, filter platform.BucketFilter, opt ...platform.FindOptions) ([]*platform.Bucket, int, error)

FindBuckets returns a list of buckets that match filter and the total count of matching buckets. Additional options provide pagination & sorting.

func (*BucketService) UpdateBucket 

func (s *BucketService) UpdateBucket(ctx context.Context, id platform.ID, upd platform.BucketUpdate) (*platform.Bucket, error)

UpdateBucket updates a single bucket with changeset. Returns the new bucket state after update.

type Config 

type Config struct {
	// Frequency of retention in seconds.
	RetentionInterval toml.Duration `toml:"retention-interval"`

	// Enables unicode validation on series keys on write.
	ValidateKeys bool `toml:"validate-keys"`

	// Enables trace logging for the engine.
	TraceLoggingEnabled bool `toml:"trace-logging-enabled"`

	// Series file config.
	SeriesFilePath string `toml:"series-file-path"` // Overrides the default path.

	// WAL config.
	WAL     tsm1.WALConfig `toml:"wal"`
	WALPath string         `toml:"wal-path"` // Overrides the default path.

	// Engine config.
	Engine     tsm1.Config `toml:"engine"`
	EnginePath string      `toml:"engine-path"` // Overrides the default path.

	// Index config.
	Index     tsi1.Config `toml:"index"`
	IndexPath string      `toml:"index-path"` // Overrides the default path.
}

Config holds the configuration for an Engine.

func NewConfig 

func NewConfig() Config

NewConfig initialises a new config for an Engine.

func (Config) GetEnginePath 

func (c Config) GetEnginePath(base string) string

GetEnginePath returns the path to the engine.

func (Config) GetIndexPath 

func (c Config) GetIndexPath(base string) string

GetIndexPath returns the path to the index.

func (Config) GetSeriesFilePath 

func (c Config) GetSeriesFilePath(base string) string

GetSeriesFilePath returns the path to the series file.

func (Config) GetWALPath 

func (c Config) GetWALPath(base string) string

GetWALPath returns the path to the WAL.

type Deleter 

type Deleter interface {
	CreateSeriesCursor(context.Context, SeriesCursorRequest, influxql.Expr) (SeriesCursor, error)
	DeleteSeriesRangeWithPredicate(tsdb.SeriesIterator, func([]byte, models.Tags) (int64, int64, bool)) error
}

A Deleter implementation is capable of deleting data from a storage engine.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

func NewEngine 

func NewEngine(path string, c Config, options ...Option) *Engine

NewEngine initialises a new storage engine, including a series file, index and TSM engine.

func (*Engine) ApplyFnToSeriesIDSet 

func (e *Engine) ApplyFnToSeriesIDSet(fn func(*tsdb.SeriesIDSet))

ApplyFnToSeriesIDSet allows the caller to apply fn to the SeriesIDSet held within the engine's index.

func (*Engine) Close 

func (e *Engine) Close() error

Close closes the store and all underlying resources. It returns an error if any of the underlying systems fail to close.

func (*Engine) CreateCursorIterator 

func (e *Engine) CreateCursorIterator(ctx context.Context) (tsdb.CursorIterator, error)

func (*Engine) CreateSeriesCursor 

func (e *Engine) CreateSeriesCursor(ctx context.Context, req SeriesCursorRequest, cond influxql.Expr) (SeriesCursor, error)

func (*Engine) DeleteBucket 

func (e *Engine) DeleteBucket(orgID, bucketID platform.ID) error

DeleteBucket deletes an entire bucket from the storage engine.

func (*Engine) DeleteSeriesRangeWithPredicate 

func (e *Engine) DeleteSeriesRangeWithPredicate(itr tsdb.SeriesIterator, fn func([]byte, models.Tags) (int64, int64, bool)) error

DeleteSeriesRangeWithPredicate deletes all series data iterated over if fn returns true for that series.

func (*Engine) MeasurementCardinalityStats 

func (e *Engine) MeasurementCardinalityStats() tsi1.MeasurementCardinalityStats

MeasurementCardinalityStats returns cardinality stats for all measurements.

func (*Engine) MeasurementStats 

func (e *Engine) MeasurementStats() (tsm1.MeasurementStats, error)

MeasurementStats returns the current measurement stats for the engine.

func (*Engine) Open 

func (e *Engine) Open() error

Open opens the store and all underlying resources. It returns an error if any of the underlying systems fail to open.

func (*Engine) Path 

func (e *Engine) Path() string

Path returns the path of the engine's base directory.

func (*Engine) PrometheusCollectors 

func (e *Engine) PrometheusCollectors() []prometheus.Collector

PrometheusCollectors returns all the prometheus collectors associated with the engine and its components.

func (*Engine) SeriesCardinality 

func (e *Engine) SeriesCardinality() int64

SeriesCardinality returns the number of series in the engine.

func (*Engine) WithLogger 

func (e *Engine) WithLogger(log *zap.Logger)

WithLogger sets the logger on the Store. It must be called before Open.

func (*Engine) WritePoints 

func (e *Engine) WritePoints(points []models.Point) error

WritePoints writes the provided points to the engine.

The Engine expects all points to have been correctly validated by the caller. WritePoints will however determine if there are any field type conflicts, and return an appropriate error in that case.

type Option 

type Option func(*Engine)

Option provides a set

func WithCompactionPlanner 

func WithCompactionPlanner(planner tsm1.CompactionPlanner) Option

WithCompactionPlanner makes the engine have the provided compaction planner.

func WithEngineID 

func WithEngineID(id int) Option

WithEngineID sets an engine id, which can be useful for logging when multiple engines are in use.

func WithFileStoreObserver 

func WithFileStoreObserver(obs tsm1.FileStoreObserver) Option

WithFileStoreObserver makes the engine have the provided file store observer.

func WithNodeID 

func WithNodeID(id int) Option

WithNodeID sets a node id on the engine, which can be useful for logging when a system has engines running on multiple nodes.

func WithRetentionEnforcer 

func WithRetentionEnforcer(finder BucketFinder) Option

WithRetentionEnforcer initialises a retention enforcer on the engine. WithRetentionEnforcer must be called after other options to ensure that all metrics are labelled correctly.

func WithTSMFilenameFormatter 

func WithTSMFilenameFormatter(fn tsm1.FormatFileNameFunc) Option

WithTSMFilenameFormatter sets a function on the underlying tsm1.Engine to specify how TSM files are named.

type PointsWriter 

type PointsWriter interface {
	WritePoints([]models.Point) error
}

PointsWriter describes the ability to write points into a storage engine.

type SeriesCursor 

type SeriesCursor interface {
	Close() error
	Next() (*SeriesCursorRow, error)
}

type SeriesCursorRequest 

type SeriesCursorRequest struct {
	Measurements tsdb.MeasurementIterator
}

type SeriesCursorRow 

type SeriesCursorRow struct {
	Name []byte
	Tags models.Tags
}

Source Files

View all Source files

Directories

Path Synopsis
package compat helps with migrating toml files from influxdb.
 package compat helps with migrating toml files from influxdb.
datatypes

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.