tsdb

package
v0.10.0-beta2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 23, 2016 License: MIT Imports: 32 Imported by: 0

README

Line Protocol

The line protocol is a text based format for writing points to InfluxDB. Each line defines a single point. Multiple lines must be separated by the newline character \n. The format of the line consists of three parts:

[key] [fields] [timestamp]

Each section is separated by spaces. The minimum required point consists of a measurement name and at least one field. Points without a specified timestamp will be written using the server's local timestamp. Timestamps are assumed to be in nanoseconds unless a precision value is passed in the query string.

Key

The key is the measurement name and any optional tags separated by commas. Measurement names, tag keys, and tag values must escape any spaces or commas using a backslash (\). For example: \ and \,. All tag values are stored as strings and should not be surrounded in quotes.

Tags should be sorted by key before being sent for best performance. The sort should match that from the Go bytes.Compare function (http://golang.org/pkg/bytes/#Compare).

Examples
# measurement only
cpu

# measurment and tags
cpu,host=serverA,region=us-west

# measurment with commas
cpu\,01,host=serverA,region=us-west

# tag value with spaces
cpu,host=server\ A,region=us\ west

Fields

Fields are key-value metrics associated with the measurement. Every line must have at least one field. Multiple fields must be separated with commas and not spaces.

Field keys are always strings and follow the same syntactical rules as described above for tag keys and values. Field values can be one of four types. The first value written for a given field on a given measurement defines the type of that field for all series under that measurement.

  • integer - Numeric values that do not include a decimal and are followed by a trailing i when inserted (e.g. 1i, 345i, 2015i, -10i). Note that all values must have a trailing i. If they do not they will be written as floats.
  • float - Numeric values tha are not followed by a trailing i. (e.g. 1, 1.0, -3.14, 6.0+e5, 10).
  • boolean - A value indicating true or false. Valid boolean strings are (t, T, true, TRUE, f, F, false, and FALSE).
  • string - A text value. All string values must be surrounded in double-quotes ". If the string contains a double-quote or backslashes, it must be escaped with a backslash, e.g. \", \\.
# integer value
cpu value=1i

cpu value=1.1i # will result in a parse error

# float value
cpu_load value=1

cpu_load value=1.0

cpu_load value=1.2

# boolean value
error fatal=true

# string value
event msg="logged out"

# multiple values
cpu load=10,alert=true,reason="value above maximum threshold"

Timestamp

The timestamp section is optional but should be specified if possible. The value is an integer representing nanoseconds since the epoch. If the timestamp is not provided the point will inherit the server's local timestamp.

Some write APIs allow passing a lower precision. If the API supports a lower precision, the timestamp may also be an integer epoch in microseconds, milliseconds, seconds, minutes or hours.

Full Example

A full example is shown below.

cpu,host=server01,region=uswest value=1 1434055562000000000
cpu,host=server02,region=uswest value=3 1434055562000010000

In this example the first line shows a measurement of "cpu", there are two tags "host" and "region, the value is 1.0, and the timestamp is 1434055562000000000. Following this is a second line, also a point in the measurement "cpu" but belonging to a different "host".

cpu,host=server\ 01,region=uswest value=1,msg="all systems nominal"
cpu,host=server\ 01,region=us\,west value_int=1i

In these examples, the "host" is set to server 01. The field value associated with field key msg is double-quoted, as it is a string. The second example shows a region of us,west with the comma properly escaped. In the first example value is written as a floating point number. In the second, value_int is an integer.

Distributed Queries

Documentation

Overview

Package tsdb implements a durable time series database.

Index

Constants

View Source 
const (
	// DefaultEngine is the default engine for new shards
	DefaultEngine = "tsm1"

	// DefaultMaxWALSize is the default size of the WAL before it is flushed.
	DefaultMaxWALSize = 100 * 1024 * 1024 // 100MB

	// DefaultWALFlushInterval is the frequency the WAL will get flushed if
	// it doesn't reach its size threshold.
	DefaultWALFlushInterval = 10 * time.Minute

	// DefaultWALPartitionFlushDelay is the sleep time between WAL partition flushes.
	DefaultWALPartitionFlushDelay = 2 * time.Second

	// DefaultReadySeriesSize of 32KB specifies when a series is eligible to be flushed
	DefaultReadySeriesSize = 30 * 1024

	// DefaultCompactionThreshold flush and compact a partition once this ratio of keys are over the flush size
	DefaultCompactionThreshold = 0.5

	// DefaultMaxSeriesSize specifies the size at which a series will be forced to flush
	DefaultMaxSeriesSize = 1024 * 1024

	// DefaultFlushColdInterval specifies how long after a partition has been cold
	// for writes that a full flush and compaction are forced
	DefaultFlushColdInterval = 5 * time.Second

	// DefaultParititionSizeThreshold specifies when a partition gets to this size in
	// memory, we should slow down writes until it gets a chance to compact.
	// This will force clients to get backpressure if they're writing too fast. We need
	// this because the WAL can take writes much faster than the index. So eventually
	// we'll need to create backpressure, otherwise we'll fill up the memory and die.
	// This number multiplied by the parition count is roughly the max possible memory
	// size for the in-memory WAL cache.
	DefaultPartitionSizeThreshold = 50 * 1024 * 1024 // 50MB

	// DefaultCacheMaxMemorySize is the maximum size a shard's cache can
	// reach before it starts rejecting writes.
	DefaultCacheMaxMemorySize = 500 * 1024 * 1024 // 500MB

	// DefaultCacheSnapshotMemorySize is the size at which the engine will
	// snapshot the cache and write it to a TSM file, freeing up memory
	DefaultCacheSnapshotMemorySize = 25 * 1024 * 1024 // 25MB

	// DefaultCacheSnapshotWriteColdDuration is the length of time at which
	// the engine will snapshot the cache and write it to a new TSM file if
	// the shard hasn't received writes or deletes
	DefaultCacheSnapshotWriteColdDuration = time.Duration(time.Hour)

	// DefaultCompactFullWriteColdDuration is the duration at which the engine
	// will compact all TSM files in a shard if it hasn't received a write or delete
	DefaultCompactFullWriteColdDuration = time.Duration(24 * time.Hour)

	// DefaultMaxPointsPerBlock is the maximum number of points in an encoded
	// block in a TSM file
	DefaultMaxPointsPerBlock = 1000
)
View Source 
const (
	// Return an error if the user is trying to select more than this number of points in a group by statement.
	// Most likely they specified a group by interval without time boundaries.
	MaxGroupByPoints = 100000

	// Since time is always selected, the column count when selecting only a single other value will be 2
	SelectColumnCountWithOneValue = 2

	// IgnoredChunkSize is what gets passed into Mapper.Begin for aggregate queries as they don't chunk points out
	IgnoredChunkSize = 0
)
View Source 
const EOF = int64(-1)

EOF represents a "not found" key returned by a Cursor.

View Source 
const (
	MaintenanceCheckInterval = time.Minute
)

Variables

View Source 
var (
	// ErrInvalidQuery is returned when executing an unknown query type.
	ErrInvalidQuery = errors.New("invalid query")

	// ErrNotExecuted is returned when a statement is not executed in a query.
	// This can occur when a previous statement in the same query has errored.
	ErrNotExecuted = errors.New("not executed")
)
View Source 
var (
	// ErrFieldOverflow is returned when too many fields are created on a measurement.
	ErrFieldOverflow = errors.New("field overflow")

	// ErrFieldTypeConflict is returned when a new field already exists with a different type.
	ErrFieldTypeConflict = errors.New("field type conflict")

	// ErrFieldNotFound is returned when a field cannot be found.
	ErrFieldNotFound = errors.New("field not found")

	// ErrFieldUnmappedID is returned when the system is presented, during decode, with a field ID
	// there is no mapping for.
	ErrFieldUnmappedID = errors.New("field ID not mapped")
)
View Source 
var (
	ErrShardNotFound = fmt.Errorf("shard not found")
	ErrStoreClosed   = fmt.Errorf("store is closed")
)
View Source 
var (
	// ErrFormatNotFound is returned when no format can be determined from a path.
	ErrFormatNotFound = errors.New("format not found")
)

Functions

func DedupeEntries added in v0.9.3 

func DedupeEntries(a [][]byte) [][]byte

DedupeEntries returns slices with unique keys (the first 8 bytes).

func ErrDatabaseNotFound 

func ErrDatabaseNotFound(name string) error

func ErrMeasurementNotFound 

func ErrMeasurementNotFound(name string) error

func IsNumeric added in v0.10.0 

func IsNumeric(c *influxql.Call) bool

IsNumeric returns whether a given aggregate can only be run on numeric fields.

func IsRetryable 

func IsRetryable(err error) bool

IsRetryable returns true if this error is temporary and could be retried

func MapCount added in v0.10.0 

func MapCount(input *MapInput) interface{}

MapCount computes the number of values in an iterator.

func MapCountDistinct added in v0.10.0 

func MapCountDistinct(input *MapInput) interface{}

MapCountDistinct computes the unique count of values in an iterator.

func MapDistinct added in v0.10.0 

func MapDistinct(input *MapInput) interface{}

MapDistinct computes the unique values in an iterator.

func MapEcho added in v0.10.0 

func MapEcho(input *MapInput) interface{}

MapEcho emits the data points for each group by interval

func MapFirst added in v0.10.0 

func MapFirst(input *MapInput, fieldName string) interface{}

MapFirst collects the values to pass to the reducer This function assumes time ordered input

func MapLast added in v0.10.0 

func MapLast(input *MapInput, fieldName string) interface{}

MapLast collects the values to pass to the reducer

func MapMax added in v0.10.0 

func MapMax(input *MapInput, fieldName string) interface{}

MapMax collects the values to pass to the reducer

func MapMean added in v0.10.0 

func MapMean(input *MapInput) interface{}

MapMean computes the count and sum of values in an iterator to be combined by the reducer.

func MapMin added in v0.10.0 

func MapMin(input *MapInput, fieldName string) interface{}

MapMin collects the values to pass to the reducer

func MapRawQuery added in v0.10.0 

func MapRawQuery(input *MapInput) interface{}

MapRawQuery is for queries without aggregates

func MapSpread added in v0.10.0 

func MapSpread(input *MapInput) interface{}

MapSpread collects the values to pass to the reducer

func MapStddev added in v0.10.0 

func MapStddev(input *MapInput) interface{}

MapStddev collects the values to pass to the reducer

func MapSum added in v0.10.0 

func MapSum(input *MapInput) interface{}

MapSum computes the summation of values in an iterator.

func MapTopBottom added in v0.10.0 

func MapTopBottom(input *MapInput, limit int, fields []string, argCount int, callName string) interface{}

MapTopBottom emits the top/bottom data points for each group by interval

func MarshalTags added in v0.9.3 

func MarshalTags(tags map[string]string) []byte

used to convert the tag set to bytes for use as a lookup key

func MeasurementFromSeriesKey added in v0.9.3 

func MeasurementFromSeriesKey(key string) string

func ProcessAggregateDerivative added in v0.9.3 

func ProcessAggregateDerivative(results [][]interface{}, isNonNegative bool, interval time.Duration) [][]interface{}

ProcessAggregateDerivative returns the derivatives of an aggregate result set

func ReduceCountDistinct added in v0.10.0 

func ReduceCountDistinct(values []interface{}) interface{}

ReduceCountDistinct finds the unique counts of values.

func ReduceDistinct added in v0.10.0 

func ReduceDistinct(values []interface{}) interface{}

ReduceDistinct finds the unique values for each key.

func ReduceFirst added in v0.10.0 

func ReduceFirst(values []interface{}) interface{}

ReduceFirst computes the first of value.

func ReduceLast added in v0.10.0 

func ReduceLast(values []interface{}) interface{}

ReduceLast computes the last of value.

func ReduceMax added in v0.10.0 

func ReduceMax(values []interface{}) interface{}

ReduceMax computes the max of value.

func ReduceMean added in v0.10.0 

func ReduceMean(values []interface{}) interface{}

ReduceMean computes the mean of values for each key.

func ReduceMedian added in v0.10.0 

func ReduceMedian(values []interface{}) interface{}

ReduceMedian computes the median of values

func ReduceMin added in v0.10.0 

func ReduceMin(values []interface{}) interface{}

ReduceMin computes the min of value.

func ReducePercentile added in v0.10.0 

func ReducePercentile(values []interface{}, percentile float64) interface{}

ReducePercentile computes the percentile of values for each key.

func ReduceSpread added in v0.10.0 

func ReduceSpread(values []interface{}) interface{}

ReduceSpread computes the spread of values.

func ReduceStddev added in v0.10.0 

func ReduceStddev(values []interface{}) interface{}

ReduceStddev computes the stddev of values.

func ReduceSum added in v0.10.0 

func ReduceSum(values []interface{}) interface{}

ReduceSum computes the sum of values for each key.

func ReduceTopBottom added in v0.10.0 

func ReduceTopBottom(values []interface{}, limit int, fields []string, callName string) interface{}

ReduceTop computes the top values for each key. This function assumes that its inputs are in sorted ascending order.

func RegisterEngine added in v0.9.3 

func RegisterEngine(name string, fn NewEngineFunc)

RegisterEngine registers a storage engine initializer by name.

func RegisteredEngines added in v0.10.0 

func RegisteredEngines() []string

RegisteredEngines returns the slice of currently registered engines.

Types

type AggregateExecutor 

type AggregateExecutor struct {
	// contains filtered or unexported fields
}

AggregateExecutor represents a mapper for execute aggregate SELECT statements.

func NewAggregateExecutor 

func NewAggregateExecutor(stmt *influxql.SelectStatement, mappers []Mapper) *AggregateExecutor

NewAggregateExecutor returns a new AggregateExecutor.

func (*AggregateExecutor) Execute 

func (e *AggregateExecutor) Execute(closing <-chan struct{}) <-chan *models.Row

Execute begins execution of the query and returns a channel to receive rows.

type AggregateMapper added in v0.10.0 

type AggregateMapper struct {
	// contains filtered or unexported fields
}

AggregateMapper runs the map phase for aggregate SELECT queries.

func NewAggregateMapper added in v0.10.0 

func NewAggregateMapper(sh *Shard, stmt *influxql.SelectStatement) *AggregateMapper

NewAggregateMapper returns a new instance of AggregateMapper.

func (*AggregateMapper) Close added in v0.10.0 

func (m *AggregateMapper) Close()

Close closes the mapper.

func (*AggregateMapper) Fields added in v0.10.0 

func (m *AggregateMapper) Fields() []string

Fields returns all SELECT fields.

func (*AggregateMapper) NextChunk added in v0.10.0 

func (m *AggregateMapper) NextChunk() (interface{}, error)

NextChunk returns the next interval of data. Tagsets are always processed in the same order as AvailTagsSets(). When there is no more data for any tagset nil is returned.

func (*AggregateMapper) Open added in v0.10.0 

func (m *AggregateMapper) Open() error

Open opens and initializes the mapper.

func (*AggregateMapper) TagSets added in v0.10.0 

func (m *AggregateMapper) TagSets() []string

TagSets returns the list of tag sets for which this mapper has data.

type ByteSlices added in v0.9.3 

type ByteSlices [][]byte

func (ByteSlices) Len added in v0.9.3 

func (a ByteSlices) Len() int

func (ByteSlices) Less added in v0.9.3 

func (a ByteSlices) Less(i, j int) bool

func (ByteSlices) Swap added in v0.9.3 

func (a ByteSlices) Swap(i, j int)

type Config 

type Config struct {
	Enabled bool   `toml:"enabled"`
	Dir     string `toml:"dir"`
	Engine  string `toml:"engine"`

	// WAL config options for b1 (introduced in 0.9.2)
	MaxWALSize             int           `toml:"max-wal-size"`
	WALFlushInterval       toml.Duration `toml:"wal-flush-interval"`
	WALPartitionFlushDelay toml.Duration `toml:"wal-partition-flush-delay"`

	// WAL configuration options for bz1 (introduced in 0.9.3)
	WALDir                    string        `toml:"wal-dir"`
	WALLoggingEnabled         bool          `toml:"wal-logging-enabled"`
	WALReadySeriesSize        int           `toml:"wal-ready-series-size"`
	WALCompactionThreshold    float64       `toml:"wal-compaction-threshold"`
	WALMaxSeriesSize          int           `toml:"wal-max-series-size"`
	WALFlushColdInterval      toml.Duration `toml:"wal-flush-cold-interval"`
	WALPartitionSizeThreshold uint64        `toml:"wal-partition-size-threshold"`

	// Query logging
	QueryLogEnabled bool `toml:"query-log-enabled"`

	// Compaction options for tsm1 (descriptions above with defaults)
	CacheMaxMemorySize             uint64        `toml:"cache-max-memory-size"`
	CacheSnapshotMemorySize        uint64        `toml:"cache-snapshot-memory-size"`
	CacheSnapshotWriteColdDuration toml.Duration `toml:"cache-snapshot-write-cold-duration"`
	CompactFullWriteColdDuration   toml.Duration `toml:"compact-full-write-cold-duration"`
	MaxPointsPerBlock              int           `toml:"max-points-per-block"`

	DataLoggingEnabled bool `toml:"data-logging-enabled"`
}

func NewConfig 

func NewConfig() Config

func (*Config) Validate added in v0.10.0 

func (c *Config) Validate() error

type Cursor added in v0.9.3 

type Cursor interface {
	SeekTo(seek int64) (key int64, value interface{})
	Next() (key int64, value interface{})
	Ascending() bool
}

Cursor represents an iterator over a series.

func MultiCursor added in v0.9.3 

func MultiCursor(cursors ...Cursor) Cursor

MultiCursor returns a single cursor that combines the results of all cursors in order.

If the same key is returned from multiple cursors then the first cursor specified will take precendence. A key will only be returned once from the returned cursor.

type CursorSet added in v0.10.0 

type CursorSet struct {
	Measurement string
	Tags        map[string]string
	Key         string
	Cursors     []*TagsCursor
}

type CursorSets added in v0.10.0 

type CursorSets []CursorSet

CursorSets represents a sortable slice of CursorSet.

func (CursorSets) Keys added in v0.10.0 

func (a CursorSets) Keys() []string

func (CursorSets) Len added in v0.10.0 

func (a CursorSets) Len() int

func (CursorSets) Less added in v0.10.0 

func (a CursorSets) Less(i, j int) bool

func (CursorSets) Swap added in v0.10.0 

func (a CursorSets) Swap(i, j int)

type DatabaseIndex 

type DatabaseIndex struct {
	// contains filtered or unexported fields
}

DatabaseIndex is the in memory index of a collection of measurements, time series, and their tags. Exported functions are goroutine safe while un-exported functions assume the caller will use the appropriate locks

func NewDatabaseIndex 

func NewDatabaseIndex() *DatabaseIndex

func (*DatabaseIndex) CreateMeasurementIndexIfNotExists added in v0.9.3 

func (s *DatabaseIndex) CreateMeasurementIndexIfNotExists(name string) *Measurement

CreateMeasurementIndexIfNotExists creates or retrieves an in memory index object for the measurement

func (*DatabaseIndex) CreateSeriesIndexIfNotExists added in v0.9.3 

func (s *DatabaseIndex) CreateSeriesIndexIfNotExists(measurementName string, series *Series) *Series

CreateSeriesIndexIfNotExists adds the series for the given measurement to the index and sets its ID or returns the existing series object

func (*DatabaseIndex) DropMeasurement 

func (db *DatabaseIndex) DropMeasurement(name string)

DropMeasurement removes the measurement and all of its underlying series from the database index

func (*DatabaseIndex) DropSeries 

func (db *DatabaseIndex) DropSeries(keys []string)

DropSeries removes the series keys and their tags from the index

func (*DatabaseIndex) ExpandSources added in v0.10.0 

func (di *DatabaseIndex) ExpandSources(sources influxql.Sources) (influxql.Sources, error)

expandSources expands regex sources and removes duplicates. NOTE: sources must be normalized (db and rp set) before calling this function.

func (*DatabaseIndex) ExpandWildcards added in v0.10.0 

func (db *DatabaseIndex) ExpandWildcards(stmt *influxql.SelectStatement) (*influxql.SelectStatement, error)

expandWildcards returns a new SelectStatement with wildcards expanded If only a `SELECT *` is present, without a `GROUP BY *`, both tags and fields expand in the SELECT If a `SELECT *` and a `GROUP BY *` are both present, then only fiels are expanded in the `SELECT` and only tags are expanded in the `GROUP BY`

func (*DatabaseIndex) Measurement 

func (d *DatabaseIndex) Measurement(name string) *Measurement

Measurement returns the measurement object from the index by the name

func (*DatabaseIndex) MeasurementSeriesCounts 

func (d *DatabaseIndex) MeasurementSeriesCounts() (nMeasurements int, nSeries int)

MeasurementSeriesCounts returns the number of measurements and series currently indexed by the database. Useful for reporting and monitoring.

func (*DatabaseIndex) Measurements 

func (db *DatabaseIndex) Measurements() Measurements

Measurements returns a list of all measurements.

func (*DatabaseIndex) MeasurementsByName added in v0.10.0 

func (d *DatabaseIndex) MeasurementsByName(names []string) []*Measurement

MeasurementsByName returns a list of measurements.

func (*DatabaseIndex) RewriteSelectStatement added in v0.10.0 

func (db *DatabaseIndex) RewriteSelectStatement(stmt *influxql.SelectStatement) (*influxql.SelectStatement, error)

RewriteSelectStatement performs any necessary query re-writing.

func (*DatabaseIndex) Series added in v0.9.3 

func (d *DatabaseIndex) Series(key string) *Series

Series returns a series by key.

func (*DatabaseIndex) SeriesN added in v0.9.3 

func (d *DatabaseIndex) SeriesN() int

SeriesN returns the number of series.

func (*DatabaseIndex) TagsForSeries added in v0.9.3 

func (s *DatabaseIndex) TagsForSeries(key string) map[string]string

TagsForSeries returns the tag map for the passed in series

type Engine added in v0.9.3 

type Engine interface {
	Open() error
	Close() error

	SetLogOutput(io.Writer)
	LoadMetadataIndex(shard *Shard, index *DatabaseIndex, measurementFields map[string]*MeasurementFields) error

	Begin(writable bool) (Tx, error)
	WritePoints(points []models.Point, measurementFieldsToSave map[string]*MeasurementFields, seriesToCreate []*SeriesCreate) error
	DeleteSeries(keys []string) error
	DeleteMeasurement(name string, seriesKeys []string) error
	SeriesCount() (n int, err error)

	// PerformMaintenance will get called periodically by the store
	PerformMaintenance()

	// Format will return the format for the engine
	Format() EngineFormat

	io.WriterTo

	Backup(w io.Writer, basePath string, since time.Time) error
}

Engine represents a swappable storage engine for the shard.

func NewEngine added in v0.9.3 

func NewEngine(path string, walPath string, options EngineOptions) (Engine, error)

NewEngine returns an instance of an engine based on its format. If the path does not exist then the DefaultFormat is used.

type EngineFormat added in v0.10.0 

type EngineFormat int
const (
	B1Format EngineFormat = iota
	BZ1Format
	TSM1Format
)

type EngineOptions added in v0.9.3 

type EngineOptions struct {
	EngineVersion          string
	MaxWALSize             int
	WALFlushInterval       time.Duration
	WALPartitionFlushDelay time.Duration

	Config Config
}

EngineOptions represents the options used to initialize the engine.

func NewEngineOptions added in v0.9.3 

func NewEngineOptions() EngineOptions

NewEngineOptions returns the default options.

type ErrAuthorize 

type ErrAuthorize struct {
	// contains filtered or unexported fields
}

ErrAuthorize represents an authorization error.

func NewErrAuthorize 

func NewErrAuthorize(qe *QueryExecutor, q *influxql.Query, u, db, m string) *ErrAuthorize

newAuthorizationError returns a new instance of AuthorizationError.

func (ErrAuthorize) Error 

func (e ErrAuthorize) Error() string

Error returns the text of the error.

type Executor 

type Executor interface {
	Execute(closing <-chan struct{}) <-chan *models.Row
}

Executor is an interface for a query executor.

type Field added in v0.9.3 

type Field struct {
	ID   uint8             `json:"id,omitempty"`
	Name string            `json:"name,omitempty"`
	Type influxql.DataType `json:"type,omitempty"`
}

Field represents a series field.

type FieldCodec 

type FieldCodec struct {
	// contains filtered or unexported fields
}

FieldCodec provides encoding and decoding functionality for the fields of a given Measurement. It is a distinct type to avoid locking writes on this node while potentially long-running queries are executing.

It is not affected by changes to the Measurement object after codec creation. TODO: this shouldn't be exported. nothing outside the shard should know about field encodings. 

However, this is here until tx.go and the engine get refactored into tsdb.

func NewFieldCodec added in v0.9.3 

func NewFieldCodec(fields map[string]*Field) *FieldCodec

NewFieldCodec returns a FieldCodec for the given Measurement. Must be called with a RLock that protects the Measurement.

func (*FieldCodec) DecodeByID 

func (f *FieldCodec) DecodeByID(targetID uint8, b []byte) (interface{}, error)

DecodeByID scans a byte slice for a field with the given ID, converts it to its expected type, and return that value. TODO: shouldn't be exported. refactor engine

func (*FieldCodec) DecodeByName 

func (f *FieldCodec) DecodeByName(name string, b []byte) (interface{}, error)

DecodeByName scans a byte slice for a field with the given name, converts it to its expected type, and return that value.

func (*FieldCodec) DecodeFields 

func (f *FieldCodec) DecodeFields(b []byte) (map[uint8]interface{}, error)

DecodeFields decodes a byte slice into a set of field ids and values.

func (*FieldCodec) DecodeFieldsWithNames 

func (f *FieldCodec) DecodeFieldsWithNames(b []byte) (map[string]interface{}, error)

DecodeFieldsWithNames decodes a byte slice into a set of field names and values TODO: shouldn't be exported. refactor engine

func (*FieldCodec) EncodeFields 

func (f *FieldCodec) EncodeFields(values map[string]interface{}) ([]byte, error)

EncodeFields converts a map of values with string keys to a byte slice of field IDs and values.

If a field exists in the codec, but its type is different, an error is returned. If a field is not present in the codec, the system panics.

func (*FieldCodec) FieldByName added in v0.10.0 

func (f *FieldCodec) FieldByName(name string) *Field

FieldByName returns the field by its name. It will return a nil if not found

func (*FieldCodec) FieldIDByName 

func (f *FieldCodec) FieldIDByName(s string) (uint8, error)

TODO: this shouldn't be exported. remove when tx.go and engine.go get refactored into tsdb

func (*FieldCodec) Fields added in v0.10.0 

func (f *FieldCodec) Fields() (a []*Field)

type FieldCreate added in v0.9.3 

type FieldCreate struct {
	Measurement string
	Field       *Field
}

struct to hold information for a field to create on a measurement

type FilterExprs added in v0.10.0 

type FilterExprs map[uint64]influxql.Expr

FilterExprs represents a map of series IDs to filter expressions.

func (FilterExprs) DeleteBoolLiteralTrues added in v0.10.0 

func (fe FilterExprs) DeleteBoolLiteralTrues()

DeleteBoolLiteralTrues deletes all elements whose filter expression is a boolean literal true.

func (FilterExprs) Len added in v0.10.0 

func (fe FilterExprs) Len() int

Len returns the number of elements.

type InterfaceValues added in v0.10.0 

type InterfaceValues []interface{}

func (InterfaceValues) Len added in v0.10.0 

func (d InterfaceValues) Len() int

func (InterfaceValues) Less added in v0.10.0 

func (d InterfaceValues) Less(i, j int) bool

func (InterfaceValues) Swap added in v0.10.0 

func (d InterfaceValues) Swap(i, j int)

type IntoWriteRequest added in v0.10.0 

type IntoWriteRequest struct {
	Database        string
	RetentionPolicy string
	Points          []models.Point
}

partial copy of cluster.WriteRequest

type MapInput added in v0.10.0 

type MapInput struct {
	TMin  int64
	Items []MapItem
}

MapInput represents a collection of values to be processed by the mapper.

type MapItem added in v0.10.0 

type MapItem struct {
	Timestamp int64
	Value     interface{}

	// TODO(benbjohnson):
	//   Move fields and tags up to MapInput. Currently the engine combines
	//   multiple series together during processing. This needs to be fixed so
	//   that each map function only operates on a single series at a time instead.
	Fields map[string]interface{}
	Tags   map[string]string
}

MapItem represents a single item in a collection that's processed by the mapper.

type MapItems added in v0.10.0 

type MapItems []MapItem

func (MapItems) Len added in v0.10.0 

func (a MapItems) Len() int

func (MapItems) Less added in v0.10.0 

func (a MapItems) Less(i, j int) bool

func (MapItems) Swap added in v0.10.0 

func (a MapItems) Swap(i, j int)

type Mapper 

type Mapper interface {
	Open() error
	TagSets() []string
	Fields() []string
	NextChunk() (interface{}, error)
	Close()
}

Mapper is the interface all Mapper types must implement.

type MapperOutput added in v0.9.3 

type MapperOutput struct {
	Name      string            `json:"name,omitempty"`
	Tags      map[string]string `json:"tags,omitempty"`
	Fields    []string          `json:"fields,omitempty"`    // Field names of returned data.
	Values    []*MapperValue    `json:"values,omitempty"`    // For aggregates contains a single value at [0]
	CursorKey string            `json:"cursorkey,omitempty"` // Tagset-based key for the source cursor. Cached for performance reasons.
}

func (*MapperOutput) MarshalJSON added in v0.10.0 

func (mo *MapperOutput) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON-encoded representation of a MapperOutput.

type MapperOutputJSON added in v0.10.0 

type MapperOutputJSON struct {
	Name      string            `json:"name,omitempty"`
	Tags      map[string]string `json:"tags,omitempty"`
	Fields    []string          `json:"fields,omitempty"`    // Field names of returned data.
	CursorKey string            `json:"cursorkey,omitempty"` // Tagset-based key for the source cursor.
	Values    json.RawMessage   `json:"values,omitempty"`
}

MapperOutputJSON is the JSON-encoded representation of MapperOutput. The query data is represented as a raw JSON message, so decode is delayed, and can proceed in a custom manner.

type MapperValue added in v0.9.3 

type MapperValue struct {
	Time  int64             `json:"time,omitempty"`  // Ignored for aggregate output.
	Value interface{}       `json:"value,omitempty"` // For aggregate, contains interval time multiple values.
	Tags  map[string]string `json:"tags,omitempty"`  // Meta tags for results
}

MapperValue is a complex type, which can encapsulate data from both raw and aggregate mappers. This currently allows marshalling and network system to remain simpler. For aggregate output Time is ignored, and actual Time-Value pairs are contained soley within the Value field.

func (*MapperValue) MarshalJSON added in v0.10.0 

func (mv *MapperValue) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON-encoded representation of a MapperValue.

type MapperValueJSON added in v0.10.0 

type MapperValueJSON struct {
	Time    int64             `json:"time,omitempty"`
	RawData []byte            `json:"rdata,omitempty"`
	AggData [][]byte          `json:"adata,omitempty"`
	Tags    map[string]string `json:"tags,omitempty"`
}

MapperValueJSON is the JSON-encoded representation of MapperValue. Because MapperValue is a complex type, custom JSON encoding is required so that none of the types contained within a MapperValue are "lost", and so the data are encoded as byte slices where necessary.

type MapperValues added in v0.9.3 

type MapperValues []*MapperValue

func (MapperValues) Len added in v0.9.3 

func (a MapperValues) Len() int

func (MapperValues) Less added in v0.9.3 

func (a MapperValues) Less(i, j int) bool

func (MapperValues) Swap added in v0.9.3 

func (a MapperValues) Swap(i, j int)

type Measurement 

type Measurement struct {
	Name string `json:"name,omitempty"`
	// contains filtered or unexported fields
}

Measurement represents a collection of time series in a database. It also contains in memory structures for indexing tags. Exported functions are goroutine safe while un-exported functions assume the caller will use the appropriate locks

func NewMeasurement 

func NewMeasurement(name string, idx *DatabaseIndex) *Measurement

NewMeasurement allocates and initializes a new Measurement.

func (*Measurement) AddSeries 

func (m *Measurement) AddSeries(s *Series) bool

AddSeries will add a series to the measurementIndex. Returns false if already present

func (*Measurement) DimensionTagSets added in v0.10.0 

func (m *Measurement) DimensionTagSets(stmt *influxql.SelectStatement) ([]*influxql.TagSet, error)

DimensionTagSets returns list of tag sets from the GROUP BY section of stmt.

func (*Measurement) DropSeries 

func (m *Measurement) DropSeries(seriesID uint64)

DropSeries will remove a series from the measurementIndex.

func (*Measurement) FieldNames 

func (m *Measurement) FieldNames() (a []string)

FieldNames returns a list of the measurement's field names

func (*Measurement) HasField 

func (m *Measurement) HasField(name string) bool

HasField returns true if the measurement has a field by the given name

func (*Measurement) HasSeries 

func (m *Measurement) HasSeries() bool

HasSeries returns true if there is at least 1 series under this measurement

func (*Measurement) HasTagKey 

func (m *Measurement) HasTagKey(k string) bool

HasTagKey returns true if at least one series in this measurement has written a value for the passed in tag key

func (*Measurement) SelectFields added in v0.10.0 

func (m *Measurement) SelectFields(stmt *influxql.SelectStatement) []string

SelectFields returns a list of fields in the SELECT section of stmt.

func (*Measurement) SelectTags added in v0.10.0 

func (m *Measurement) SelectTags(stmt *influxql.SelectStatement) []string

SelectTags returns a list of non-field tags in the SELECT section of stmt.

func (*Measurement) SeriesByID added in v0.9.3 

func (m *Measurement) SeriesByID(id uint64) *Series

SeriesByID returns a series by identifier.

func (*Measurement) SeriesKeys 

func (m *Measurement) SeriesKeys() []string

SeriesKeys returns the keys of every series in this measurement

func (*Measurement) SetFieldName added in v0.9.3 

func (m *Measurement) SetFieldName(name string)

SetFieldName adds the field name to the measurement.

func (*Measurement) TagKeys 

func (m *Measurement) TagKeys() []string

TagKeys returns a list of the measurement's tag names.

func (*Measurement) TagSets 

func (m *Measurement) TagSets(stmt *influxql.SelectStatement, dimensions []string) ([]*influxql.TagSet, error)

tagSets returns the unique tag sets that exist for the given tag keys. This is used to determine what composite series will be created by a group by. i.e. "group by region" should return: {"region":"uswest"}, {"region":"useast"} or region, service returns {"region": "uswest", "service": "redis"}, {"region": "uswest", "service": "mysql"}, etc... This will also populate the TagSet objects with the series IDs that match each tagset and any influx filter expression that goes with the series TODO: this shouldn't be exported. However, until tx.go and the engine get refactored into tsdb, we need it.

func (*Measurement) TagValues added in v0.10.0 

func (m *Measurement) TagValues(key string) []string

TagValues returns all the values for the given tag key

func (*Measurement) ValidateGroupBy 

func (m *Measurement) ValidateGroupBy(stmt *influxql.SelectStatement) error

ValidateGroupBy ensures that the GROUP BY is not a field.

func (*Measurement) WhereFields added in v0.10.0 

func (m *Measurement) WhereFields(stmt *influxql.SelectStatement) []string

WhereFields returns a list of non-"time" fields in the WHERE section of stmt.

type MeasurementFields added in v0.9.3 

type MeasurementFields struct {
	Fields map[string]*Field `json:"fields"`
	Codec  *FieldCodec
}

func (*MeasurementFields) CreateFieldIfNotExists added in v0.9.3 

func (m *MeasurementFields) CreateFieldIfNotExists(name string, typ influxql.DataType, limitCount bool) error

CreateFieldIfNotExists creates a new field with an autoincrementing ID. Returns an error if 255 fields have already been created on the measurement or the fields already exists with a different type.

func (*MeasurementFields) MarshalBinary added in v0.9.3 

func (m *MeasurementFields) MarshalBinary() ([]byte, error)

MarshalBinary encodes the object to a binary format.

func (*MeasurementFields) UnmarshalBinary added in v0.9.3 

func (m *MeasurementFields) UnmarshalBinary(buf []byte) error

UnmarshalBinary decodes the object from a binary format.

type MeasurementTagKeys added in v0.10.0 

type MeasurementTagKeys struct {
	Measurement string   `json:"measurement"`
	TagKeys     []string `json:"tagkeys"`
}

MeasurementTagKeys represents measurement tag keys.

type Measurements 

type Measurements []*Measurement

Measurements represents a list of *Measurement.

func (Measurements) Len 

func (a Measurements) Len() int

func (Measurements) Less 

func (a Measurements) Less(i, j int) bool

func (Measurements) SelectFields added in v0.10.0 

func (a Measurements) SelectFields(stmt *influxql.SelectStatement) []string

SelectFields returns a list of fields in the SELECT section of stmt.

func (Measurements) SelectTags added in v0.10.0 

func (a Measurements) SelectTags(stmt *influxql.SelectStatement) []string

SelectTags returns a list of non-field tags in the SELECT section of stmt.

func (Measurements) Swap 

func (a Measurements) Swap(i, j int)

func (Measurements) WhereFields added in v0.10.0 

func (a Measurements) WhereFields(stmt *influxql.SelectStatement) []string

WhereFields returns a list of non-"time" fields in the WHERE section of stmt.

type MeasurementsTagKeys added in v0.10.0 

type MeasurementsTagKeys []*MeasurementTagKeys

MeasurementsTagKeys represents tag keys for multiple measurements.

func (MeasurementsTagKeys) Len added in v0.10.0 

func (a MeasurementsTagKeys) Len() int

func (MeasurementsTagKeys) Less added in v0.10.0 

func (a MeasurementsTagKeys) Less(i, j int) bool

func (MeasurementsTagKeys) Size added in v0.10.0 

func (a MeasurementsTagKeys) Size() int

Size returns the total string length of measurement names & tag keys.

func (MeasurementsTagKeys) Swap added in v0.10.0 

func (a MeasurementsTagKeys) Swap(i, j int)

type NewEngineFunc added in v0.9.3 

type NewEngineFunc func(path string, walPath string, options EngineOptions) Engine

NewEngineFunc creates a new engine.

type NumberType added in v0.10.0 

type NumberType int8
const (
	Float64Type NumberType = iota
	Int64Type
)

type PointBatcher 

type PointBatcher struct {
	// contains filtered or unexported fields
}

PointBatcher accepts Points and will emit a batch of those points when either a) the batch reaches a certain size, or b) a certain time passes.

func NewPointBatcher 

func NewPointBatcher(sz int, bp int, d time.Duration) *PointBatcher

NewPointBatcher returns a new PointBatcher. sz is the batching size, bp is the maximum number of batches that may be pending. d is the time after which a batch will be emitted after the first point is received for the batch, regardless of its size.

func (*PointBatcher) Flush 

func (b *PointBatcher) Flush()

Flush instructs the batcher to emit any pending points in a batch, regardless of batch size. If there are no pending points, no batch is emitted.

func (*PointBatcher) In 

func (b *PointBatcher) In() chan<- models.Point

In returns the channel to which points should be written.

func (*PointBatcher) Out 

func (b *PointBatcher) Out() <-chan []models.Point

Out returns the channel from which batches should be read.

func (*PointBatcher) Start 

func (b *PointBatcher) Start()

Start starts the batching process. Returns the in and out channels for points and point-batches respectively.

func (*PointBatcher) Stats 

func (b *PointBatcher) Stats() *PointBatcherStats

Stats returns a PointBatcherStats object for the PointBatcher. While the each statistic should be closely correlated with each other statistic, it is not guaranteed.

func (*PointBatcher) Stop 

func (b *PointBatcher) Stop()

type PointBatcherStats 

type PointBatcherStats struct {
	BatchTotal   uint64 // Total count of batches transmitted.
	PointTotal   uint64 // Total count of points processed.
	SizeTotal    uint64 // Number of batches that reached size threshold.
	TimeoutTotal uint64 // Number of timeouts that occurred.
}

PointBatcherStats are the statistics each batcher tracks.

type PositionPoint added in v0.10.0 

type PositionPoint struct {
	Time   int64
	Value  interface{}
	Fields map[string]interface{}
	Tags   map[string]string
}

PositionPoint will return all data points from a written point that were selected in the query to be used in the post processing phase of the query executor to fill in additional tag and field values

type PositionPoints added in v0.10.0 

type PositionPoints []PositionPoint

PositionPoints is a slice of PositionPoints used to return richer data from a reduce func

type QueryExecutor 

type QueryExecutor struct {
	// The meta store for accessing and updating cluster and schema data.
	MetaClient interface {
		Database(name string) (*meta.DatabaseInfo, error)
		Databases() ([]meta.DatabaseInfo, error)
		User(name string) (*meta.UserInfo, error)
		AdminUserExists() bool
		Authenticate(username, password string) (*meta.UserInfo, error)
		RetentionPolicy(database, name string) (rpi *meta.RetentionPolicyInfo, err error)
		UserCount() int
		ShardGroupsByTimeRange(database, policy string, min, max time.Time) (a []meta.ShardGroupInfo, err error)
		ExecuteStatement(stmt influxql.Statement) *influxql.Result
	}

	// Execute statements relating to statistics and diagnostics.
	MonitorStatementExecutor interface {
		ExecuteStatement(stmt influxql.Statement) *influxql.Result
	}

	// Maps shards for queries.
	ShardMapper interface {
		CreateMapper(shard meta.ShardInfo, stmt influxql.Statement, chunkSize int) (Mapper, error)
	}

	IntoWriter interface {
		WritePointsInto(p *IntoWriteRequest) error
	}

	Logger          *log.Logger
	QueryLogEnabled bool

	// the local data store
	Store *Store
}

QueryExecutor executes every statement in an influxdb Query. It is responsible for coordinating between the local tsdb.Store, the meta.Store, and the other nodes in the cluster to run the query against their local tsdb.Stores. There should be one executor in a running process

func NewQueryExecutor 

func NewQueryExecutor(store *Store) *QueryExecutor

NewQueryExecutor returns an initialized QueryExecutor

func (*QueryExecutor) Authorize 

func (q *QueryExecutor) Authorize(u *meta.UserInfo, query *influxql.Query, database string) error

Authorize user u to execute query q on database. database can be "" for queries that do not require a database. If no user is provided it will return an error unless the query's first statement is to create a root user.

func (*QueryExecutor) ExecuteQuery 

func (q *QueryExecutor) ExecuteQuery(query *influxql.Query, database string, chunkSize int, closing chan struct{}) (<-chan *influxql.Result, error)

ExecuteQuery executes an InfluxQL query against the server. It sends results down the passed in chan and closes it when done. It will close the chan on the first statement that throws an error.

func (*QueryExecutor) PlanSelect added in v0.10.0 

func (q *QueryExecutor) PlanSelect(stmt *influxql.SelectStatement, chunkSize int) (Executor, error)

Plan creates an execution plan for the given SelectStatement and returns an Executor.

func (*QueryExecutor) PlanShowMeasurements added in v0.10.0 

func (q *QueryExecutor) PlanShowMeasurements(stmt *influxql.ShowMeasurementsStatement, database string, chunkSize int) (Executor, error)

PlanShowMeasurements creates an execution plan for a SHOW TAG KEYS statement and returns an Executor.

func (*QueryExecutor) PlanShowTagKeys added in v0.10.0 

func (q *QueryExecutor) PlanShowTagKeys(stmt *influxql.ShowTagKeysStatement, database string, chunkSize int) (Executor, error)

PlanShowTagKeys creates an execution plan for a SHOW MEASUREMENTS statement and returns an Executor.

func (*QueryExecutor) SetLogger added in v0.10.0 

func (q *QueryExecutor) SetLogger(l *log.Logger)

SetLogger sets the internal logger to the logger passed in.

type RawExecutor 

type RawExecutor struct {
	// contains filtered or unexported fields
}

func NewRawExecutor 

func NewRawExecutor(stmt *influxql.SelectStatement, mappers []Mapper, chunkSize int) *RawExecutor

NewRawExecutor returns a new RawExecutor.

func (*RawExecutor) Execute 

func (e *RawExecutor) Execute(closing <-chan struct{}) <-chan *models.Row

Execute begins execution of the query and returns a channel to receive rows.

type RawMapper 

type RawMapper struct {
	ChunkSize int
	// contains filtered or unexported fields
}

RawMapper runs the map phase for non-aggregate, raw SELECT queries.

func NewRawMapper 

func NewRawMapper(sh *Shard, stmt *influxql.SelectStatement) *RawMapper

NewRawMapper returns a new instance of RawMapper.

func (*RawMapper) Close 

func (m *RawMapper) Close()

Close closes the mapper.

func (*RawMapper) Fields added in v0.10.0 

func (m *RawMapper) Fields() []string

Fields returns all SELECT fields.

func (*RawMapper) NextChunk 

func (m *RawMapper) NextChunk() (interface{}, error)

NextChunk returns the next chunk of data. Data is ordered the same as TagSets. Each chunk contains one tag set. If there is no more data for any tagset, nil will be returned.

func (*RawMapper) Open 

func (m *RawMapper) Open() error

Open opens and initializes the mapper.

func (*RawMapper) TagSets 

func (m *RawMapper) TagSets() []string

TagSets returns the list of tag sets for which this mapper has data.

type RawQueryDerivativeProcessor added in v0.9.3 

type RawQueryDerivativeProcessor struct {
	LastValueFromPreviousChunk *MapperValue
	IsNonNegative              bool // Whether to drop negative differences
	DerivativeInterval         time.Duration
}

func (*RawQueryDerivativeProcessor) Process added in v0.9.3 

func (rqdp *RawQueryDerivativeProcessor) Process(input []*MapperValue) []*MapperValue

type SelectInfo added in v0.10.0 

type SelectInfo struct {
	SelectFields []string
	SelectTags   []string
	WhereFields  []string
}

type Series 

type Series struct {
	Key  string
	Tags map[string]string
	// contains filtered or unexported fields
}

Series belong to a Measurement and represent unique time series in a database

func NewSeries added in v0.9.3 

func NewSeries(key string, tags map[string]string) *Series

NewSeries returns an initialized series struct

func (*Series) InitializeShards added in v0.9.3 

func (s *Series) InitializeShards()

func (*Series) MarshalBinary 

func (s *Series) MarshalBinary() ([]byte, error)

MarshalBinary encodes the object to a binary format.

func (*Series) UnmarshalBinary 

func (s *Series) UnmarshalBinary(buf []byte) error

UnmarshalBinary decodes the object from a binary format.

type SeriesCreate added in v0.9.3 

type SeriesCreate struct {
	Measurement string
	Series      *Series
}

struct to hold information for a series to create

type SeriesIDs added in v0.9.3 

type SeriesIDs []uint64

SeriesIDs is a convenience type for sorting, checking equality, and doing union and intersection of collections of series ids.

func (SeriesIDs) Equals added in v0.9.3 

func (a SeriesIDs) Equals(other SeriesIDs) bool

Equals assumes that both are sorted.

func (SeriesIDs) Intersect added in v0.9.3 

func (a SeriesIDs) Intersect(other SeriesIDs) SeriesIDs

Intersect returns a new collection of series ids in sorted order that is the intersection of the two. The two collections must already be sorted.

func (SeriesIDs) Len added in v0.9.3 

func (a SeriesIDs) Len() int

func (SeriesIDs) Less added in v0.9.3 

func (a SeriesIDs) Less(i, j int) bool

func (SeriesIDs) Reject added in v0.9.3 

func (a SeriesIDs) Reject(other SeriesIDs) SeriesIDs

Reject returns a new collection of series ids in sorted order with the passed in set removed from the original. This is useful for the NOT operator. The two collections must already be sorted.

func (SeriesIDs) Swap added in v0.9.3 

func (a SeriesIDs) Swap(i, j int)

func (SeriesIDs) Union added in v0.9.3 

func (a SeriesIDs) Union(other SeriesIDs) SeriesIDs

Union returns a new collection of series ids in sorted order that is the union of the two. The two collections must already be sorted.

type Shard 

type Shard struct {

	// The writer used by the logger.
	LogOutput io.Writer
	// contains filtered or unexported fields
}

Shard represents a self-contained time series database. An inverted index of the measurement and tag data is kept along with the raw time series data. Data can be split across many shards. The query engine in TSDB is responsible for combining the output of many shards into a single query result.

func NewShard 

func NewShard(id uint64, index *DatabaseIndex, path string, walPath string, options EngineOptions) *Shard

NewShard returns a new initialized Shard. walPath doesn't apply to the b1 type index

func (*Shard) Close 

func (s *Shard) Close() error

Close shuts down the shard's store.

func (*Shard) DeleteMeasurement added in v0.9.3 

func (s *Shard) DeleteMeasurement(name string, seriesKeys []string) error

DeleteMeasurement deletes a measurement and all underlying series.

func (*Shard) DeleteSeries added in v0.9.3 

func (s *Shard) DeleteSeries(keys []string) error

DeleteSeries deletes a list of series.

func (*Shard) DiskSize added in v0.10.0 

func (s *Shard) DiskSize() (int64, error)

DiskSize returns the size on disk of this shard

func (*Shard) FieldCodec 

func (s *Shard) FieldCodec(measurementName string) *FieldCodec

TODO: this is temporarily exported to make tx.go work. When the query engine gets refactored into the tsdb package this should be removed. No one outside tsdb should know the underlying field encoding scheme.

func (*Shard) Open 

func (s *Shard) Open() error

open initializes and opens the shard's store.

func (*Shard) Path 

func (s *Shard) Path() string

Path returns the path set on the shard when it was created.

func (*Shard) PerformMaintenance added in v0.10.0 

func (s *Shard) PerformMaintenance()

PerformMaintenance gets called periodically to have the engine perform any maintenance tasks like WAL flushing and compaction

func (*Shard) ReadOnlyTx added in v0.10.0 

func (s *Shard) ReadOnlyTx() (Tx, error)

ReadOnlyTx returns a read-only transaction for the shard. The transaction must be rolled back to release resources.

func (*Shard) SeriesCount 

func (s *Shard) SeriesCount() (int, error)

SeriesCount returns the number of series buckets on the shard.

func (*Shard) ValidateAggregateFieldsInStatement 

func (s *Shard) ValidateAggregateFieldsInStatement(measurementName string, stmt *influxql.SelectStatement) error

func (*Shard) WritePoints 

func (s *Shard) WritePoints(points []models.Point) error

WritePoints will write the raw data points and any new metadata to the index in the shard

func (*Shard) WriteTo added in v0.10.0 

func (s *Shard) WriteTo(w io.Writer) (int64, error)

WriteTo writes the shard's data to w.

type ShowMeasurementsExecutor added in v0.10.0 

type ShowMeasurementsExecutor struct {
	// contains filtered or unexported fields
}

ShowMeasurementsExecutor implements the Executor interface for a SHOW MEASUREMENTS statement.

func NewShowMeasurementsExecutor added in v0.10.0 

func NewShowMeasurementsExecutor(stmt *influxql.ShowMeasurementsStatement, mappers []Mapper, chunkSize int) *ShowMeasurementsExecutor

NewShowMeasurementsExecutor returns a new ShowMeasurementsExecutor.

func (*ShowMeasurementsExecutor) Execute added in v0.10.0 

func (e *ShowMeasurementsExecutor) Execute(closing <-chan struct{}) <-chan *models.Row

Execute begins execution of the query and returns a channel to receive rows.

type ShowMeasurementsMapper added in v0.10.0 

type ShowMeasurementsMapper struct {
	ChunkSize int
	// contains filtered or unexported fields
}

ShowMeasurementsMapper is a mapper for collecting measurement names from a shard.

func NewShowMeasurementsMapper added in v0.10.0 

func NewShowMeasurementsMapper(shard *Shard, stmt *influxql.ShowMeasurementsStatement) *ShowMeasurementsMapper

NewShowMeasurementsMapper returns a mapper for the given shard, which will return data for the meta statement.

func (*ShowMeasurementsMapper) Close added in v0.10.0 

func (m *ShowMeasurementsMapper) Close()

Close closes the mapper.

func (*ShowMeasurementsMapper) Fields added in v0.10.0 

func (m *ShowMeasurementsMapper) Fields() []string

Fields returns a list of field names for this mapper.

func (*ShowMeasurementsMapper) NextChunk added in v0.10.0 

func (m *ShowMeasurementsMapper) NextChunk() (interface{}, error)

NextChunk returns the next chunk of measurement names.

func (*ShowMeasurementsMapper) Open added in v0.10.0 

func (m *ShowMeasurementsMapper) Open() error

Open opens the mapper for use.

func (*ShowMeasurementsMapper) SetRemote added in v0.10.0 

func (m *ShowMeasurementsMapper) SetRemote(remote Mapper)

SetRemote sets the remote mapper to use.

func (*ShowMeasurementsMapper) TagSets added in v0.10.0 

func (m *ShowMeasurementsMapper) TagSets() []string

TagSets is only implemented on this mapper to satisfy the Mapper interface.

type ShowTagKeysExecutor added in v0.10.0 

type ShowTagKeysExecutor struct {
	// contains filtered or unexported fields
}

ShowTagKeysExecutor implements the Executor interface for a SHOW MEASUREMENTS statement.

func NewShowTagKeysExecutor added in v0.10.0 

func NewShowTagKeysExecutor(stmt *influxql.ShowTagKeysStatement, mappers []Mapper, chunkSize int) *ShowTagKeysExecutor

NewShowTagKeysExecutor returns a new ShowTagKeysExecutor.

func (*ShowTagKeysExecutor) Execute added in v0.10.0 

func (e *ShowTagKeysExecutor) Execute(closing <-chan struct{}) <-chan *models.Row

Execute begins execution of the query and returns a channel to receive rows.

type ShowTagKeysMapper added in v0.10.0 

type ShowTagKeysMapper struct {
	// contains filtered or unexported fields
}

ShowTagKeysMapper is a mapper for collecting measurement names from a shard.

func NewShowTagKeysMapper added in v0.10.0 

func NewShowTagKeysMapper(shard *Shard, stmt *influxql.ShowTagKeysStatement, chunkSize int) *ShowTagKeysMapper

NewShowTagKeysMapper returns a mapper for the given shard, which will return data for the meta statement.

func (*ShowTagKeysMapper) Close added in v0.10.0 

func (m *ShowTagKeysMapper) Close()

Close closes the mapper.

func (*ShowTagKeysMapper) Fields added in v0.10.0 

func (m *ShowTagKeysMapper) Fields() []string

Fields returns a list of field names for this mapper.

func (*ShowTagKeysMapper) NextChunk added in v0.10.0 

func (m *ShowTagKeysMapper) NextChunk() (interface{}, error)

NextChunk returns the next chunk of measurements and tag keys.

func (*ShowTagKeysMapper) Open added in v0.10.0 

func (m *ShowTagKeysMapper) Open() error

Open opens the mapper for use.

func (*ShowTagKeysMapper) SetRemote added in v0.10.0 

func (m *ShowTagKeysMapper) SetRemote(remote Mapper) error

SetRemote sets the remote mapper to use.

func (*ShowTagKeysMapper) TagSets added in v0.10.0 

func (m *ShowTagKeysMapper) TagSets() []string

TagSets is only implemented on this mapper to satisfy the Mapper interface.

type StatefulMapper 

type StatefulMapper struct {
	Mapper
	// contains filtered or unexported fields
}

StatefulMapper encapsulates a Mapper and some state that the executor needs to track for that mapper.

func (*StatefulMapper) NextChunk 

func (sm *StatefulMapper) NextChunk() (*MapperOutput, error)

NextChunk wraps a RawMapper and some state.

type Store 

type Store struct {
	EngineOptions EngineOptions
	Logger        *log.Logger
	// contains filtered or unexported fields
}

func NewStore 

func NewStore(path string) *Store

func (*Store) BackupShard added in v0.10.0 

func (s *Store) BackupShard(id uint64, since time.Time, w io.Writer) error

BackupShard will get the shard and have the engine backup since the passed in time to the writer

func (*Store) Close 

func (s *Store) Close() error

func (*Store) CreateMapper 

func (s *Store) CreateMapper(shardID uint64, stmt influxql.Statement, chunkSize int) (Mapper, error)

func (*Store) CreateShard 

func (s *Store) CreateShard(database, retentionPolicy string, shardID uint64) error

func (*Store) DatabaseIndex 

func (s *Store) DatabaseIndex(name string) *DatabaseIndex

func (*Store) DatabaseIndexN added in v0.9.3 

func (s *Store) DatabaseIndexN() int

DatabaseIndexN returns the number of databases indicies in the store.

func (*Store) Databases added in v0.10.0 

func (s *Store) Databases() []string

Databases returns all the databases in the indexes

func (*Store) DeleteDatabase 

func (s *Store) DeleteDatabase(name string, shardIDs []uint64) error

DeleteDatabase will close all shards associated with a database and remove the directory and files from disk.

func (*Store) DeleteShard 

func (s *Store) DeleteShard(shardID uint64) error

DeleteShard removes a shard from disk.

func (*Store) DiskSize added in v0.10.0 

func (s *Store) DiskSize() (int64, error)

DiskSize returns the size of all the shard files in bytes. This size does not include the WAL size.

func (*Store) Measurement 

func (s *Store) Measurement(database, name string) *Measurement

func (*Store) Open 

func (s *Store) Open() error

func (*Store) Path 

func (s *Store) Path() string

Path returns the store's root path.

func (*Store) Shard 

func (s *Store) Shard(id uint64) *Shard

Shard returns a shard by id.

func (*Store) ShardIDs 

func (s *Store) ShardIDs() []uint64

ShardIDs returns a slice of all ShardIDs under management.

func (*Store) ShardN added in v0.9.3 

func (s *Store) ShardN() int

ShardN returns the number of shard in the store.

func (*Store) ShardRelativePath added in v0.10.0 

func (s *Store) ShardRelativePath(id uint64) (string, error)

ShardRelativePath will return the relative path to the shard. i.e. <database>/<retention>/<id>

func (*Store) ValidateAggregateFieldsInStatement 

func (s *Store) ValidateAggregateFieldsInStatement(shardID uint64, measurementName string, stmt *influxql.SelectStatement) error

func (*Store) WriteToShard 

func (s *Store) WriteToShard(shardID uint64, points []models.Point) error

type TagFilter 

type TagFilter struct {
	Op    influxql.Token
	Key   string
	Value string
	Regex *regexp.Regexp
}

TagFilter represents a tag filter when looking up other tags or measurements.

type TagSetCursor added in v0.10.0 

type TagSetCursor struct {
	SelectFields []string // fields to be selected
	// contains filtered or unexported fields
}

TagSetCursor is virtual cursor that iterates over multiple TagsCursors.

func NewTagSetCursor added in v0.10.0 

func NewTagSetCursor(m string, t map[string]string, c []*TagsCursor, ascending bool) *TagSetCursor

NewTagSetCursor returns a instance of TagSetCursor.

func (*TagSetCursor) Fields added in v0.10.0 

func (tsc *TagSetCursor) Fields() map[string]interface{}

Fields returns the current fields of the current cursor

func (*TagSetCursor) Init added in v0.10.0 

func (tsc *TagSetCursor) Init(seek int64)

func (*TagSetCursor) Next added in v0.10.0 

func (tsc *TagSetCursor) Next(tmin, tmax int64) (int64, interface{})

Next returns the next matching series-key, timestamp byte slice and meta tags for the tagset. Filtering is enforced on the values. If there is no matching value, then a nil result is returned.

func (*TagSetCursor) Tags added in v0.10.0 

func (tsc *TagSetCursor) Tags() map[string]string

Tags returns the current tags of the current cursor if there is no current currsor, it returns nil

type TagSetCursors added in v0.10.0 

type TagSetCursors []*TagSetCursor

TagSetCursors represents a sortable slice of TagSetCursors.

func (TagSetCursors) Keys added in v0.10.0 

func (a TagSetCursors) Keys() []string

func (TagSetCursors) Len added in v0.10.0 

func (a TagSetCursors) Len() int

func (TagSetCursors) Less added in v0.10.0 

func (a TagSetCursors) Less(i, j int) bool

func (TagSetCursors) Swap added in v0.10.0 

func (a TagSetCursors) Swap(i, j int)

type TagsCursor added in v0.10.0 

type TagsCursor struct {
	// contains filtered or unexported fields
}

TagsCursor is a cursor with attached tags and filter.

func NewTagsCursor added in v0.10.0 

func NewTagsCursor(c Cursor, filter influxql.Expr, tags map[string]string) *TagsCursor

NewTagsCursor returns a new instance of a series cursor.

func (*TagsCursor) Next added in v0.10.0 

func (c *TagsCursor) Next() (int64, interface{})

Next returns the next timestamp and value from the cursor.

func (*TagsCursor) SeekTo added in v0.10.0 

func (c *TagsCursor) SeekTo(seek int64) (int64, interface{})

Seek positions returning the key and value at that key.

type Tx added in v0.9.3 

type Tx interface {
	io.WriterTo

	Size() int64
	Commit() error
	Rollback() error

	Cursor(series string, fields []string, dec *FieldCodec, ascending bool) Cursor
}

Tx represents a transaction.

type UnmarshalFunc added in v0.10.0 

type UnmarshalFunc func([]byte) (interface{}, error)

UnmarshalFunc represents a function that can take bytes from a mapper from remote server and marshal it into an interface the reducer can use

func InitializeUnmarshaller added in v0.10.0 

func InitializeUnmarshaller(c *influxql.Call) (UnmarshalFunc, error)

Source Files

View all Source files

Directories

Path Synopsis
b1
bz1
tsm1
wal
Package WAL implements a write ahead log optimized for write throughput that can be put in front of the database index.
 Package WAL implements a write ahead log optimized for write throughput that can be put in front of the database index.
Package internal is a generated protocol buffer package.
 Package internal is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.