cluster

package
v0.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 11, 2016 License: MIT Imports: 27 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// DefaultWriteTimeout is the default timeout for a complete write to succeed.
	DefaultWriteTimeout = 5 * time.Second

	// DefaultShardWriterTimeout is the default timeout set on shard writers.
	DefaultShardWriterTimeout = 5 * time.Second

	// DefaultShardMapperTimeout is the default timeout set on shard mappers.
	DefaultShardMapperTimeout = 5 * time.Second

	// DefaultMaxRemoteWriteConnections is the maximum number of open connections
	// that will be available for remote writes to another host.
	DefaultMaxRemoteWriteConnections = 3

	// DefaultMaxConcurrentQueries is the maximum number of running queries.
	// A value of zero will make the maximum query limit unlimited.
	DefaultMaxConcurrentQueries = 0

	// DefaultMaxSelectPointN is the maximum number of points a SELECT can process.
	// A value of zero will make the maximum point count unlimited.
	DefaultMaxSelectPointN = 0

	// DefaultMaxSelectSeriesN is the maximum number of series a SELECT can run.
	// A value of zero will make the maximum series count unlimited.
	DefaultMaxSelectSeriesN = 0
)
View Source
const MaxMessageSize = 1024 * 1024 * 1024 // 1GB

MaxMessageSize defines how large a message can be before we reject it

View Source
const MuxHeader = 2

MuxHeader is the header byte used in the TCP mux.

Variables

View Source
var (
	// ErrTimeout is returned when a write times out.
	ErrTimeout = errors.New("timeout")

	// ErrPartialWrite is returned when a write partially succeeds but does
	// not meet the requested consistency level.
	ErrPartialWrite = errors.New("partial write")

	// ErrWriteFailed is returned when no writes succeeded.
	ErrWriteFailed = errors.New("write failed")
)

Functions

func DecodeLV added in v0.11.0

func DecodeLV(r io.Reader, v encoding.BinaryUnmarshaler) error

DecodeLV reads the length-value record from r and unmarshals it into v.

func DecodeTLV added in v0.11.0

func DecodeTLV(r io.Reader, v encoding.BinaryUnmarshaler) (typ byte, err error)

DecodeTLV reads the type-length-value record from r and unmarshals it into v.

func EncodeLV added in v0.11.0

func EncodeLV(w io.Writer, v encoding.BinaryMarshaler) error

EncodeLV encodes v to a binary format and writes the length-value record to w.

func EncodeTLV added in v0.11.0

func EncodeTLV(w io.Writer, typ byte, v encoding.BinaryMarshaler) error

EncodeTLV encodes v to a binary format and writes the record-length-value record to w.

func NewBoundedPool added in v0.10.0

func NewBoundedPool(initialCap, maxCap int, timeout time.Duration, factory Factory) (pool.Pool, error)

NewBoundedPool returns a new pool based on buffered channels with an initial capacity, maximum capacity and timeout to wait for a connection from the pool. Factory is used when initial capacity is greater than zero to fill the pool. A zero initialCap doesn't fill the Pool until a new Get() is called. During a Get(), If there is no new connection available in the pool and total connections is less than the max, a new connection will be created via the Factory() method. Othewise, the call will block until a connection is available or the timeout is reached.

func ReadLV added in v0.11.0

func ReadLV(r io.Reader) ([]byte, error)

ReadLV reads the length-value from a TLV record.

func ReadTLV added in v0.9.0

func ReadTLV(r io.Reader) (byte, []byte, error)

ReadTLV reads a type-length-value record from r.

func ReadType added in v0.11.0

func ReadType(r io.Reader) (byte, error)

ReadType reads the type from a TLV record.

func WriteLV added in v0.11.0

func WriteLV(w io.Writer, buf []byte) error

WriteLV writes the length-value in a TLV record to w.

func WriteTLV added in v0.9.0

func WriteTLV(w io.Writer, typ byte, buf []byte) error

WriteTLV writes a type-length-value record to w.

func WriteType added in v0.11.0

func WriteType(w io.Writer, typ byte) error

WriteType writes the type in a TLV record to w.

Types

type Balancer added in v0.9.5

type Balancer interface {
	// Next returns the next Node according to the balancing method
	// or nil if there are no nodes available
	Next() *meta.NodeInfo
}

Balancer represents a load-balancing algorithm for a set of nodes

func NewNodeBalancer added in v0.9.5

func NewNodeBalancer(nodes []meta.NodeInfo) Balancer

NewNodeBalancer create a shuffled, round-robin balancer so that multiple instances will return nodes in randomized order and each each returned node will be repeated in a cycle

type Config added in v0.9.0

type Config struct {
	ForceRemoteShardMapping   bool          `toml:"force-remote-mapping"`
	WriteTimeout              toml.Duration `toml:"write-timeout"`
	ShardWriterTimeout        toml.Duration `toml:"shard-writer-timeout"`
	MaxRemoteWriteConnections int           `toml:"max-remote-write-connections"`
	ShardMapperTimeout        toml.Duration `toml:"shard-mapper-timeout"`
	MaxConcurrentQueries      int           `toml:"max-concurrent-queries"`
	QueryTimeout              toml.Duration `toml:"query-timeout"`
	LogQueriesAfter           toml.Duration `toml:"log-queries-after"`
	MaxSelectPointN           int           `toml:"max-select-point"`
	MaxSelectSeriesN          int           `toml:"max-select-series"`
	MaxSelectBucketsN         int           `toml:"max-select-buckets"`
}

Config represents the configuration for the clustering service.

func NewConfig added in v0.9.0

func NewConfig() Config

NewConfig returns an instance of Config with defaults.

type CreateIteratorRequest added in v0.11.0

type CreateIteratorRequest struct {
	ShardIDs []uint64
	Opt      influxql.IteratorOptions
}

CreateIteratorRequest represents a request to create a remote iterator.

func (*CreateIteratorRequest) MarshalBinary added in v0.11.0

func (r *CreateIteratorRequest) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*CreateIteratorRequest) UnmarshalBinary added in v0.11.0

func (r *CreateIteratorRequest) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type CreateIteratorResponse added in v0.11.0

type CreateIteratorResponse struct {
	Err   error
	Type  influxql.DataType
	Stats influxql.IteratorStats
}

CreateIteratorResponse represents a response from remote iterator creation.

func (*CreateIteratorResponse) MarshalBinary added in v0.11.0

func (r *CreateIteratorResponse) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*CreateIteratorResponse) UnmarshalBinary added in v0.11.0

func (r *CreateIteratorResponse) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type ExecuteStatementRequest added in v0.11.0

type ExecuteStatementRequest struct {
	// contains filtered or unexported fields
}

ExecuteStatementRequest represents the a request to execute a statement on a node.

func (*ExecuteStatementRequest) Database added in v0.11.0

func (r *ExecuteStatementRequest) Database() string

Database returns the database name.

func (*ExecuteStatementRequest) MarshalBinary added in v0.11.0

func (r *ExecuteStatementRequest) MarshalBinary() ([]byte, error)

MarshalBinary encodes the object to a binary format.

func (*ExecuteStatementRequest) SetDatabase added in v0.11.0

func (r *ExecuteStatementRequest) SetDatabase(database string)

SetDatabase sets the database name.

func (*ExecuteStatementRequest) SetStatement added in v0.11.0

func (r *ExecuteStatementRequest) SetStatement(statement string)

SetStatement sets the InfluxQL statement.

func (*ExecuteStatementRequest) Statement added in v0.11.0

func (r *ExecuteStatementRequest) Statement() string

Statement returns the InfluxQL statement.

func (*ExecuteStatementRequest) UnmarshalBinary added in v0.11.0

func (r *ExecuteStatementRequest) UnmarshalBinary(buf []byte) error

UnmarshalBinary populates ExecuteStatementRequest from a binary format.

type ExecuteStatementResponse added in v0.11.0

type ExecuteStatementResponse struct {
	// contains filtered or unexported fields
}

ExecuteStatementResponse represents the response returned from a remote ExecuteStatementRequest call.

func (*ExecuteStatementResponse) Code added in v0.11.0

func (w *ExecuteStatementResponse) Code() int

Code returns the response code.

func (*ExecuteStatementResponse) MarshalBinary added in v0.11.0

func (w *ExecuteStatementResponse) MarshalBinary() ([]byte, error)

MarshalBinary encodes the object to a binary format.

func (*ExecuteStatementResponse) Message added in v0.11.0

func (w *ExecuteStatementResponse) Message() string

Message returns the repsonse message.

func (*ExecuteStatementResponse) SetCode added in v0.11.0

func (w *ExecuteStatementResponse) SetCode(code int)

SetCode sets the Code

func (*ExecuteStatementResponse) SetMessage added in v0.11.0

func (w *ExecuteStatementResponse) SetMessage(message string)

SetMessage sets the Message

func (*ExecuteStatementResponse) UnmarshalBinary added in v0.11.0

func (w *ExecuteStatementResponse) UnmarshalBinary(buf []byte) error

UnmarshalBinary populates ExecuteStatementResponse from a binary format.

type ExpandSourcesRequest added in v0.12.0

type ExpandSourcesRequest struct {
	ShardIDs []uint64
	Sources  influxql.Sources
}

ExpandSourcesRequest represents a request to expand regex sources.

func (*ExpandSourcesRequest) MarshalBinary added in v0.12.0

func (r *ExpandSourcesRequest) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*ExpandSourcesRequest) UnmarshalBinary added in v0.12.0

func (r *ExpandSourcesRequest) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type ExpandSourcesResponse added in v0.12.0

type ExpandSourcesResponse struct {
	Sources influxql.Sources
	Err     error
}

ExpandSourcesResponse represents a response from source expansion.

func (*ExpandSourcesResponse) MarshalBinary added in v0.12.0

func (r *ExpandSourcesResponse) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*ExpandSourcesResponse) UnmarshalBinary added in v0.12.0

func (r *ExpandSourcesResponse) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type Factory added in v0.10.0

type Factory func() (net.Conn, error)

Factory is a function to create new connections.

type FieldDimensionsRequest added in v0.11.0

type FieldDimensionsRequest struct {
	ShardIDs []uint64
	Sources  influxql.Sources
}

FieldDimensionsRequest represents a request to retrieve unique fields & dimensions.

func (*FieldDimensionsRequest) MarshalBinary added in v0.11.0

func (r *FieldDimensionsRequest) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*FieldDimensionsRequest) UnmarshalBinary added in v0.11.0

func (r *FieldDimensionsRequest) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type FieldDimensionsResponse added in v0.11.0

type FieldDimensionsResponse struct {
	Fields     map[string]struct{}
	Dimensions map[string]struct{}
	Err        error
}

FieldDimensionsResponse represents a response from remote iterator creation.

func (*FieldDimensionsResponse) MarshalBinary added in v0.11.0

func (r *FieldDimensionsResponse) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*FieldDimensionsResponse) UnmarshalBinary added in v0.11.0

func (r *FieldDimensionsResponse) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type IntoWriteRequest added in v0.11.0

type IntoWriteRequest struct {
	Database        string
	RetentionPolicy string
	Points          []models.Point
}

IntoWriteRequest is a partial copy of cluster.WriteRequest

type LocalTSDBStore added in v0.13.0

type LocalTSDBStore struct {
	*tsdb.Store
}

func (LocalTSDBStore) IteratorCreator added in v0.13.0

func (s LocalTSDBStore) IteratorCreator(shards []meta.ShardInfo) (influxql.IteratorCreator, error)

type MetaClient added in v0.11.0

type MetaClient interface {
	CreateContinuousQuery(database, name, query string) error
	CreateDatabase(name string) (*meta.DatabaseInfo, error)
	CreateDatabaseWithRetentionPolicy(name string, rpi *meta.RetentionPolicyInfo) (*meta.DatabaseInfo, error)
	CreateRetentionPolicy(database string, rpi *meta.RetentionPolicyInfo) (*meta.RetentionPolicyInfo, error)
	CreateSubscription(database, rp, name, mode string, destinations []string) error
	CreateUser(name, password string, admin bool) (*meta.UserInfo, error)
	Database(name string) *meta.DatabaseInfo
	Databases() []meta.DatabaseInfo
	DropShard(id uint64) error
	DropContinuousQuery(database, name string) error
	DropDatabase(name string) error
	DropRetentionPolicy(database, name string) error
	DropSubscription(database, rp, name string) error
	DropUser(name string) error
	RetentionPolicy(database, name string) (rpi *meta.RetentionPolicyInfo, err error)
	SetAdminPrivilege(username string, admin bool) error
	SetDefaultRetentionPolicy(database, name string) error
	SetPrivilege(username, database string, p influxql.Privilege) error
	ShardsByTimeRange(sources influxql.Sources, tmin, tmax time.Time) (a []meta.ShardInfo, err error)
	UpdateRetentionPolicy(database, name string, rpu *meta.RetentionPolicyUpdate) error
	UpdateUser(name, password string) error
	UserPrivilege(username, database string) (*influxql.Privilege, error)
	UserPrivileges(username string) (map[string]influxql.Privilege, error)
	Users() []meta.UserInfo
}

MetaClient is an interface for accessing meta data.

type PointsWriter added in v0.9.0

type PointsWriter struct {
	WriteTimeout time.Duration
	Logger       *log.Logger

	Node *influxdb.Node

	MetaClient interface {
		Database(name string) (di *meta.DatabaseInfo)
		RetentionPolicy(database, policy string) (*meta.RetentionPolicyInfo, error)
		CreateShardGroup(database, policy string, timestamp time.Time) (*meta.ShardGroupInfo, error)
		ShardOwner(shardID uint64) (string, string, *meta.ShardGroupInfo)
	}

	TSDBStore interface {
		CreateShard(database, retentionPolicy string, shardID uint64) error
		WriteToShard(shardID uint64, points []models.Point) error
	}

	ShardWriter interface {
		WriteShard(shardID, ownerID uint64, points []models.Point) error
	}

	HintedHandoff interface {
		WriteShard(shardID, ownerID uint64, points []models.Point) error
	}

	Subscriber interface {
		Points() chan<- *WritePointsRequest
	}
	// contains filtered or unexported fields
}

PointsWriter handles writes across multiple local and remote data nodes.

func NewPointsWriter added in v0.9.0

func NewPointsWriter() *PointsWriter

NewPointsWriter returns a new instance of PointsWriter for a node.

func (*PointsWriter) Close added in v0.9.0

func (w *PointsWriter) Close() error

Close closes the communication channel with the point writer

func (*PointsWriter) MapShards added in v0.9.0

func (w *PointsWriter) MapShards(wp *WritePointsRequest) (*ShardMapping, error)

MapShards maps the points contained in wp to a ShardMapping. If a point maps to a shard group or shard that does not currently exist, it will be created before returning the mapping.

func (*PointsWriter) Open added in v0.9.0

func (w *PointsWriter) Open() error

Open opens the communication channel with the point writer

func (*PointsWriter) SetLogOutput added in v0.13.0

func (w *PointsWriter) SetLogOutput(lw io.Writer)

SetLogOutput sets the writer to which all logs are written. It must not be called after Open is called.

func (*PointsWriter) WritePoints added in v0.9.0

func (w *PointsWriter) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, points []models.Point) error

WritePoints writes across multiple local and remote data nodes according the consistency level.

func (*PointsWriter) WritePointsInto added in v0.9.5

func (w *PointsWriter) WritePointsInto(p *IntoWriteRequest) error

WritePointsInto is a copy of WritePoints that uses a tsdb structure instead of a cluster structure for information. This is to avoid a circular dependency

type SeriesKeysRequest added in v0.11.0

type SeriesKeysRequest struct {
	ShardIDs []uint64
	Opt      influxql.IteratorOptions
}

SeriesKeysRequest represents a request to retrieve a list of series keys.

func (*SeriesKeysRequest) MarshalBinary added in v0.11.0

func (r *SeriesKeysRequest) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*SeriesKeysRequest) UnmarshalBinary added in v0.11.0

func (r *SeriesKeysRequest) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type SeriesKeysResponse added in v0.11.0

type SeriesKeysResponse struct {
	SeriesList influxql.SeriesList
	Err        error
}

SeriesKeysResponse represents a response from retrieving series keys.

func (*SeriesKeysResponse) MarshalBinary added in v0.11.0

func (r *SeriesKeysResponse) MarshalBinary() ([]byte, error)

MarshalBinary encodes r to a binary format.

func (*SeriesKeysResponse) UnmarshalBinary added in v0.11.0

func (r *SeriesKeysResponse) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes data into r.

type Service added in v0.9.0

type Service struct {
	Listener net.Listener

	TSDBStore TSDBStore

	Logger *log.Logger
	// contains filtered or unexported fields
}

Service processes data received over raw TCP connections.

func NewService added in v0.9.0

func NewService(c Config) *Service

NewService returns a new instance of Service.

func (*Service) Close added in v0.9.0

func (s *Service) Close() error

Close shuts down the listener and waits for all connections to finish.

func (*Service) Open added in v0.9.0

func (s *Service) Open() error

Open opens the network listener and begins serving requests.

func (*Service) SetLogOutput added in v0.13.0

func (s *Service) SetLogOutput(w io.Writer)

SetLogOutput sets the writer to which all logs are written. It must not be called after Open is called.

type ShardIteratorCreator added in v0.13.0

type ShardIteratorCreator interface {
	ShardIteratorCreator(id uint64) influxql.IteratorCreator
}

ShardIteratorCreator is an interface for creating an IteratorCreator to access a specific shard.

type ShardMapping added in v0.9.0

type ShardMapping struct {
	Points map[uint64][]models.Point  // The points associated with a shard ID
	Shards map[uint64]*meta.ShardInfo // The shards that have been mapped, keyed by shard ID
}

ShardMapping contains a mapping of a shards to a points.

func NewShardMapping added in v0.9.0

func NewShardMapping() *ShardMapping

NewShardMapping creates an empty ShardMapping

func (*ShardMapping) MapPoint added in v0.9.0

func (s *ShardMapping) MapPoint(shardInfo *meta.ShardInfo, p models.Point)

MapPoint maps a point to shard

type StatementExecutor added in v0.13.0

type StatementExecutor struct {
	MetaClient MetaClient

	// TSDB storage for local node.
	TSDBStore TSDBStore

	// Holds monitoring data for SHOW STATS and SHOW DIAGNOSTICS.
	Monitor *monitor.Monitor

	// Used for rewriting points back into system for SELECT INTO statements.
	PointsWriter interface {
		WritePointsInto(*IntoWriteRequest) error
	}

	// Select statement limits
	MaxSelectPointN   int
	MaxSelectSeriesN  int
	MaxSelectBucketsN int
}

StatementExecutor executes a statement in the query.

func (*StatementExecutor) ExecuteStatement added in v0.13.0

func (e *StatementExecutor) ExecuteStatement(stmt influxql.Statement, ctx *influxql.ExecutionContext) error

func (*StatementExecutor) NormalizeStatement added in v0.13.0

func (e *StatementExecutor) NormalizeStatement(stmt influxql.Statement, defaultDatabase string) (err error)

NormalizeStatement adds a default database and policy to the measurements in statement.

type TSDBStore added in v0.11.0

type TSDBStore interface {
	CreateShard(database, policy string, shardID uint64) error
	WriteToShard(shardID uint64, points []models.Point) error

	DeleteDatabase(name string) error
	DeleteMeasurement(database, name string) error
	DeleteRetentionPolicy(database, name string) error
	DeleteSeries(database string, sources []influxql.Source, condition influxql.Expr) error
	DeleteShard(id uint64) error
	IteratorCreator(shards []meta.ShardInfo) (influxql.IteratorCreator, error)
}

TSDBStore is an interface for accessing the time series data store.

type WritePointsRequest added in v0.9.0

type WritePointsRequest struct {
	Database        string
	RetentionPolicy string
	Points          []models.Point
}

WritePointsRequest represents a request to write point data to the cluster

func (*WritePointsRequest) AddPoint added in v0.9.0

func (w *WritePointsRequest) AddPoint(name string, value interface{}, timestamp time.Time, tags map[string]string)

AddPoint adds a point to the WritePointRequest with field key 'value'

type WriteShardRequest added in v0.9.0

type WriteShardRequest struct {
	// contains filtered or unexported fields
}

WriteShardRequest represents the a request to write a slice of points to a shard

func (*WriteShardRequest) AddPoint added in v0.9.0

func (w *WriteShardRequest) AddPoint(name string, value interface{}, timestamp time.Time, tags map[string]string)

AddPoint adds a new time series point

func (*WriteShardRequest) AddPoints added in v0.9.0

func (w *WriteShardRequest) AddPoints(points []models.Point)

AddPoints adds a new time series point

func (*WriteShardRequest) Database added in v0.11.0

func (w *WriteShardRequest) Database() string

func (*WriteShardRequest) MarshalBinary added in v0.9.0

func (w *WriteShardRequest) MarshalBinary() ([]byte, error)

MarshalBinary encodes the object to a binary format.

func (*WriteShardRequest) Points added in v0.9.0

func (w *WriteShardRequest) Points() []models.Point

Points returns the time series Points

func (*WriteShardRequest) RetentionPolicy added in v0.11.0

func (w *WriteShardRequest) RetentionPolicy() string

func (*WriteShardRequest) SetDatabase added in v0.11.0

func (w *WriteShardRequest) SetDatabase(db string)

func (*WriteShardRequest) SetRetentionPolicy added in v0.11.0

func (w *WriteShardRequest) SetRetentionPolicy(rp string)

func (*WriteShardRequest) SetShardID added in v0.9.0

func (w *WriteShardRequest) SetShardID(id uint64)

SetShardID sets the ShardID

func (*WriteShardRequest) ShardID added in v0.9.0

func (w *WriteShardRequest) ShardID() uint64

ShardID gets the ShardID

func (*WriteShardRequest) UnmarshalBinary added in v0.9.0

func (w *WriteShardRequest) UnmarshalBinary(buf []byte) error

UnmarshalBinary populates WritePointRequest from a binary format.

type WriteShardResponse added in v0.9.0

type WriteShardResponse struct {
	// contains filtered or unexported fields
}

WriteShardResponse represents the response returned from a remote WriteShardRequest call

func (*WriteShardResponse) Code added in v0.9.0

func (w *WriteShardResponse) Code() int

Code returns the Code

func (*WriteShardResponse) MarshalBinary added in v0.9.0

func (w *WriteShardResponse) MarshalBinary() ([]byte, error)

MarshalBinary encodes the object to a binary format.

func (*WriteShardResponse) Message added in v0.9.0

func (w *WriteShardResponse) Message() string

Message returns the Message

func (*WriteShardResponse) SetCode added in v0.9.0

func (w *WriteShardResponse) SetCode(code int)

SetCode sets the Code

func (*WriteShardResponse) SetMessage added in v0.9.0

func (w *WriteShardResponse) SetMessage(message string)

SetMessage sets the Message

func (*WriteShardResponse) UnmarshalBinary added in v0.9.0

func (w *WriteShardResponse) UnmarshalBinary(buf []byte) error

UnmarshalBinary populates WritePointRequest from a binary format.

Directories

Path Synopsis
Package cluster is a generated protocol buffer package.
Package cluster is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL