Documentation
¶
Index ¶
- Variables
- func Register(name string, fn func(*pipe.Pipe, string, Config) (StopStartListener, error))
- type Config
- type Elasticsearch
- type Error
- type ErrorLevel
- type File
- type FileConfig
- type Influxdb
- type Mongodb
- type MongodbConfig
- type Rethinkdb
- type StopStartListener
- func Createadaptor(kind, path string, extra Config, p *pipe.Pipe) (adaptor StopStartListener, err error)
- func NewElasticsearch(p *pipe.Pipe, path string, extra Config) (StopStartListener, error)
- func NewFile(p *pipe.Pipe, path string, extra Config) (StopStartListener, error)
- func NewInfluxdb(p *pipe.Pipe, path string, extra Config) (StopStartListener, error)
- func NewMongodb(p *pipe.Pipe, path string, extra Config) (StopStartListener, error)
- func NewRethinkdb(p *pipe.Pipe, path string, extra Config) (StopStartListener, error)
- func NewTransformer(p *pipe.Pipe, path string, extra Config) (StopStartListener, error)
- type Transformer
- type TransformerConfig
Constants ¶
This section is empty.
Variables ¶
var ( // ErrMissingNode is returned when the requested node type was not found in the map ErrMissingNode = errors.New("adaptor not found in registry") )
Functions ¶
Types ¶
type Config ¶
type Config map[string]interface{}
Config is an alias to map[string]interface{} and helps us turn a fuzzy document into a conrete named struct
type Elasticsearch ¶
type Elasticsearch struct {
// contains filtered or unexported fields
}
Elasticsearch is an adaptor to connect a pipeline to an elasticsearch cluster.
func (*Elasticsearch) Start ¶
func (e *Elasticsearch) Start() error
Start the adaptor as a source (not implemented)
type Error ¶
type Error struct { Lvl ErrorLevel Str string Path string Record bson.M }
Error is an error that happened during an adaptor's operation. Error's include both an indication of the severity, Level, as well as a reference to the Record that was in process when the error occured
type ErrorLevel ¶
type ErrorLevel int
ErrorLevel indicated the severity of the error
const ( NOTICE ErrorLevel = iota WARNING ERROR CRITICAL )
Adaptor errors have levels to indicate their severity. CRITICAL errors indicate that the program cannot continue running.
ERROR errors indicate a problem with a specific document or message. a document might not have been applied properly to a source, but the program can continue
WARNING Todo ¶
NOTICE ToDo
type File ¶
type File struct {
// contains filtered or unexported fields
}
File is an adaptor that can be used as a source / sink for file's on disk, as well as a sink to stdout.
type FileConfig ¶
type FileConfig struct { // URI pointing to the resource. We only recognize file:// and stdout:// currently URI string `json:"uri"` }
FileConfig is used to configure the File Adaptor,
type Influxdb ¶
type Influxdb struct {
// contains filtered or unexported fields
}
Influxdb is an adaptor that writes metrics to influxdb (http://influxdb.com/) a high performant time series database
type Mongodb ¶
type Mongodb struct {
// contains filtered or unexported fields
}
Mongodb is an adaptor to read / write to mongodb. it works as a source by copying files, and then optionally tailing the oplog
type MongodbConfig ¶
type MongodbConfig struct { URI string `json:"uri"` Namespace string `json:"namespace"` Debug bool `json:"debug"` Tail bool `json:"tail"` }
MongodbConfig provides configuration options for a mongodb adaptor the notable difference between this and dbConfig is the presence of the Tail option
type Rethinkdb ¶
type Rethinkdb struct {
// contains filtered or unexported fields
}
Rethinkdb is an adaptor that writes metrics to rethinkdb (http://rethinkdb.com/) An open-source distributed database
type StopStartListener ¶
StopStartListener defines the interface that all database connectors and nodes must follow. Start() consumes data from the interface, Listen() listens on a pipe, processes data, and then emits it. Stop() shuts down the adaptor
func Createadaptor ¶
func Createadaptor(kind, path string, extra Config, p *pipe.Pipe) (adaptor StopStartListener, err error)
Createadaptor instantiates an adaptor given the adaptor type and the Config. Constructors are expected to be in the form
func NewWhatever(p *pipe.Pipe, extra Config) (*Whatever, error) {}
and are expected to confirm to the adaptor interface
func NewElasticsearch ¶
NewElasticsearch creates a new Elasticsearch adaptor. Elasticsearch adaptors cannot be used as a source,
func NewInfluxdb ¶
NewInfluxdb creates an Influxdb adaptor
func NewMongodb ¶
NewMongodb creates a new Mongodb adaptor
func NewRethinkdb ¶
NewRethinkdb creates a new Rethinkdb database adaptor
func NewTransformer ¶
NewTransformer creates a new transformer object
type Transformer ¶
type Transformer struct {
// contains filtered or unexported fields
}
Transformer is an adaptor which consumes data from a source, transforms it using a supplied javascript function and then emits it. The javascript transformation function is supplied as a seperate file on disk, and is called by calling the defined module.exports function
func (*Transformer) Listen ¶
func (t *Transformer) Listen() (err error)
Listen starts the transformer's listener, reads each message from the incoming channel transformers it into mejson, and then uses the supplied javascript module.exports function to transform the document. The document is then emited to this adaptor's children
func (*Transformer) Start ¶
func (t *Transformer) Start() error
Start the adaptor as a source (not implemented for this adaptor)
type TransformerConfig ¶
type TransformerConfig struct { // file containing transformer javascript // must define a module.exports = function(doc) { .....; return doc } Filename string `json:"filename"` // verbose output Debug bool `json:"debug"` // debug mode }
TransformerConfig holds config options for a transformer adaptor