format

package
v0.4.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 29, 2016 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Base64Decode 

type Base64Decode struct {
	// contains filtered or unexported fields
}

Base64Decode formatter plugin Base64Decode is a formatter that decodes a base64 message. If a message is not or only partly base64 encoded an error will be logged and the decoded part is returned. RFC 4648 is expected. Configuration example

  • "stream.Broadcast": Formatter: "format.Base64Decode" Base64Formatter: "format.Forward" Base64Dictionary: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890+/"

Base64Dictionary defines the 64-character base64 lookup dictionary to use. When left empty a dictionary as defined by RFC4648 is used. This is the default.

Base64DataFormatter defines a formatter that is applied before the base64 decoding takes place. By default this is set to "format.Forward"

func (*Base64Decode) Configure 

func (format *Base64Decode) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Base64Decode) Format 

func (format *Base64Decode) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format returns the original message payload

type Base64Encode 

type Base64Encode struct {
	// contains filtered or unexported fields
}

Base64Encode formatter plugin Base64Encode is a formatter that encodes a message as base64. Configuration example

  • "stream.Broadcast": Formatter: "format.Base64Encode" Base64Formatter: "format.Forward" Base64Dictionary: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890+/"

Base64Dictionary defines the 64-character base64 lookup dictionary to use. When left empty a dictionary as defined by RFC4648 is used. This is the default.

Base64DataFormatter defines a formatter that is applied before the base64 encoding takes place. By default this is set to "format.Forward"

func (*Base64Encode) Configure 

func (format *Base64Encode) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Base64Encode) Format 

func (format *Base64Encode) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format returns the original message payload

type CollectdToInflux08 added in v0.4.0 

type CollectdToInflux08 struct {
	// contains filtered or unexported fields
}

CollectdToInflux08 formatter plugin CollectdToInflux08 provides a transformation from collectd JSON data to InfluxDB 0.8.x compatible JSON data. Trailing and leading commas are removed from the Collectd message beforehand. Configuration example

  • "stream.Broadcast": Formatter: "format.CollectdToInflux08" CollectdToInfluxFormatter: "format.Forward"

CollectdToInfluxFormatter defines the formatter applied before the conversion from Collectd to InfluxDB. By default this is set to format.Forward.

func (*CollectdToInflux08) Configure added in v0.4.0 

func (format *CollectdToInflux08) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*CollectdToInflux08) Format added in v0.4.0 

func (format *CollectdToInflux08) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format transforms collectd data to influx 0.8.x data

type CollectdToInflux09 added in v0.4.0 

type CollectdToInflux09 struct {
	// contains filtered or unexported fields
}

CollectdToInflux09 formatter plugin CollectdToInflux09 provides a transformation from collectd JSON data to InfluxDB 0.9.x compatible JSON data. Trailing and leading commas are removed from the Collectd message beforehand. Configuration example

  • "stream.Broadcast": Formatter: "format.CollectdToInflux09" CollectdToInfluxFormatter: "format.Forward"

CollectdToInfluxFormatter defines the formatter applied before the conversion from Collectd to InfluxDB. By default this is set to format.Forward.

func (*CollectdToInflux09) Configure added in v0.4.0 

func (format *CollectdToInflux09) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*CollectdToInflux09) Format added in v0.4.0 

func (format *CollectdToInflux09) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format transforms collectd data to influx 0.9.x data

type CollectdToInflux10 added in v0.4.1 

type CollectdToInflux10 struct {
	// contains filtered or unexported fields
}

CollectdToInflux10 formatter plugin CollectdToInflux10 provides a transformation from collectd JSON data to InfluxDB 0.9.1+ compatible line protocol data. Trailing and leading commas are removed from the Collectd message beforehand. Configuration example

  • "stream.Broadcast": Formatter: "format.CollectdToInflux10" CollectdToInflux10Formatter: "format.Forward"

CollectdToInfluxFormatter defines the formatter applied before the conversion from Collectd to InfluxDB. By default this is set to format.Forward.

func (*CollectdToInflux10) Configure added in v0.4.1 

func (format *CollectdToInflux10) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*CollectdToInflux10) Format added in v0.4.1 

func (format *CollectdToInflux10) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format transforms collectd data to influx 0.9.x data

type Envelope 

type Envelope struct {
	// contains filtered or unexported fields
}

Envelope formatter plugin Envelope is a formatter that allows prefixing and/or postfixing a message with configurable strings. Configuration example

  • "stream.Broadcast": Formatter: "format.Envelope" EnvelopeFormatter: "format.Forward" EnvelopePrefix: "" EnvelopePostfix: "\n"

EnvelopePrefix defines the message prefix. By default this is set to "". Special characters like \n \r \t will be transformed into the actual control characters.

EnvelopePostfix defines the message postfix. By default this is set to "\n". Special characters like \n \r \t will be transformed into the actual control characters.

EnvelopeFormatter defines the formatter for the data transferred as message. By default this is set to "format.Forward"

func (*Envelope) Configure 

func (format *Envelope) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Envelope) Format 

func (format *Envelope) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format adds prefix and postfix to the message formatted by the base formatter

type ExtractJSON added in v0.4.3 

type ExtractJSON struct {
	// contains filtered or unexported fields
}

ExtractJSON formatter plugin ExtractJSON is a formatter that extracts a single value from a JSON message. Configuration example

  • "stream.Broadcast": Formatter: "format.ExtractJSON" ExtractJSONdataFormatter: "format.Forward" ExtractJSONField: "" ExtractJSONTrimValues: true ExtractJSONPrecision: 0

ExtractJSONDataFormatter formatter that will be applied before the field is extracted. Set to format.Forward by default.

ExtractJSONField defines the field to extract. This value is empty by default. If the field does not exist an empty string is returned.

ExtractJSONTrimValues will trim whitspaces from the value if enabled. Enabled by default.

ExtractJSONPrecision defines the floating point precision of number values. By default this is set to 0 i.e. all decimal places will be omitted.

func (*ExtractJSON) Configure added in v0.4.3 

func (format *ExtractJSON) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*ExtractJSON) Format added in v0.4.3 

func (format *ExtractJSON) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format modifies the JSON payload of this message

type Forward 

type Forward struct {
}

Forward formatter plugin Forward is a formatter that passes a message as is Configuration example

  • "stream.Broadcast": Formatter: "format.Forward"

func (*Forward) Configure 

func (format *Forward) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Forward) Format 

func (format *Forward) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format returns the original message payload

type Hostname 

type Hostname struct {
	// contains filtered or unexported fields
}

Hostname formatter plugin Hostname is a formatter that prefixes a message with the hostname. Configuration example

  • "stream.Broadcast": Formatter: "format.Hostname" HostnameFormatter: "format.Envelope" HostnameSeparator: " "

HostnameDataFormatter defines the formatter for the data transferred as message. By default this is set to "format.Envelope"

HostnameSeparator sets the separator character placed after the hostname. This is set to " " by default.

func (*Hostname) Configure 

func (format *Hostname) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Hostname) Format 

func (format *Hostname) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format prepends the Hostname of the message to the message.

type Identifier 

type Identifier struct {
	// contains filtered or unexported fields
}

Identifier formatter plugin Identifier is a formatter that will generate a (mostly) unique 64 bit identifier number from the message timestamp and sequence number. The message payload will not be encoded. Configuration example

  • "stream.Broadcast": Formatter: "format.Identifier" IdentifierType: "hash" IdentifierDataFormatter: "format.Forward"

IdentifierType defines the algorithm used to generate the message id. This my be one of the following: "hash", "time", "seq", "seqhex". By default this is set to "time".

  • When using "hash" the message payload will be hashed using fnv1a and returned as hex.
  • When using "time" the id will be formatted YYMMDDHHmmSSxxxxxxx where x denotes the sequence number modulo 10000000. I.e. 10mil messages per second are possible before there is a collision.
  • When using "seq" the id will be returned as the integer representation of the sequence number.
  • When using "seqhex" the id will be returned as the hex representation of the sequence number.

IdentifierDataFormatter defines the formatter for the data that is used to build the identifier from. By default this is set to "format.Forward"

func (*Identifier) Configure 

func (format *Identifier) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Identifier) Format 

func (format *Identifier) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format generates a unique identifier from the message contents or metadata.

type JSON 

type JSON struct {
	// contains filtered or unexported fields
}

JSON formatter plugin JSON is a formatter that passes a message encapsulated as JSON in the form {"message":"..."}. The actual message is formatted by a nested formatter and HTML escaped. Configuration example

  • "stream.Broadcast": Formatter: "format.JSON" JSONStartState: "findKey" JSONDirectives:
  • 'findKey :": key ::'
  • 'findKey :}: : pop : end'
  • 'key :": findVal : : key'
  • 'findVal :\:: value ::'

JSONStartState defines the initial parser state when parsing a message. By default this is set to "" which will fall back to the first state used in the JSONDirectives array.

JSONTimestampRead defines the go timestamp format expected from fields that are parsed as "dat". By default this is set to "20060102150405".

JSONTimestampWrite defines the go timestamp format that "dat" fields will be converted to. By default this is set to "2006-01-02 15:04:05 MST".

JSONDirectives defines an array of parser directives. This setting is mandatory and has no default value. Each string must be of the following format: "State:Token:NextState:Flags:Function". Spaces will be stripped from all fields but Token. If a fields requires a colon it has to be escaped with a backslash. Other escape characters supported are \n, \r and \t.

Flags (JSONDirectives) can be a comma separated set of the following flags.

  • continue -> Prepend the token to the next match.
  • append -> Append the token to the current match and continue reading.
  • include -> Append the token to the current match.
  • push -> Push the current state to the stack.
  • pop -> Pop the stack and use the returned state if possible.

Function (JSONDirectives) can hold one of the following names.

  • key -> Write the current match as a key.
  • val -> Write the current match as a value without quotes.
  • esc -> Write the current match as a escaped string value.
  • dat -> Write the current match as a timestamp value.
  • arr -> Start a new array.
  • obj -> Start a new object.
  • end -> Close an array or object.
  • arr+val -> arr followed by val.
  • arr+esc -> arr followed by esc.
  • arr+dat -> arr followed by dat.
  • val+end -> val followed by end.
  • esc+end -> esc followed by end.
  • dat+end -> dat followed by end.

Rules for storage (JSONDirectives): if a value is written without a previous key write, a key will be auto generated from the current parser state name. This does not happen when inside an array. If key is written without a previous value write, a null value will be written. This does not happen after an object has been started. A key write inside an array will cause the array to be closed. If the array is nested, all arrays will be closed.

func (*JSON) Configure 

func (format *JSON) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*JSON) Format 

func (format *JSON) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format parses the incoming message and generates JSON from it. This function is mutex locked.

type ProcessJSON added in v0.4.2 

type ProcessJSON struct {
	// contains filtered or unexported fields
}

ProcessJSON formatter plugin ProcessJSON is a formatter that allows modifications to fields of a given JSON message. The message is modified and returned again as JSON. Configuration example

  • "stream.Broadcast": Formatter: "format.processJSON" ProcessJSONDataFormatter: "format.Forward" ProcessJSONDirectives:
  • "host:split: :host:@timestamp"
  • "@timestamp:time:20060102150405:2006-01-02 15\\:04\\:05"
  • "error:replace:°:\n"
  • "text:trim: \t"
  • "foo:rename:bar" ProcessJSONTrimValues: true

ProcessJSONDataFormatter formatter that will be applied before ProcessJSONDirectives are processed.

ProcessJSONDirectives defines the action to be applied to the json payload. Directives are processed in order of appearance. The directives have to be given in the form of key:operation:parameters, where operation can be one of the following: - split:<string>:{<key>:<key>:...} Split the value by a string and set the resulting array elements to the given fields in order of appearance. - replace:<old>:<new> replace a given string in the value with a new one - trim:<characters> remove the given characters (not string!) from the start and end of the value - rename:<old>:<new> rename a given field - timestamp:<read>:<write> read a timestamp and transform it into another format

ProcessJSONTrimValues will trim whitspaces from all values if enabled. Enabled by default.

func (*ProcessJSON) Configure added in v0.4.2 

func (format *ProcessJSON) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*ProcessJSON) Format added in v0.4.2 

func (format *ProcessJSON) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format modifies the JSON payload of this message

type Runlength 

type Runlength struct {
	// contains filtered or unexported fields
}

Runlength formatter plugin Runlength is a formatter that prepends the length of the message, followed by a ":". The actual message is formatted by a nested formatter. Configuration example

  • "stream.Broadcast": Formatter: "format.Runlength" RunlengthSeparator: ":" RunlengthFormatter: "format.Envelope"

RunlengthSeparator sets the separator character placed after the runlength. This is set to ":" by default.

RunlengthDataFormatter defines the formatter for the data transferred as message. By default this is set to "format.Forward"

func (*Runlength) Configure 

func (format *Runlength) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Runlength) Format 

func (format *Runlength) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format prepends the length of the message (followed by ":") to the message.

type Sequence 

type Sequence struct {
	// contains filtered or unexported fields
}

Sequence formatter plugin Sequence is a formatter that allows prefixing a message with the message's sequence number Configuration example

  • "stream.Broadcast": Formatter: "format.Sequence" SequenceFormatter: "format.Envelope" SequenceSeparator: ":"

SequenceSeparator sets the separator character placed after the sequence number. This is set to ":" by default.

SequenceDataFormatter defines the formatter for the data transferred as message. By default this is set to "format.Forward"

func (*Sequence) Configure 

func (format *Sequence) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Sequence) Format 

func (format *Sequence) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format prepends the sequence number of the message (followed by ":") to the message.

type Serialize added in v0.4.0 

type Serialize struct {
	// contains filtered or unexported fields
}

Serialize formatter plugin Serialize is a formatter that serializes a message for later retrieval. Configuration example

  • "stream.Broadcast": Formatter: "format.Serialize" SerializeFormatter: "format.Envelope" SerializeStringEncode: true

SerializeFormatter defines the formatter for the data transferred as message. By default this is set to "format.Forward"

SerializeStringEncode causes the serialized data to be base64 encoded and newline separated. This is enabled by default.

func (*Serialize) Configure added in v0.4.0 

func (format *Serialize) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Serialize) Format added in v0.4.0 

func (format *Serialize) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format prepends the sequence number of the message (followed by ":") to the message.

type SplitToJSON added in v0.4.2 

type SplitToJSON struct {
	// contains filtered or unexported fields
}

SplitToJSON formatter plugin SplitToJSON is a formatter that splits a message by a given token and puts the result into a JSON object by using an array based mapping Configuration example

  • "stream.Broadcast": Formatter: "format.SplitToJSON" SplitToJSONDataFormatter: "format.Forward" SplitToJSONToken: "|" SplitToJSONKeys:
  • "timestamp"
  • "server"
  • "error"

SplitToJSONDataFormatter defines the formatter to apply before executing this formatter. Set to "format.Forward" by default.

SplitToJSONToken defines the separator character to use when processing a message. By default this is set to "|".

SplitToJSONKeys defines an array of keys to apply to the tokens generated by splitting a message by SplitToJSONToken. The keys listed here are applied to the resulting token array by index. This list is empty by default.

func (*SplitToJSON) Configure added in v0.4.2 

func (format *SplitToJSON) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*SplitToJSON) Format added in v0.4.2 

func (format *SplitToJSON) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format returns the splitted message payload as json

type StreamName added in v0.4.0 

type StreamName struct {
	// contains filtered or unexported fields
}

StreamName formatter plugin StreamName is a formatter that prefixes a message with the StreamName. Configuration example

  • "stream.Broadcast": Formatter: "format.StreamName" StreamNameFormatter: "format.Envelope" StreamNameSeparator: " " StreamNameHistory: false

StreamNameFormatter defines the formatter for the data transferred as message. By default this is set to "format.Envelope"

StreamNameHistory can be set to true to not use the current but the previous stream name. This can be usefull to e.g. get the name of the stream messages were dropped from. By default this is set to false.

StreamNameSeparator sets the separator character placed after the stream name. This is set to " " by default.

func (*StreamName) Configure added in v0.4.0 

func (format *StreamName) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*StreamName) Format added in v0.4.0 

func (format *StreamName) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format prepends the StreamName of the message to the message.

type StreamRevert added in v0.4.0 

type StreamRevert struct {
	// contains filtered or unexported fields
}

StreamRevert formatter plugin StreamRevert is a formatter that recovers the last used stream from a message and sets it as a new target stream. Streams change whenever the Stream.Route or Message.Route function is used. This e.g. happens after a Drop call. Configuration example

  • "stream.Broadcast": Formatter: "format.StreamRevert" StreamRevertFormatter: "format.Forward"

StreamRevertFormatter defines the formatter applied after reading the stream. This formatter is applied to the data after StreamRevertDelimiter. By default this is set to "format.Forward"

func (*StreamRevert) Configure added in v0.4.0 

func (format *StreamRevert) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*StreamRevert) Format added in v0.4.0 

func (format *StreamRevert) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format adds prefix and postfix to the message formatted by the base formatter

type StreamRoute added in v0.4.0 

type StreamRoute struct {
	// contains filtered or unexported fields
}

StreamRoute formatter plugin StreamRoute is a formatter that modifies a message's stream by reading a prefix from the message's data (and discarding it). The prefix is defined by everything before a given delimiter in the message. If no delimiter is found or the prefix is empty the message stream is not changed. Configuration example

  • "stream.Broadcast": Formatter: "format.StreamRoute" StreamRouteFormatter: "format.Forward" StreamRouteStreamFormatter: "format.Forward" StreamRouteDelimiter: ":" StreamRouteFormatStream: false

StreamRouteFormatter defines the formatter applied after reading the stream. This formatter is applied to the data after StreamRouteDelimiter. By default this is set to "format.Forward"

StreamRouteStreamFormatter is used when StreamRouteFormatStream is set to true. By default this is the same value as StreamRouteFormatter.

StreamRouteDelimiter defines the delimiter to search when extracting the stream name. By default this is set to ":".

StreamRouteFormatStream can be set to true to apply StreamRouteFormatter to both parts of the message (stream and data). Set to false by default.

func (*StreamRoute) Configure added in v0.4.0 

func (format *StreamRoute) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*StreamRoute) Format added in v0.4.0 

func (format *StreamRoute) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format adds prefix and postfix to the message formatted by the base formatter

type Timestamp 

type Timestamp struct {
	// contains filtered or unexported fields
}

Timestamp formatter plugin Timestamp is a formatter that allows prefixing a message with a timestamp (time of arrival at gollum) as well as postfixing it with a delimiter string. Configuration example

  • "stream.Broadcast": Formatter: "format.Timestamp" TimestampFormatter: "format.Envelope" Timestamp: "2006-01-02T15:04:05.000 MST | "

Timestamp defines a Go time format string that is used to format the actual timestamp that prefixes the message. By default this is set to "2006-01-02 15:04:05 MST | "

TimestampFormatter defines the formatter for the data transferred as message. By default this is set to "format.Forward"

func (*Timestamp) Configure 

func (format *Timestamp) Configure(conf core.PluginConfig) error

Configure initializes this formatter with values from a plugin config.

func (*Timestamp) Format 

func (format *Timestamp) Format(msg core.Message) ([]byte, core.MessageStreamID)

Format prepends the timestamp of the message to the message.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.