parser

Documentation

Overview

• skogul, test avro parser
• Copyright (c) 2022 Telenor Norge AS
• Author:
• - Roshini Narasimha Raghavan <roshiragavi@gmail.com>
• - Kristian Lyngstøl <kly@kly.no> *
• This library is free software; you can redistribute it and/or
• modify it under the terms of the GNU Lesser General Public
• License as published by the Free Software Foundation; either
• version 2.1 of the License, or (at your option) any later version. *
• This library is distributed in the hope that it will be useful,
• but WITHOUT ANY WARRANTY; without even the implied warranty of
• MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
• Lesser General Public License for more details. *
• You should have received a copy of the GNU Lesser General Public
• License along with this library; if not, write to the Free Software
• Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
• 02110-1301 USA

Package parser is responsible for interpreting raw byte data into Containers. Each parser is disconnected from how the byte data is received and what happens with it afterwards.

The simplest parser is probably the JSON parser, which simply assumes the byte slice is a JSON representation of a Skogul Container.

• skogul, gob parser *

• Copyright (c) 2022 Telenor Norge AS

• Author(s):

• - Roshini Narasimha Raghavan <roshiragavi@gmail.com>

• - Kristian Lyngstøl <kly@kly.no>

• This library is free software; you can redistribute it and/or

• modify it under the terms of the GNU Lesser General Public

• License as published by the Free Software Foundation; either

• version 2.1 of the License, or (at your option) any later version. *

• This library is distributed in the hope that it will be useful,

• but WITHOUT ANY WARRANTY; without even the implied warranty of

• MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU

• Lesser General Public License for more details. *

• You should have received a copy of the GNU Lesser General Public

• License along with this library; if not, write to the Free Software

• Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA

• 02110-1301 USA

• skogul, prometheus parser *

• Copyright (c) 2022 Telenor Norge AS

• Author(s):

• - Roshini Narasimha Raghavan <roshiragavi@gmail.com>

• - Kristian Lyngstøl <kly@kly.no>

• This library is free software; you can redistribute it and/or

• modify it under the terms of the GNU Lesser General Public

• License as published by the Free Software Foundation; either

• version 2.1 of the License, or (at your option) any later version. *

• This library is distributed in the hope that it will be useful,

• but WITHOUT ANY WARRANTY; without even the implied warranty of

• MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU

• Lesser General Public License for more details. *

• You should have received a copy of the GNU Lesser General Public

• License along with this library; if not, write to the Free Software

• Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA

• 02110-1301 USA

Index

• Variables
• type AVRO
  • func (x *AVRO) Parse(b []byte) (*skogul.Container, error)
  • func (x *AVRO) ParseMetric(m *skogul.Metric) ([]byte, error)
• type AvroContainer
• type AvroMetric
• type Blob
  • func (x Blob) Parse(b []byte) (*skogul.Container, error)
• type DummyStore
  • func (x *DummyStore) Parse(b []byte) (*skogul.Container, error)
• type GOB
  • func (x GOB) Parse(b []byte) (*skogul.Container, error)
• type GOBMetric
  • func (x GOBMetric) ParseMetric(b []byte) (*skogul.Container, error)
• type InfluxDB
  • func (influxdb InfluxDB) Parse(bytes []byte) (*skogul.Container, error)
• type InfluxDBLineProtocol
  • func (line *InfluxDBLineProtocol) Metric() *skogul.Metric
  • func (line *InfluxDBLineProtocol) ParseLine(s string) error
• type JSON
  • func (data JSON) Parse(b []byte) (*skogul.Container, error)
• type JSONMetric
  • func (x JSONMetric) Parse(b []byte) (*skogul.Container, error)
• type MNR
  • func (mnr *MNR) Parse(bytes []byte) (*skogul.Container, error)
• type Prometheus
  • func (data Prometheus) Parse(b []byte) (*skogul.Container, error)
• type ProtoBuf
  • func (x *ProtoBuf) GetStats() *skogul.Metric
  • func (x *ProtoBuf) Parse(b []byte) (*skogul.Container, error)
• type SkogulJSON
  • func (x SkogulJSON) Parse(b []byte) (*skogul.Container, error)
• type StructuredData
  • func (sd *StructuredData) Parse(bytes []byte) (*skogul.Container, error)
• type USP_Parser
  • func (p *USP_Parser) Parse(b []byte) (*skogul.Container, error)

Variables

var Auto skogul.ModuleMap

Auto maps parser-names to parser implementation, used for auto configuration.

Types

type AVRO

type AVRO struct {
	Schema string
	// contains filtered or unexported fields
}

func (*AVRO) Parse

func (x *AVRO) Parse(b []byte) (*skogul.Container, error)

func (*AVRO) ParseMetric

func (x *AVRO) ParseMetric(m *skogul.Metric) ([]byte, error)

type AvroContainer

type AvroContainer struct {
	Template *AvroMetric
	Metrics  []*AvroMetric
}

type AvroMetric

type AvroMetric struct {
	Time     time.Time
	Metadata map[string]interface{}
	Data     map[string]interface{}
}

type Blob

type Blob struct{}

func (Blob) Parse

func (x Blob) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of arbitrary data and stores it on data["data"] unprocessed

type DummyStore

type DummyStore struct {
	File   string `doc:"File name to write data to"`
	Append bool   `doc:"Append or overwrite. By default, the file is repeatedly overwritten to capture just 1 packet"`
	// contains filtered or unexported fields
}

DummyStore is a fake parser used to capture traffic Skogul doesn't understand, mainly for future development

func (*DummyStore) Parse

func (x *DummyStore) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of arbitrary data and returns nothing.

type GOB

type GOB struct{}

func (GOB) Parse

func (x GOB) Parse(b []byte) (*skogul.Container, error)

Parser accepts the byte buffer of GOB

type GOBMetric

type GOBMetric struct{}

func (GOBMetric) ParseMetric

func (x GOBMetric) ParseMetric(b []byte) (*skogul.Container, error)

parses the bytes.buffer to skogul metrics and wraps in a container.

type InfluxDB

type InfluxDB struct{}

InfluxDB provides a byte sequence parser for the InfluxDB Line Protocol https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/

func (InfluxDB) Parse

func (influxdb InfluxDB) Parse(bytes []byte) (*skogul.Container, error)

Parse marshals a byte sequence of InfluxDB line protocol into a skogul container

type InfluxDBLineProtocol

type InfluxDBLineProtocol struct {
	// contains filtered or unexported fields
}

InfluxDBLineProtocol is a struct with the same data types as defined in the InfluxDB Line Protocol; namely the measurement name, a set of tags, a set of fields and a timestamp.

func (*InfluxDBLineProtocol) Metric

func (line *InfluxDBLineProtocol) Metric() *skogul.Metric

Metric converts an internal InfluxDBLineProtocol struct to a skogul.Metric

func (*InfluxDBLineProtocol) ParseLine

func (line *InfluxDBLineProtocol) ParseLine(s string) error

ParseLine parses a single line into an internal InfluxDBLineProtocol

type JSON

type JSON struct{}

JSON is schemaless JSON. If data is sent between Skogul instances, SkogulJSON should be used instead which retains the data structure Skogul works with. A distinction between 'Skogul' and 'JSON' is made because historically, Skogul accepted 'json' as a configuration option for its own JSON format, now named 'skogul'. JSON can be useful e.g. as the first step of parsing from a third party source where modifying the source data structure might be hard/impossible

func (JSON) Parse

func (data JSON) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of JSON data and marshals it into an empty skogul.Container

type JSONMetric

type JSONMetric struct{}

JSONMetric matches encoder's EncodeMetric - reads the byte as a metric, not container

func (JSONMetric) Parse

func (x JSONMetric) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of JSON data and marshals it into a metric, then wraps it in a container

type MNR

type MNR struct {
	ExtractFieldName bool     `doc:"Extract field name from 'name' parameter in MNR properties."`
	DefaultFieldName string   `doc:"Name of field to store variable value in, in the case that 'name' is not present."`
	ParseAsString    []string `doc:"Parse these properties as string (do not try to parse their value)."`
	StoreVariable    bool     `doc:"Store the uniquely generated variable as a data field."`
	// contains filtered or unexported fields
}

MNR supports parsing MNR data through the Parse() function

func (*MNR) Parse

func (mnr *MNR) Parse(bytes []byte) (*skogul.Container, error)

Parse converts data from MNR into a skogul Container

type Prometheus

type Prometheus struct{}

func (Prometheus) Parse

func (data Prometheus) Parse(b []byte) (*skogul.Container, error)

type ProtoBuf

type ProtoBuf struct {
	Debug bool `doc:"Logs the entire protobuf-packet if marshaling fails"`
	// contains filtered or unexported fields
}

ProtoBuf parses a byte string-representation of a Container

func (*ProtoBuf) GetStats

func (x *ProtoBuf) GetStats() *skogul.Metric

GetStats prepares a skogul metric with stats for the protobuf parser.

func (*ProtoBuf) Parse

func (x *ProtoBuf) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of protobuf data and marshals it into a container

type SkogulJSON

type SkogulJSON struct{}

SkogulJSON parses a byte string-representation of a Container in the format Skogul produces.

func (SkogulJSON) Parse

func (x SkogulJSON) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of JSON data and marshals it into a container

type StructuredData

type StructuredData struct {
	SDIDField string `doc:"The field to store the SDID in if it was present." default:"SD-ID"`
	// contains filtered or unexported fields
}

StructuredData supports parsing RFC5424 structured data through the Parse() function Note: This does not parse a full syslog message. In the resulting metrics, there may be a metadata field which contains the SD-ID from the message if it was present. By default the metadata field is named SD-ID, but it may be overridden in configuration. SD-ID: https://datatracker.ietf.org/doc/html/draft-ietf-syslog-protocol-23#section-6.3.2

func (*StructuredData) Parse

func (sd *StructuredData) Parse(bytes []byte) (*skogul.Container, error)

Parse converts RFC5424 Structured Data data into a skogul Container

type USP_Parser

type USP_Parser struct {
	// contains filtered or unexported fields
}

func (*USP_Parser) Parse

func (p *USP_Parser) Parse(b []byte) (*skogul.Container, error)

Parse accepts a byte slice of protobuf data and marshals it into a container