ingest

Documentation

Overview

Package ingest provides the datatypes and functions helping with ingesting data into Axiom.

Index

• Constants
• type Failure
• type Option
  • func SetCSVDelimiter(delim string) Option
  • func SetTimestampField(field string) Option
  • func SetTimestampFormat(format string) Option
• type Options
• type Status
  • func (s *Status) Add(other *Status)

Constants

const TimestampField = "_time"

TimestampField is the default field the server will look for a timestamp to use as the ingestion time. If not present, the server will set the ingestion time to the current server time.

Types

type Failure

type Failure struct {
	// Timestamp of the event that failed to ingest.
	Timestamp time.Time `json:"timestamp"`
	// Error that made the event fail to ingest.
	Error string `json:"error"`
}

Failure describes the ingestion failure of a single event.

type Option

type Option func(*Options)

An Option applies optional parameters to an ingest operation.

func SetCSVDelimiter

func SetCSVDelimiter(delim string) Option

SetCSVDelimiter specifies the delimiter that separates CSV fields. Only valid when the content to be ingested is CSV formatted.

func SetTimestampField

func SetTimestampField(field string) Option

SetTimestampField specifies the field Axiom will use to extract the events time from. Defaults to TimestampField

func SetTimestampFormat

func SetTimestampFormat(format string) Option

SetTimestampFormat specifies the format of the timestamp field. The reference time is "Mon Jan 2 15:04:05 -0700 MST 2006", as specified in https://pkg.go.dev/time/?tab=doc#Parse.

type Options

type Options struct {
	// TimestampField defines a custom field to extract the ingestion timestamp
	// from. Defaults to [TimestampField].
	TimestampField string `url:"timestamp-field,omitempty"`
	// TimestampFormat defines a custom format for the [Options.TimestampField].
	// The reference time is "Mon Jan 2 15:04:05 -0700 MST 2006", as specified
	// in https://pkg.go.dev/time/?tab=doc#Parse.
	TimestampFormat string `url:"timestamp-format,omitempty"`
	// CSVDelimiter is the delimiter that separates CSV fields. Only valid when
	// the content to be ingested is CSV formatted.
	CSVDelimiter string `url:"csv-delimiter,omitempty"`
}

Options specifies the optional parameters for ingestion.

type Status

type Status struct {
	// Ingested is the amount of events that have been ingested.
	Ingested uint64 `json:"ingested"`
	// Failed is the amount of events that failed to ingest.
	Failed uint64 `json:"failed"`
	// Failures are the ingestion failures, if any.
	Failures []*Failure `json:"failures"`
	// ProcessedBytes is the number of bytes processed.
	ProcessedBytes uint64 `json:"processedBytes"`
	// BlocksCreated is the amount of blocks created.
	//
	// Deprecated: BlocksCreated is deprecated and will be removed in a future
	// release.
	BlocksCreated uint32 `json:"blocksCreated"`
	// WALLength is the length of the Write-Ahead Log.
	//
	// Deprecated: WALLength is deprecated and will be removed in a future
	// release.
	WALLength uint32 `json:"walLength"`
}

Status is the status of an event ingestion operation.

func (*Status) Add

func (s *Status) Add(other *Status)

Add adds the status of another ingestion operation to the current status.