exports

Documentation

Index

• func CSVColumnStreamer(defs map[string]EventColumnDef, records <-chan mixpanel.EventData)
• func CSVStreamer(w io.Writer, records <-chan mixpanel.EventData)
• func JSONStreamer(w io.Writer, records <-chan mixpanel.EventData)
• type EventColumnDef
  • func NewEventColumnDef(w io.Writer, columns []string) EventColumnDef

Functions

func CSVColumnStreamer

func CSVColumnStreamer(defs map[string]EventColumnDef, records <-chan mixpanel.EventData)

CSVColumnStreamer writes CSVs with explicitly defined events and properties. This is useful if only a subset of the properties attached to an event type are useful or the data needs to be stored in a traditional SQL table with columns known ahead of time.

This will write to a unique io.Writer for each specified event.

The `defs` map contains a mapping of the event names to capture to their EventColumnDefs. Any event received that is not in this map will simply be dropped.

func CSVStreamer

func CSVStreamer(w io.Writer, records <-chan mixpanel.EventData)

CSVStreamer writes the records passed on the given chan in a schema-less way. An initial header row containing the names of the columns is written first.

Format is:

event_id,key,value

This way, it is possible to GROUP BY event_id to get the full view of a single event.

The reason for this format is because it is not possible to know all of the column names beforehand, and making multiple passes over the data to find a common set of columns is a nonstarter because of the time and memory requirements this requires.

func JSONStreamer

func JSONStreamer(w io.Writer, records <-chan mixpanel.EventData)

JSONStreamer writes records to an io.Writer in JSON format line by line, simply serializing the JSON directly.

Format is simply: `{"key": "value", ...}`. `value` is usually scalar, but can be any valid JSON type.

Types

type EventColumnDef

type EventColumnDef struct {
	// contains filtered or unexported fields
}

EventColumnDef represents the definition of an event's CSV columns to be passed on to the `CSVColumnStreamer` function.

• `columns` contains the names of the columns.
• `values` represents a row, in the same order as specified by `columns`. This is to avoid creating excessive garbage by allocating and destroying the array on each iteration.

func NewEventColumnDef

func NewEventColumnDef(w io.Writer, columns []string) EventColumnDef

NewEventColumnDef oddly enough creates an instance of the EventColumnDef struct from the given io.Writer and list of column names.

Columns in the output will be in the same order as they passed in here.