influxdbhelper

README

InfluxDb Helper Library

easily write and query InfluxDb from Go programs

This library allows you to encode/decode InfluxDb data to/from Go structs -- similiar to JSON and MongoDb using Go struct field tags.

See GoDoc for more documentation.

Install

(assuming you are using Go modules)

go get github.com/cbrake/influxdbhelper/v2

Note, the /v2 is required if using Go modules because the version number is 2+.

Example

package main

import (
	"log"
	"time"

	"github.com/cbrake/influxdbhelper"
	client "github.com/influxdata/influxdb/client/v2"
)

const (
	// Static connection configuration
	influxURL = "http://localhost:8086"
	db        = "dbhelper"
)

var c influxdbhelper.Client

// Init initializes the database connection
func Init() (err error) {
	c, err = influxdbhelper.NewClient(influxURL, "", "", "ns")
	if err != nil {
		return
	}
	// Create test database if it doesn't already exist
	q := client.NewQuery("CREATE DATABASE "+db, "", "")
	res, err := c.Query(q)
	if err != nil {
		return err
	}
	if res.Error() != nil {
		return res.Error()
	}
	log.Println("dbhelper db initialized")
	return nil
}

type envSample struct {
	InfluxMeasurement influxdbhelper.Measurement
	Time              time.Time `influx:"time"`
	Location          string    `influx:"location,tag"`
	Temperature       float64   `influx:"temperature"`
	Humidity          float64   `influx:"humidity"`
	ID                string    `influx:"-"`
}

// we populate a few more fields when reading back
// date to verify unused fields are handled correctly
type envSampleRead struct {
	InfluxMeasurement influxdbhelper.Measurement
	Time              time.Time `influx:"time"`
	Location          string    `influx:"location,tag"`
	City              string    `influx:"city,tag,field"`
	Temperature       float64   `influx:"temperature"`
	Humidity          float64   `influx:"humidity"`
	Cycles            float64   `influx:"cycles"`
	ID                string    `influx:"-"`
}

func generateSampleData() []envSample {
	ret := make([]envSample, 10)

	for i := range ret {
		ret[i] = envSample{
			InfluxMeasurement: "test",
			Time:              time.Now(),
			Location:          "Rm 243",
			Temperature:       70 + float64(i),
			Humidity:          60 - float64(i),
			ID:                "12432as32",
		}
	}

	return ret
}

func main() {
	err := Init()
	if err != nil {
		log.Fatal("Failed to initialize db")
	}

	// write sample data to database
	samples := generateSampleData()
	c = c.UseDB(db)
	for _, p := range samples {
		err := c.WritePoint(p)
		if err != nil {
			log.Fatal("Error writing point: ", err)
		}
	}

	// query data from db
	samplesRead := []envSampleRead{}

	q := `SELECT * FROM test ORDER BY time DESC LIMIT 10`
	err = c.UseDB(db).DecodeQuery(q, &samplesRead)
	if err != nil {
		log.Fatal("Query error: ", err)
	}
	log.Printf("Samples read: %+v\n", samplesRead)
}

Details

There are several advantages decoding and encoding data directly from Go Structs:

1. The database bschema is documented by the Go type definition. This helps ensure data is written consistently to the database. When all your data is clearly defined in Go structs, it is much more obvious how to organize it, what goes in what measurement, when to create a new measurement, etc. When writing straight tags/values, it is much easier to create a disorganized mess.
2. All the code for decoding and encoding the various data types supported by InfluxDb are handled in one place, rather than repeating this logic over and over for every Query.
3. Likewise, code for handling arrays (translating Go array to InfluxDb fields like temp0, temp1, temp2, ...) can be in one place.
4. Reading and Writing data is much simpler and requires way less code.

Using Go reflection to automate data decode may be slightly slower than custom decode logic for every query, but it seems the time to decode the data will relatively fast compared to the time to run a InfluxDb query, so may be negligable (this is an assumption at this point and has not been proven).

The decode_test.go file contains a number of tests that illustrate the conversion from influx JSON to Go struct values.

Acknowledgments

The mapstructure library provided a very useful reference for learning how to use the Go reflect functionality.

Status

Todo:

• handle larger query datasets (multiple series, etc)
• add write capability (directly write Go structs into influxdb)
• add godoc documentation
• get working with influxdb 1.7 client
• see if still applicable for influxdb 2.x
• decode/encode val0, val1, val2 fields in influx to Go array
• use Go struct field tags to help build SELECT statement
• optimize query for performace (pre-allocate slices, etc)
• come up with a better name (indecode, etc)
• finish error checking

Review/Pull requests welcome!

License

MIT

Documentation

Overview

Package influxdbhelper allows you to encode/decode InfluxDb data to/from Go structs -- similiar to JSON and MongoDb using Go struct field tags.

Index

• func CleanQuery(query string) string
• type Client
  • func NewClient(url, user, passwd, precision string) (Client, error)
• type Error
  • func (e *Error) Error() string
  • func (e *Error) WrappedErrors() []error
• type Measurement

Functions

func CleanQuery

func CleanQuery(query string) string

CleanQuery can be used to strip a query string of newline characters. Typically only used for debugging.

Types

type Client

type Client interface {
	influxClient.Client

	// UseDB sets the DB to use for Query, WritePoint, and WritePointTagsFields.
	// This field must be set before WritePoint... calls.
	UseDB(db string) Client

	// UseMeasurement sets the measurement to use for WritePoint, and WritePointTagsFields.
	// If this is not set, a struct field with named InfluxMeasurement is required
	// in the write data. The data passed in this call has priority over data fields in
	// writes.
	UseMeasurement(measurement string) Client

	// UseTimeField sets the time field to use for WritePoint, and WritePointTagsFields. This
	// call is optional, and a data struct field with a `influx:"time"` tag can also be used.
	UseTimeField(fieldName string) Client

	// Query executes an InfluxDb query, and unpacks the result into the
	// result data structure.
	DecodeQuery(query string, result interface{}) error

	// WritePoint is used to write arbitrary data into InfluxDb.
	WritePoint(data interface{}) error

	// WritePointTagsFields is used to write a point specifying tags and fields.
	WritePointTagsFields(tags map[string]string, fields map[string]interface{}, t time.Time) error
}

A Client represents an influxdbhelper influxClient connection to an InfluxDb server.

func NewClient

func NewClient(url, user, passwd, precision string) (Client, error)

NewClient returns a new influxdbhelper influxClient given a url, user, password, and precision strings.

url is typically something like: http://localhost:8086

precision can be ‘h’, ‘m’, ‘s’, ‘ms’, ‘u’, or ‘ns’ and is used during write operations.

type Error

type Error struct {
	Errors []string
}

Error implements the error interface and can represents multiple errors that occur in the course of a single decode.

func (*Error) Error

func (e *Error) Error() string

func (*Error) WrappedErrors

func (e *Error) WrappedErrors() []error

WrappedErrors implements the errwrap.Wrapper interface to make this return value more useful with the errwrap and go-multierror libraries.

type Measurement

type Measurement = string

Measurement is a type that defines the influx db measurement.