greptime

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2024 License: Apache-2.0 Imports: 13 Imported by: 0

README

Build Status codecov Go Reference

GreptimeDB Go Ingester

Provide API to insert data into GreptimeDB.

How To Use

Installation
go get -u github.com/GreptimeTeam/greptimedb-ingester-go
Import
import greptime "github.com/GreptimeTeam/greptimedb-ingester-go"
Config

Initiate a Config for Client

cfg := greptime.NewConfig("<host>").
    WithPort(4001).
    WithAuth("<username>", "<password>").
    WithDatabase("<database>")
Options
Secure
cfg.WithInsecure(false) // default insecure=true
keepalive
cfg.WithKeepalive(time.Second*30, time.Second*5) // keepalive isn't enabled by default
Client
cli, err := greptime.NewClient(cfg)
Insert & StreamInsert

  • you can Insert data into GreptimeDB via different style:

  • streaming insert is to Send data into GreptimeDB without waiting for response.

Table style

you can define schema via Table and Column, and then AddRow to include the real data you want to write.

define table schema, and add rows
import(
    "github.com/GreptimeTeam/greptimedb-ingester-go/table"
    "github.com/GreptimeTeam/greptimedb-ingester-go/table/types"
)

tbl, err := table.New("<table_name>")

tbl.AddTagColumn("id", types.INT64)
tbl.AddFieldColumn("host", types.STRING)
tbl.AddTimestampColumn("ts", types.TIMESTAMP_MILLISECOND)

err := tbl.AddRow(1, "127.0.0.1", time.Now())
err := tbl.AddRow(2, "127.0.0.2", time.Now())
...
Write into GreptimeDB
resp, err := cli.Write(context.Background(), tbl)
Delete from GreptimeDB
dtbl, err := table.New("<table_name>")
dtbl.AddTagColumn("id", types.INT64)
dtbl.AddTimestampColumn("ts", types.TIMESTAMP_MILLISECOND)

// timestamp is the time you want to delete row
err := dtbl.AddRow(1, "127.0.0.1",timestamp)

affected, err := cli.Delete(context.Background(),dtbl)
Stream Write into GreptimeDB
err := cli.StreamWrite(context.Background(), tbl)
...
affected, err := cli.CloseStream(ctx)
Stream Delete from GreptimeDB
err := cli.StreamDelete(context.Background(), tbl)
...
affected, err := cli.CloseStream(ctx)
ORM style

If you prefer ORM style, and define column-field relationship via struct field tag, you can try the following way.

Tag
  • greptime is the struct tag key
  • tag, field, timestamp is for SemanticType, and the value is ignored
  • column is to define the column name
  • type is to define the data type. if type is timestamp, precision is supported
  • the metadata separator is ; and the key value separator is :

type supported is the same as described Datatypes supported, and case insensitive.

When fields marked with greptime:"-", writing field will be ignored.

define struct with tags
type Monitor struct {
    ID          int64     `greptime:"tag;column:id;type:int64"`
    Host        string    `greptime:"field;column:host;type:string"`
    Ts          time.Time `greptime:"timestamp;column:ts;type:timestamp;precision:millisecond"`
}

// TableName is to define the table name.
func (Monitor) TableName() string {
    return "<table_name>"
}
instance your struct
monitors := []Monitor{
    {
        ID:          randomId(),
        Host:        "127.0.0.1",
        Running:     true,
    },
    {
        ID:          randomId(),
        Host:        "127.0.0.2",
        Running:     true,
    },
}
WriteObject into GreptimeDB
resp, err := cli.WriteObject(context.Background(), monitors)
DeleteObject in GreptimeDB
deleteMonitors := monitors[:1]

affected, err := cli.DeleteObject(context.Background(), deleteMonitors)
Stream WriteObject into GreptimeDB
err := cli.StreamWriteObject(context.Background(), monitors)
...
affected, err := cli.CloseStream(ctx)
Stream DeleteObject in GreptimeDB
deleteMonitors := monitors[:1]

err := cli.StreamDeleteObject(context.Background(), deleteMonitors)
...
affected, err := cli.CloseStream(ctx)

Datatypes supported

The GreptimeDB column is for the datatypes supported in library, and the Go column is the matched Go type.

GreptimeDB Go Description
INT8 int8
INT16 int16
INT32 int32
INT64, INT int64
UINT8 uint8
UINT16 uint16
UINT32 uint32
UINT64, UINT uint64
FLOAT32 float32
FLOAT64, FLOAT float64
BOOLEAN, BOOL bool
STRING string
BINARY, BYTES []byte
DATE Int or time.Time the day elapsed since 1970-1-1
DATETIME Int or time.Time the millisecond elapsed since 1970-1-1
TIMESTAMP_SECOND Int or time.Time
TIMESTAMP_MILLISECOND, TIMESTAMP Int or time.Time
TIMESTAMP_MICROSECOND Int or time.Time
TIMESTAMP_NANOSECOND Int or time.Time
JSON string

NOTE: Int is for all of Integer and Unsigned Integer in Go

Query

You can use ORM library like gorm with MySQL or PostgreSQL driver to connect GreptimeDB and retrieve data from it.

type Monitor struct {
    ID          int64     `gorm:"primaryKey;column:id"`
    Host        string    `gorm:"column:host"`
    Ts          time.Time `gorm:"column:ts"`
}

// Get all monitors
var monitors []Monitor
result := db.Find(&monitors)

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client added in v0.2.0 

type Client struct {
	// contains filtered or unexported fields
}

Client helps to write data into GreptimeDB. A Client is safe for concurrent use by multiple goroutines,you can have one Client instance in your application.

func NewClient added in v0.2.0 

func NewClient(cfg *Config) (*Client, error)

NewClient helps to create the greptimedb client, which will be responsible write data into GreptimeDB.

func (*Client) CloseStream added in v0.2.0 

func (c *Client) CloseStream(ctx context.Context) (*gpb.AffectedRows, error)

CloseStream closes the stream. Once we’ve finished writing our client’s requests to the stream using client.StreamWrite or client.StreamWriteObject, we need to call client.CloseStream to let GreptimeDB know that we’ve finished writing and are expecting to receive a response.

func (*Client) Delete added in v0.5.0 

func (c *Client) Delete(ctx context.Context, tables ...*table.Table) (*gpb.GreptimeResponse, error)

Delete is to delete the data from GreptimeDB via explicit schema. 

tbl, err := table.New(<tableName>)

// add column at first. This is to define the schema of the table.
tbl.AddTagColumn("tag1", types.INT64)
tbl.AddTimestampColumn("timestamp", types.TIMESTAMP_MILLISECOND)

// you can add multiple row(s). This is the real data.
tbl.AddRow("tag1", timestamp)

// delete the data from GreptimeDB
resp, err := client.Delete(context.Background() tbl)

func (*Client) DeleteObject added in v0.5.0 

func (c *Client) DeleteObject(ctx context.Context, obj any) (*gpb.GreptimeResponse, error)

DeleteObject is like [Delete] to delete the data from GreptimeDB, but schema is defined in the struct tag. resp, err := client.DeleteObject(context.Background(), deleteMonitors)

func (*Client) HealthCheck added in v0.5.2 

func (c *Client) HealthCheck(ctx context.Context) (*gpb.HealthCheckResponse, error)

HealthCheck will check GreptimeDB health status.

func (*Client) StreamDelete added in v0.5.0 

func (c *Client) StreamDelete(ctx context.Context, tables ...*table.Table) error

StreamDelete is to delete the data from GreptimeDB via explicit schema. 

tbl, err := table.New(<tableName>)

// add column at first. This is to define the schema of the table.
tbl.AddTagColumn("tag1", types.INT64)
tbl.AddTimestampColumn("timestamp", types.TIMESTAMP_MILLISECOND)

// you can add multiple row(s). This is the real data.
tbl.AddRow("tag1", timestamp)

// delete the data from GreptimeDB
resp, err := client.StreamDelete(context.Background(), tbl)

func (*Client) StreamDeleteObject added in v0.5.0 

func (c *Client) StreamDeleteObject(ctx context.Context, body any) error

StreamDeleteObject is like [StreamDelete] to Delete the data from GreptimeDB, but schema is defined in the struct tag. resp, err := client.StreamDeleteObject(context.Background(), deleteMonitors)

func (*Client) StreamWrite added in v0.2.0 

func (c *Client) StreamWrite(ctx context.Context, tables ...*table.Table) error

StreamWrite is to send the data into GreptimeDB via explicit schema. 

tbl, err := table.New(<tableName>)

// add column at first. This is to define the schema of the table.
tbl.AddTagColumn("tag1", types.INT64)
tbl.AddFieldColumn("field1", types.STRING)
tbl.AddFieldColumn("field2", types.FLOAT64)
tbl.AddTimestampColumn("timestamp", types.TIMESTAMP_MILLISECOND)
timestamp = time.Now()
// you can add multiple row(s). This is the real data.
tbl.AddRow(1, "hello", 1.1, timestamp)

// send data into GreptimeDB
resp, err := client.StreamWrite(context.Background(), tbl)

func (*Client) StreamWriteObject added in v0.3.0 

func (c *Client) StreamWriteObject(ctx context.Context, body any) error

StreamWriteObject is like [StreamWrite] to send the data into GreptimeDB, but schema is defined in the struct tag. 

type monitor struct {
  ID          int64     `greptime:"tag;column:id;type:int64"`
  Host        string    `greptime:"tag;column:host;type:string"`
  Memory      uint64    `greptime:"field;column:memory;type:uint64"`
  Cpu         float64   `greptime:"field;column:cpu;type:float64"`
  Temperature int64     `greptime:"field;column:temperature;type:int64"`
  Running     bool      `greptime:"field;column:running;type:boolean"`
  Ts          time.Time `greptime:"timestamp;column:ts;type:timestamp;precision:millisecond"`
}

func (monitor) TableName() string {
  return monitorTableName
}

monitors := []monitor{
	{
	    ID:          randomId(),
	    Host:        "127.0.0.1",
	    Memory:      1,
	    Cpu:         1.0,
	    Temperature: -1,
	    Ts:          time1,
	    Running:     true,
	},
	{
	    ID:          randomId(),
	    Host:        "127.0.0.2",
	    Memory:      2,
	    Cpu:         2.0,
	    Temperature: -2,
	    Ts:          time2,
	    Running:     true,
	},
}

resp, err := client.StreamWriteObject(context.Background(), monitors)

func (*Client) Write added in v0.2.0 

func (c *Client) Write(ctx context.Context, tables ...*table.Table) (*gpb.GreptimeResponse, error)

Write is to write the data into GreptimeDB via explicit schema. 

tbl, err := table.New(<tableName>)

// add column at first. This is to define the schema of the table.
tbl.AddTagColumn("tag1", types.INT64)
tbl.AddFieldColumn("field1", types.STRING)
tbl.AddFieldColumn("field2", types.FLOAT64)
tbl.AddTimestampColumn("timestamp", types.TIMESTAMP_MILLISECOND)
timestamp := time.Now()
// you can add multiple row(s). This is the real data.
tbl.AddRow(1, "hello", 1.1, timestamp)

// write data into GreptimeDB
resp, err := client.Write(context.Background(), tbl)

func (*Client) WriteObject added in v0.3.0 

func (c *Client) WriteObject(ctx context.Context, obj any) (*gpb.GreptimeResponse, error)

WriteObject is like [Write] to write the data into GreptimeDB, but schema is defined in the struct tag. 

type Monitor struct {
  ID          int64     `greptime:"tag;column:id;type:int64"`
  Host        string    `greptime:"tag;column:host;type:string"`
  Memory      uint64    `greptime:"field;column:memory;type:uint64"`
  Cpu         float64   `greptime:"field;column:cpu;type:float64"`
  Temperature int64     `greptime:"field;column:temperature;type:int64"`
  Running     bool      `greptime:"field;column:running;type:boolean"`
  Ts          time.Time `greptime:"timestamp;column:ts;type:timestamp;precision:millisecond"`
}

func (Monitor) TableName() string {
  return monitorTableName
}

monitors := []Monitor{
	{
	    ID:          randomId(),
	    Host:        "127.0.0.1",
	    Memory:      1,
	    Cpu:         1.0,
	    Temperature: -1,
	    Ts:          time1,
	    Running:     true,
	},
	{
	    ID:          randomId(),
	    Host:        "127.0.0.2",
	    Memory:      2,
	    Cpu:         2.0,
	    Temperature: -2,
	    Ts:          time2,
	    Running:     true,
	},
}

resp, err := client.WriteObject(context.Background(), monitors)

type Config added in v0.2.0 

type Config struct {
	Host     string // no scheme or port included. example: 127.0.0.1
	Port     int    // default: 4001
	Username string
	Password string
	Database string // the default database
	// contains filtered or unexported fields
}

Config is to define how the Client behaves.

  • Host is 127.0.0.1 in local environment.
  • Port default value is 4001.
  • Username and Password can be left to empty in local environment. you can find them in GreptimeCloud service detail page.
  • Database is the default database the client will operate on. But you can change the database in InsertRequest or QueryRequest.

func NewConfig added in v0.2.0 

func NewConfig(host string) *Config

NewConfig helps to init Config with host only

func (*Config) WithAuth added in v0.2.0 

func (c *Config) WithAuth(username, password string) *Config

WithAuth helps to specify the Basic Auth username and password. Leave them empty if you are in local environment.

func (*Config) WithDatabase added in v0.2.0 

func (c *Config) WithDatabase(database string) *Config

WithDatabase helps to specify the default database the client operates on.

func (*Config) WithDialOption added in v0.4.0 

func (c *Config) WithDialOption(opt grpc.DialOption) *Config

WithDialOption helps to specify the dial option which has not been supported by ingester sdk yet.

func (*Config) WithInsecure added in v0.4.0 

func (c *Config) WithInsecure(insecure bool) *Config

TODO(yuanbohan): support more tls options

func (*Config) WithKeepalive added in v0.2.0 

func (c *Config) WithKeepalive(time, timeout time.Duration) *Config

WithKeepalive helps to set the keepalive option.

  • time. After a duration of this time if the client doesn't see any activity it pings the server to see if the transport is still alive. If set below 10s, a minimum value of 10s will be used instead.
  • timeout. After having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed.

func (*Config) WithMeterProvider added in v0.5.3 

func (c *Config) WithMeterProvider(p metric.MeterProvider) *Config

WithMeterProvider provides a MeterProvider for SDK. If metrics colleciton is not enabled, then this option has no effect. If metrics colleciton is enabled and this option is not provide. the global MeterProvider will be used.

func (*Config) WithMetricsEnabled added in v0.5.3 

func (c *Config) WithMetricsEnabled(b bool) *Config

WithMetricsEnabled enables/disables collection of SDK's metrics. Disabled by default.

func (*Config) WithPort added in v0.2.0 

func (c *Config) WithPort(port int) *Config

WithPort set the Port field. Do not change it if you have no idea what it is.

func (*Config) WithTraceProvider added in v0.5.3 

func (c *Config) WithTraceProvider(p trace.TracerProvider) *Config

WithTraceProvider provides a TracerProvider for SDK. If traces colleciton is not enabled, then this option has no effect. If traces colleciton is enabled and this option is not provide. the global MeterProvider will be used.

func (*Config) WithTracesEnabled added in v0.5.3 

func (c *Config) WithTracesEnabled(b bool) *Config

WithTracesEnabled enables/disables collection of SDK's traces. Disabled by default.

Source Files

View all Source files

Directories

Path Synopsis
examples
healthcheck
hints
jsondata
object
table
header
cell
types

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.