tine

README

TINE

TINE a data pipeline runner.

Install

go install github.com/OutOfBedlam/tine@latest

Find documents from https://tine.thingsme.xyz/

Usage

Define pipeline in TOML

Set the pipeline's inputs and outputs.

[[inlets.cpu]]
    interval = "3s"
[[flows.select]]
    includes = ["#*", "*"]  # all tags and all fields
[[outlets.file]]
    path  = "-"
    decimal = 2

Run

tine run <config.toml>

It generates CPU usage in CSV format which is default format of 'outlets.file'.

1721635296,cpu,1.57
1721635299,cpu,0.66
1721635302,cpu,1.07
1721635305,cpu,2.08

Change output format to "json" from "csv", add format = "json" at the end of the file.

[[outlets.file]]
    path  = "-"
    decimal = 2
    format = "json"

{"_in":"cpu","_ts":1721780188,"total_percent":0.91}
{"_in":"cpu","_ts":1721780191,"total_percent":1.04}
{"_in":"cpu","_ts":1721780194,"total_percent":0.25}
{"_in":"cpu","_ts":1721780197,"total_percent":1.20}

Shebang

1. Save this file as load.toml

#!/path/to/tine run
[[inlets.load]]
    loads = [1, 5]
    interval = "3s"
[[flows.select]]
    includes = ["**"]  # equivalent to ["#*", "*"]
[[outlets.file]]
    path  = "-"
    decimal = 2

1. Chmod for executable.

chmod +x load.toml

1. Run

$ ./load.toml

1721635438,load,0.03,0.08
1721635441,load,0.03,0.08
1721635444,load,0.03,0.08
^C

Embedding in Go

TINE is not only designed as a standalone application, but also allows for embedding in other Go applications.

Examples

How to use TINE as a library for your application.

• helloworld

How to set a custom inlet/outlet/flows.

• custom_out
• custom_in
• custom_flow
• custom_out_reg
• custom_in_reg
• custom_flow_reg

How to use pipelines as a HTTP handler

• httpsvr

How to collect metrics into RRD and display rrdgraph in a web page

• rrd_graph_web

How to collect metrics into Sqlite and display it on web page

This example also shows how to utilize HTTP query parameters as variables of Go Templates to build pipeline configuration.

• sqlite_graph_web

Documents

Please visit https://tine.thingsme.xyz for the documents.