openGemini

README

README

English | 简体中文

Official Website | Docs | Contribution Guide | RoadMap | Performance

About openGemini

openGemini is a cloud-native distributed time series database, focusing on the storage and analysis of massive telemetry data.

If you have used openGemini, please let us know and click submit Usage Information

openGemini is an sandbox project of the Cloud Native Computing Foundation (CNCF). Please consider joining the CNCF if you are an organization that wants to take an active role in supporting the growth and evolution of the cloud native ecosystem.

Why openGemini

Compared with other time series database, openGemini has the following advantages:

• High Performance

  Store and query data quickly and efficiently with automatic partitioning, LSM-based storage techniques, and better data process engineering, see performance reports

• High Scalability

  Adopting MPP architecture, support distributed cluster deployment and can be flexibly expanded as business grows to meet higher load requirements, see Architecture

• High Cardinality

  A new high cardinality storage engine solves problems such as excessive index memory usage and low read and write performance, see high cardinality storage engine

• Data Compression

  Data is stored in column format, and different data types use dedicated data compression algorithms. The data compression ratio is as high as 15:1 or higher, see Data compression algorithm

• Flexible Deployment

  Supports stand-alone and cluster deployment, and supports various forms of deployment such as virtual machines, physical machines, containers, and clouds. Provides one-click container deployment tool (openGemini-operator) and one-click normal installation and deployment tool (gemix)

• Ecosystem compatibility

  Compatible with InfluxDB v1.x Line Protocol, Influxql and read/write APIs, also compatible with existing InfluxDB toolchain

Support for Cloud Native Scenarios

• Containerized deployment, refer to openGemini-operator

  

• openTelemetry backend storage, simplified architecture

  

• Prometheus backend storage，support remote read/write, and Support PromQL

  

• KubeEdge integration with openGemini, better manage edge device data

  

Application Scenes

Quick Start

For a more detailed introduction, please visit our official website User Guide

This section mainly contains the following:

• How to compile openGemini source code
• How to run openGemini

Compiling environment information

GO version v1.20+

Python version v3.7+

How to set GO environment variables

Open ~/.profile configuration file and add the following configurations to the end of the file:

export GOPATH=/path/to/dir
export GOBIN=$GOPATH/bin
export GO111MODULE=on
export GONOSUMDB=*
export GOSUMDB=off

Compiling

1. Clone source codes from Github

> cd $GOPATH
> mkdir -p {pkg,bin,src}
> cd src
> git clone https://github.com/openGemini/openGemini.git

1. Enter the home directory

> cd openGemini

1. Compiling

> export CGO_LDFLAGS="-Wl,-z,now -Wl,-z,relro -Wl,-z,noexecstack -fPIE -ftrapv"
> export CGO_CFLAGS="-fstack-protector-strong -D_FORTIFY_SOURCE=2 -O2"
> python build.py

The compiled binary file is in the build directory

> ls build
ts-cli ts-meta ts-monitor ts-server  ts-sql  ts-store

Configuration

The configuration file is in the conf directory. For more information about the configuration items, please refer to User Guide --> Reference --> Configuration Files

Run openGemini

Standalone operation

> cd openGemini
> mkdir -p /tmp/openGemini
> sh scripts/install.sh

Refer to cluster deployments in User Guide

Using openGemini

Use the client to connect to openGemini

> ts-cli --host 127.0.0.1 --port 8086

After successful login, the following message will be displayed

> ts-cli --host 127.0.0.1 --port 8086
openGemini CLI 0.1.0 (rev-revision)
Please use 'quit', 'exit' or 'Ctrl-D' to exit this program
>

Create a database

> create database sensordb
> use sensordb

The openGemini supports three ways for creating a measurement.

• Implicit creation, automatic creation measurement when data is written. By default, partition is based on time.
• explicit creation, without specifying the partition key, as the same as implicit creation.

> create measurement sensor

• explicit creation, specifying partition keys during table creation, During data storage, the table will be primarily partitioned by time, and then secondarily partitioned according to the specified partition keys.

> create measurement sensor with shardkey farmID

Write data

insert sensor,farmID=f1,deviceID=d0 sensorID="s20",value=50.98

Query data

> select * from sensor
name: sensor
+---------------------+----------+--------+----------+-------+
| time                | deviceID | farmID | sensorID | value |
+---------------------+----------+--------+----------+-------+
| 1657959880895515464 | d0       | f1     | s20      | 50.98 |
+---------------------+----------+--------+----------+-------+
5 columns,1 rows in set
Elapsed: 7.723332ms

Join & Contribute

Tips for Contribution

Code of Conduct

openGemini follows the CNCF Code of Conduct.

Contact Us

1. Slack

2. Twitter

3. Email

4. mailing list

License

openGemini is licensed under the Apache License 2.0. Refer to LICENSE for more details.

For third-party software usage notice, see Open_Source_Software_Notice