README
¶
Clymene
The Clymene is a time series data collection platform for distributed systems inspired by Prometheus and Jaeger. Time series data from various environments can be collected and stored in different types of databases. It can be configured in a variety of architectures. Also, Clymene Agent's time series data collection uses fewer resources than Prometheus' remote_write. Various time series data analysis functions will be added.
Architecture
Use only agent
Agent can store time series data in a variety of DBs and support Composite Writers that can be stored in multiple DBs at once.
Including kafka and ingester
Ingester consumes time series data loaded on kafka, can store time series data in various DBs, and supports Composite Writers that can be stored in multiple DBs at once.
Use gateway
Gateway collects time series data through gRPC communication, stores time series data in various DBs, and supports Composite Writers that can be stored in multiple DBs at once.
components
Clymene Agent(Getting Started)
dockerhub : bourbonkk/clymene-agent:v1.x.x
redhatQuay: quay.io/clymene/clymene-agent:v1.x.x
The Clymene-agent is service that collects time series data(does not use disks)
- Service Discovery
- Prometheus's Service Discovery feature finds Metric collection endpoints.
- scrape time series data
- Time-series data transfer to gateway(gRPC) (Optional)
- Time-series data transfer to kafka (Optional)
- Time-series data insert to Database(supported DB) (Optional)
Clymene Ingester(Optional) (Getting Started)
dockerhub : bourbonkk/clymene-ingester:v1.x.x
redhatQuay: quay.io/clymene/clymene-ingester:v1.x.x
The Clymene ingester is an optional service responsible for insert time series data loaded on kafka into the database.
- Kafka message consume
- Time-series data insert to Database(supported DB) (Optional)
Clymene Gateway(Optional) (Getting Started)
dockerhub : bourbonkk/clymene-gateway:v1.x.x
redhatQuay: quay.io/clymene/clymene-gateway:v1.x.x
The Clymene Gateway is an optional service that can receive metric data from the another component through gRPC
communication.
- gRPC Service
- Time-series data insert to Database(supported DB) (Optional)
How to work it on k8s
For example, see the clymene_using_gateway.yaml
How to work it on docker-compose
For example, see the docker-compose-clymene_using_gateway.yml
Roadmap
- Various DB support(ElasticSearch, Prometheus, Cortex, OpenTSDB, influxdb, TDengine, kdb)
- AI/ML platform for clymene(https://github.com/Clymene-project/clymene-analyzer)
Get in Touch
Have questions, suggestions, bug reports? Reach the project community via these channels:
- GitHub issues and discussions
Further reading
To learn more about Clymene, consult the following talks and articles.
Recent talks and articles
- Jan 2022 blog post "Efficient time series data collection and management plan in a distributed environment"
License
Apache License 2.0, see LICENSE.
Code Style
1. https://github.com/golang/go/wiki/CodeReviewComments
2. https://peter.bourgon.org/go-in-production/#formatting-and-style
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
agent/app/discovery/install
Package install has the side-effect of registering all builtin service discovery config types.
|
Package install has the side-effect of registering all builtin service discovery config types. |
|
pkg
|
|
|
metrics
Package metrics provides command line flags for configuring the metrics backend.
|
Package metrics provides command line flags for configuring the metrics backend. |
|
Package plugin is the collection of implementations of different interfaces defined across Clymene
|
Package plugin is the collection of implementations of different interfaces defined across Clymene |
|
Package prompb is a reverse proxy.
|
Package prompb is a reverse proxy. |
|
util
|
|
|
promlint
Package promlint provides a linter for Prometheus metrics.
|
Package promlint provides a linter for Prometheus metrics. |