otc-prometheus-exporter

README

OTC Prometheus Exporter

This software gathers metrics from the Open Telekom Cloud (OTC) for Prometheus. The metrics are then usable in any service which can use Prometheus as a datasource. For example Grafana

Available Metrics

Metrics for the following services are available

• Elastic Cloud Server (ECS)
• Virtual Private Cloud (VPC)
• NAT Gateway (NAT)
• Elastic Load Balancing (ELB)
• Distributed Message Service (DMS)
• Special Case: Object Storage Service (OBS)

Special Case: Object Storage Service (OBS)

The Object Storage Service (OBS) metrics have some unique considerations:

1. Global Project ID Requirement: Unlike other services OBS is a global service. This means you need a global project ID to gather OBS metrics. The supported projects are eu_de and eu_nl.
2. Limitation on Project Scoped Metrics: OBS metrics cannot be collected with project scoped metrics since a global project ID is needed, which transcends individual project scopes.

Requirements

• OTC-Credentials

Usage & Configuration

In this section you will learn how to use and configure this software. The configuration happens via environment variables and one configuration file.

1. Obtain the necessary credentials for the OTC. You need a username, password, project id and domain name.

2. Set the desired namespaces as a list of comma seperated values in the environment variable NAMESPACES.

3. The other environment variables are not required. The following table covers all environment variables.

environment variable	default value	allowed values	description
OS_USERNAME	none	valid username	REQUIRED User in the OTC with access to the API
OS_PASSWORD	none	valid password	REQUIRED Password for the user
OS_ACCESS_KEY	none	valid access key	REQUIRED You can instead of username/password also provide the users AK and SK
OS_SECRET_KEY	none	valid secret key	REQUIRED You can instead of username/password also provide the users AK and SK
OS_PROJECT_ID	none	valid project id	REQUIRED Project from which the metrics should be gathered. Obtainable in the OTC console IAM -> Projects -> View your Project -> You can now see the ProjectID
OS_DOMAIN_NAME	none	valid domain name	REQUIRED Domainname/Tenant ID. The value in the OTC console on the top right OTC-EU-DE-{somenumberhere}.
NAMESPACES	none	e.g. SYS.ECS,SERVICE.BMS,ECS,BMS	REQUIRED Specific namespaces for instances you want to get the metrics from. See list of all namespaces in the CloudEye docs CloudEyeDoc. If namespace is available in CloudEye docs you can also use namespace without prefix (SYS.ECS -> ECS, SERVICE.BMS -> BMS).
REGION	eu-de	eu-de, eu-nl	Region where your project is located
PORT	39100	any valid unused port	Port on which metrics are served
WAITDURATION	60	any positive integer	Time in seconds between two API call fetches
FETCH_RESOURCE_ID_TO_NAME	false	boolean	Turns the mapping of resource id to resource name on or off
LOG_LEVEL	INFO	INFO, WARN, DEBUG, ERROR	Shows the corresponding logs

Below is a comprehensive list of metrics that this software can gather. Note: The "Translatable" column indicates whether the resource ID can be mapped to its corresponding resource name:

Category	Service	Namespace	Translatable	Reference
Compute	Elastic Cloud Server	SYS.ECS	Yes	Basic ECS metrics
	Bare Metal Server	SERVICE.BMS	No	BMS Metrics Under OS Monitoring
	Auto Scaling	SYS.AS	No	AS metrics
Storage	Elastic Volume Service (attached to an ECS or BMS)	SYS.EVS	Yes	EVS metrics
	Object Storage Service (OBS)	SYS.OBS	Yes	OBS metrics
	Scalable File Service	SYS.SFS	No	SFS metrics
	SFS Turbo	SYS.EFS	Yes	SFS Turbo metrics
	Cloud Backup and Recovery	SYS.CBR	Yes	CBR metrics
Network	Elastic IP and bandwidth	SYS.VPC	Yes	VPC metrics
	Elastic Load Balance	SYS.ELB	Yes	ELB metrics
	NAT Gateway	SYS.NAT	Yes	NAT Gateway metrics
Security	Web Application Firewall	SYS.WAF	Yes	WAF metrics
Application	Distributed Message Service	SYS.DMS	Yes	DMS metrics
	Distributed Cache Service	SYS.DCS	Yes	DCS metrics
Database	Relational Database Service	SYS.RDS	Yes	RDS for MySQL metrics RDS for PostgreSQL metrics RDS for SQL Server metrics
	Document Database Service	SYS.DDS	No	DDS metrics
	GaussDB NoSQL	SYS.NoSQL	Yes	GaussDB(for Cassandra) metrics
	GaussDB(for MySQL)	SYS.GAUSSDB	No	GaussDB(for MySQL) metrics
	GaussDB(for openGauss)	SYS.GAUSSDBV5	Yes	GaussDB(for openGauss) metrics
Data analysis	Data Warehouse Service	SYS.DWS	Yes	DWS metrics
	Cloud Search Service	SYS.ES	No	CSS metrics

Kubernetes (Helm)

helm repo add otc-prometheus-exporter https://iits-consulting.github.io/otc-prometheus-exporter/
helm search repo otc-prometheus-exporter
helm install otc-prometheus-exporter otc-prometheus-exporter/otc-prometheus-exporter --set your_values.yaml

Docker and Docker Compose

You can choose to run the application either as a single docker container or in combination with a Prometheus and Grafana server using Docker Compose.

Running a Single Docker Container

If you want a single docker container with the application running then you can do it by following these steps.

1. Make sure you have docker installed and running.
2. Copy the .env.template to .env and fill it out. This makes the docker command much shorter this way and your secrets are not listed in your shell history.
3. Run the following:

docker pull ghcr.io/iits-consulting/otc-prometheus-exporter:latest
docker run --env-file .env ghcr.io/iits-consulting/otc-prometheus-exporter:latest

If you want to start the application, a Prometheus, and Grafana server all at once, then Docker Compose will allow you to do so. This is particularly useful for a quick test or local development, as the entire tool chain is running in this scenario.

1. Make sure you have Docker and Docker Compose installed and running.
2. Copy the .env.template to .env and fill it out, similar to the single container setup. This keeps your docker command short and avoids having your secrets listed in your shell history.
3. Run the following: docker-compose --env-file .env up

Binary

If you want to run the application directly as a binary then you can do it by following these steps.

1. Download and decompress the binary from the release page
2. chmod +x otc-prometheus-exporter to make it executable.
3. On macOS, it might be necessary to remove the Apple quarantine property from it too. This can be done with: xattr -d com.apple.quarantine otc-prometheues-exporter
4. cp .env.template .env
5. Fill out the values in the .env file
6. Run the programm: env $(cat .env) ./otc-prometheues-exporter

Dashboards

Here are pictures of how the dashboards look. Keep in mind that not always all metrics are populated because they are not sent by the Cloud Eye Service.

References

• Open Telekom Cloud Docs
• https://github.com/tiagoReichert/otc-cloudeye-prometheus-exporter