tracetest/

README

Welcome to Testkube Tracetest Executor

Testkube Tracetest Executor is a test executor to run Tracetest tests with Testkube.

Running Tracetest with Testkube

Tracetest is a testing tool based on OpenTelemetry that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.

Testkube is a Kubernetes-native testing framework for Testers and Developers that allows you to automate the executions of your existing testing tools inside your Kubernetes cluster, removing all the complexity from your CI/CD/GitOps pipelines.

By using the Testkube Tracetest Executor you can unlock Testkube's capacity in conjunction with Tracetest, and leverage the work you have already done to instrument your services.

Quickstart

Here we will show how to use Testkube alongside Tracetest to run your tests in a Kubernetes cluster.

Prerequisites

In your Kubernetes cluster you should have:

1. Testkube: Use HELM or the Testkube CLI to install Testkube Server components in your cluster.
2. Trecetest Server: You need a running instance of Tracetest which is going to be executing your assertions. To do so you can follow the instructions defined in the Tracetest documentation.
3. OpenTelemetry Instrumented Service: In order to generate traces and spans, the service under test must support the basics for propagation through HTTP requests, and also store traces and spans into a Data Store Backend (Jaeger, Grafana Tempo, OpenSearch, etc) or use the OpenTelemetry Collector.

On your machine you should have:

1. Kubectl installed
2. Testkube CLI installed

With everything set up, we will start configuring Testkube and Tracetest.

1. Deploy the Tracetest Executor

Testkube works with the concept of Executors. An Executor is a wrapper around a testing framework (Tracetest in this case) in the form of a Docker container and runs as a Kubernetes job. To start you need to register and deploy the Tracetest executor in your cluster using the Testkube CLI:

kubectl testkube create executor --image kubeshop/testkube-executor-tracetest:latest --types "tracetest/test" --name tracetest-executor

2. Create your test

Now you need a Tracetest test. Have a look at the Tracetest documentation for details on writing tests. Here is a simple test definition example:

type: Test
spec:
  id: R5NITR14g
  name: Pokeshop - List
  description: Get a Pokemon
  trigger:
    type: http
    httpRequest:
      url: http://demo-pokemon-api.demo/pokemon?take=20&skip=0
      method: GET
      headers:
        - key: Content-Type
          value: application/json
  specs:
    - selector: span[tracetest.span.type="http"]
      assertions:
        - attr:http.method = "GET"
    - selector: span[tracetest.span.type="database"]
      assertions:
        - attr:db.name = "pokeshop"

Execute the following command to create the test executor object in Testkube. Do not forget to provide the path to your Tracetest definition file using the --file argument, and also the Tracetest Server endpoint using the TRACETEST_ENDPOINT --variable:

kubectl testkube create test --file my/file/location.yaml --type "tracetest/test" --name pokeshop-tracetest-test --variable TRACETEST_ENDPOINT=http://tracetest

3. Run your test

Finally, to see the integration working, you only need to run the test by executing the following command:

kubectl testkube run test --watch pokeshop-tracetest-test

Architecture

The following is high level sequence diagram on how Testkube and Tracetest interact with the different pieces of the system:

sequenceDiagram
    testkube client->>+testkube: Trigger Testkube test run
    testkube->>+executor CRDs: Get executor details
    executor CRDs-->>-testkube: Send details
    testkube->>+tracetest executor job: Schedules execution
    tracetest executor job->>+tracetest executor job: Configure Tracetest CLI
    tracetest executor job->>+tracetest server: Executes the Tracetest test run
    tracetest server->>+instrumented service: Trigger request
    instrumented service-->>-tracetest server: Get response
    instrumented service->>+data store: Send telemetry data
    tracetest server->>+data store: Fetch trace
    data store-->>-tracetest server: Get trace
    tracetest server->>+tracetest server: Run assertions
    tracetest server-->>-tracetest executor job: Return test run results
    tracetest executor job-->>-testkube: Return test run results
    testkube-->>-testkube client: Send details

Issues and enchancements

Please follow the main Testkube repository for reporting any issues or discussions

Known Error

There might be a version mismatch error coming up:

✖️ Error: Version Mismatch
The CLI version and the server version are not compatible. To fix this, you'll need to make sure that both your CLI and server are using compatible versions.
We recommend upgrading both of them to the latest available version. Check out our documentation https://docs.tracetest.io/configuration/upgrade for simple instructions on how to upgrade.
Thank you for using Tracetest! We apologize for any inconvenience caused.

The quickest way to fix this, is to use your own executor Docker image. For the Dockerfile, check out the file in build/agent. Here you can specify the link to the exact Tracetest executable version you need. To build this, please use the following command in the root of the Testkube repository:

make docker-build-executor EXECUTOR=tracetest GITHUB_TOKEN="" DOCKER_BUILDX_CACHE_FROM=type=registry,ref=docker.io/kubeshop/testkube-tracetest-executor:latest ALPINE_IMAGE=alpine:3.18.0

Don't forget to make this image available to your Testkube environment and update the image used by the executor:

kubectl edit executors/tracetest-executor -n testkube

Testkube

For more info go to main testkube repo

Documentation | Discord

Tracetest

For more info go to main tracetest repo

Documentation | Discord