loadtime/

README

loadtime

This directory contains the loadtime tools, a set of tools for generating transaction load against CometBFT and measuring their resulting latency. loadtime generates transactions that contain the timestamp of when they were generated as well as additional metadata to track the variables used when generating the load.

Building the tool set

The Makefile contains a target for building the loadtime tools.

The following command will build the tool and place the resulting binaries in ./build/.

make build

load

The load binary is built when make build is invoked. The load tool generates transactions and broadcasts them to CometBFT.

load leverages the tm-load-test framework. As a result, all flags and options specified on the tm-load-test apply to load.

Below is a basic invocation for generating load against a CometBFT websocket running on localhost:25567

./build/load \
    -c 1 -T 10 -r 1000 -s 1024 \
    --broadcast-tx-method sync \
    --endpoints ws://localhost:26657/websocket

report

The report binary is built when make build is invoked. The report tool reads all of the blocks from the specified blockstore database and calculates transaction latency metrics. report reads transactions generated by load and uses the difference between the timestamp contained in the transaction and the timestamp of the block the transaction was executed in to determine transaction latency. report outputs a set of metrics calculated on the list of latencies, including minimum, maximum, and average latency as well as the standard deviation.

Below is a basic invocation of the report tool with a data directory under /home/test/.cometbft/data/ where the data was saved in a goleveldb database.

./build/report --database-type goleveldb --data-dir ~/.cometbft/data

The report tool also supports outputting the raw data as csv. This can be useful if you want to use a more powerful tool to aggregate and analyze the data.

Below is an invocation of the report tool that outputs the data to a csv file in out.csv

./build/report --database-type goleveldb --data-dir ~/.cometbft/data --csv out.csv

The report tool outputs the data for each experiment separately, identified by the UUID generated by the load tool at the start of the experiment. It also outputs the experimental values used for the run.