soak_clients

README

This program orchestrates the Trogdor agents that operate on the Soak Cluster. It has multiple responsibilities:

• Create Trogdor scenarios with realistic workloads that run on the cluster for extended periods of time.
• Run periodic jobs that inspect the health and status of said workloads

Usage

To run the soak clients in CPD:

cd ../
vi charts/cc-soak-clients/values.yaml # edit bootstrapServer and apiKey/Secret
vi charts/cc-soak-clients/templates/configMaps.yaml # optionally modify topics.json
helm install charts/cc-trogdor-service/  # spawn trogdor coordinator and agents
helm install charts/cc-soak-clients/  # spawn soak clients and cronjob for status reporting

Possible future improvements

• Allow for interactive task start/pause/stoppage
• Support more diverse scenarios

Documentation

Index

• Constants
• func Report(topicConfigPath string, trogdorCoordinatorHost string)
• func Run(topicConfigPath string, trogdorCoordinatorHost string, trogdorAgentsCount int, ...)
• type SoakTestConfig
• type TopicConfiguration

Constants

const (
	UNIFORM_WORKLOAD  = "uniform"
	GAUSSIAN_WORKLOAD = "gaussian"
)

Functions

func Report

func Report(topicConfigPath string, trogdorCoordinatorHost string)

func Run

func Run(topicConfigPath string, trogdorCoordinatorHost string, trogdorAgentsCount int, bootstrapServers string)

Types

type SoakTestConfig

type SoakTestConfig struct {
	Topics                          []TopicConfiguration `json:"topics"`
	LongLivedTaskDurationMs         uint64               `json:"long_lived_task_duration_ms"`
	ShortLivedTaskDurationMs        uint64               `json:"short_lived_task_duration_ms"`
	ShortLivedTaskRescheduleDelayMs uint64               `json:"short_lived_task_reschedule_delay_ms"`
}

type TopicConfiguration

type TopicConfiguration struct {
	Name                                  string                          `json:"name"`
	PartitionsCount                       int                             `json:"partitions_count"`
	ProduceMBsThroughput                  float32                         `json:"produce_mbs_throughput"`
	ConsumeMBsThroughput                  float32                         `json:"consume_mbs_throughput"`
	LongLivedProduceCount                 int                             `json:"long_lived_producer_count"`
	ShortLivedProduceCount                int                             `json:"short_lived_producer_count"`
	LongLivedConsumeCount                 int                             `json:"long_lived_consumer_count"`
	ShortLivedConsumeCount                int                             `json:"short_lived_consumer_count"`
	TransactionsEnabled                   bool                            `json:"transactions_enabled"`
	IdempotenceEnabled                    bool                            `json:"idempotence_enabled"`
	WorkloadType                          string                          `json:"workload_type"`
	ShortLivedRandomConsumerGroup         bool                            `json:"short_lived_random_consumer_group"`
	ShortLivedConsumerRecordBatchVerifier trogdor.RecordBatchVerifierSpec `json:"short_lived_consumer_record_batch_verifier"`
	LongLivedRandomConsumerGroup          bool                            `json:"long_lived_random_consumer_group"`
	LongLivedConsumerRecordBatchVerifier  trogdor.RecordBatchVerifierSpec `json:"long_lived_consumer_record_batch_verifier"`
}