stress

package
v1.8.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 30, 2020 License: MIT Imports: 12 Imported by: 0

README

influx_stress usage and configuration

The binary for influx_stress comes bundled with all influx installations. To run it against an influxd instance located at localhost:8086 with the default configuration options:

See more about the default configuration options

$ influx_stress

To run influx_stress with a configuration file:

$ influx_stress -config my_awesome_test.toml

To daemonize influx_stress and save the output to a results file:

$ influx_stress -config my_awesome_test.toml > my_awesome_test_out.txt 2>&1 &

To run multiple instances of influx_stress just change the measurement each test writes to, details below

$ influx_stress -config my_awesome_test1.toml > my_awesome_test_out1.txt 2>&1 &
$ influx_stress -config my_awesome_test2.toml > my_awesome_test_out2.txt 2>&1 &

Below is a sample configuration file with comments explaining the different options

# The [provision] section creates a new database on the target instance for the stress test to write points to and perform queries against
# This section can be deleted if the instance is manually configured. In that case make sure that the database referenced in [write] exists
# The provisioner will try to delete the database before trying to recreate it.

[provision]
  [provision.basic]
    # If set to false you can delete this section from the config
    enabled = true
    # address of the node to be provisioned
    address = "<node1_ip>:8086"
    # name of the database to create
    database = "stress"
    # This must be set to true
    reset_database = true

# The [write] section defines the shape of the generated data and configures the InfluxDB client
[write]
  # The [write.point_generator] defines the shape of the generated data
  [write.point_generator]
    [write.point_generator.basic]
      # This needs to be set to true
      enabled = true
      # The total number of points a stress_test will write is determined by multiplying the following two numbers:
      # point_count * series_count = total_points
      # Number of points to write to the database for each series
      point_count = 100
      # Number of series to write to the database?
      series_count = 100000
      # This simulates collection interval in the timestamps of generated points
      tick = "10s"
      # This must be set to true
      jitter = true
      # The measurement name for the generated points
      measurement = "cpu"
      # The generated timestamps follow the pattern of { start_date + (n * tick) }
      # This sequence is preserved for each series and is always increasing
      start_date = "2006-Jan-02"
      # Precision for generated points
      # This setting MUST be the same as [write.influx_client.basic]precision
      precision = "s"
      # The '[[]]' in toml format indicates that the element is an array of items. 
      # [[write.point_generator.basic.tag]] defines a tag on the generated points
      # key is the tag key
      # value is the tag value
      # The first tag defined will have '-0' through '-{series_count}' added to the end of the string
      [[write.point_generator.basic.tag]]
        key = "host"
        value = "server"
      [[write.point_generator.basic.tag]]
        key = "location"
        value = "us-west"
      # [[write.point_generator.basic.field]] defines a field on the generated points
      # key is the field key
      # value is the type of the field
      [[write.point_generator.basic.field]]
        key = "value"
        # Can be either "float64", "int", "bool"
        value = "float64"

  # The [write.influx_client] defines what influx instances the stress_test targets
  [write.influx_client]
    [write.influx_client.basic]
      # This must be set to true
      enabled = true
      # This is an array of addresses
      # addresses = ["<node1_ip>:8086","<node2_ip>:8086","<node3_ip>:8086"] to target a cluster
      addresses = ["<node1_ip>:8086"] # to target an individual node 
      # This database in the in the target influx instance to write to
      # This database MUST be created in the target instance or the test will fail
      database = "stress"
      # Write precision for points
      # This setting MUST be the same as [write.point_generator.basic]precision
      precision = "s"
      # The number of point to write to the database with each POST /write sent
      batch_size = 5000
      # An optional amount of time for a worker to wait between POST requests
      batch_interval = "0s"
      # The number of workers to use to write to the database
      # More workers == more load with diminishing returns starting at ~5 workers
      # 10 workers provides a medium-high level of load to the database
      concurrency = 10
      # This must be set to false
      ssl = false
      # This must be set to "line_http"
      format = "line_http"

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewOutputConfig added in v0.10.0 

func NewOutputConfig() *outputConfig

Types

type AbstractField added in v0.9.6 

type AbstractField struct {
	Key  string `toml:"key"`
	Type string `toml:"type"`
}

AbstractField is a struct that abstractly defines a field

type AbstractFields added in v0.9.6 

type AbstractFields []AbstractField

AbstractFields is a slice of abstract fields

func (AbstractFields) Template added in v0.9.6 

func (f AbstractFields) Template() (string, []string)

Template returns a templated string of fields

type AbstractTag added in v0.9.6 

type AbstractTag struct {
	Key   string `toml:"key"`
	Value string `toml:"value"`
}

AbstractTag is a struct that abstractly defines a tag

type AbstractTags added in v0.9.6 

type AbstractTags []AbstractTag

AbstractTags is a slice of abstract tags

func (AbstractTags) Template added in v0.9.6 

func (t AbstractTags) Template() string

Template returns a templated string of tags

type BasicClient added in v0.9.6 

type BasicClient struct {
	Enabled         bool     `toml:"enabled"`
	Addresses       []string `toml:"addresses"`
	Database        string   `toml:"database"`
	RetentionPolicy string   `toml:"retention-policy"`
	Precision       string   `toml:"precision"`
	BatchSize       int      `toml:"batch_size"`
	BatchInterval   string   `toml:"batch_interval"`
	Concurrency     int      `toml:"concurrency"`
	SSL             bool     `toml:"ssl"`
	Format          string   `toml:"format"`
	// contains filtered or unexported fields
}

BasicClient implements the InfluxClient interface.

func (*BasicClient) BasicWriteHandler added in v0.10.0 

func (b *BasicClient) BasicWriteHandler(rs <-chan response, wt *Timer)

BasicWriteHandler handles write responses.

func (*BasicClient) Batch added in v0.9.6 

func (c *BasicClient) Batch(ps <-chan Point, r chan<- response) error

Batch groups together points

type BasicPointGenerator added in v0.9.6 

type BasicPointGenerator struct {
	PointCount  int            `toml:"point_count"`
	Tick        string         `toml:"tick"`
	Jitter      bool           `toml:"jitter"`
	Measurement string         `toml:"measurement"`
	SeriesCount int            `toml:"series_count"`
	Tags        AbstractTags   `toml:"tag"`
	Fields      AbstractFields `toml:"field"`
	StartDate   string         `toml:"start_date"`
	Precision   string         `toml:"precision"`
	// contains filtered or unexported fields
}

BasicPointGenerator implements the PointGenerator interface

func (*BasicPointGenerator) Generate added in v0.9.6 

func (b *BasicPointGenerator) Generate() (<-chan Point, error)

Generate returns a point channel. Implements the Generate method for the PointGenerator interface

func (*BasicPointGenerator) Template added in v0.9.6 

func (b *BasicPointGenerator) Template() func(i int, t time.Time) *Pnt

Template returns a function that returns a pointer to a Pnt.

func (*BasicPointGenerator) Time added in v0.9.6 

func (b *BasicPointGenerator) Time() time.Time

Time returns the timestamp for the latest points that are being generated. Implements the Time method for the PointGenerator interface.

type BasicProvisioner added in v0.9.6 

type BasicProvisioner struct {
	Enabled       bool   `toml:"enabled"`
	Address       string `toml:"address"`
	Database      string `toml:"database"`
	ResetDatabase bool   `toml:"reset_database"`
}

BasicProvisioner implements the Provisioner interface.

func (*BasicProvisioner) Provision added in v0.9.6 

func (b *BasicProvisioner) Provision() error

Provision runs the resetDB function.

type BasicQuery added in v0.9.6 

type BasicQuery struct {
	Template   Query `toml:"template"`
	QueryCount int   `toml:"query_count"`
	// contains filtered or unexported fields
}

BasicQuery implements the QueryGenerator interface

func (*BasicQuery) QueryGenerate added in v0.9.6 

func (q *BasicQuery) QueryGenerate(now func() time.Time) (<-chan Query, error)

QueryGenerate returns a Query channel

func (*BasicQuery) SetTime added in v0.9.6 

func (q *BasicQuery) SetTime(t time.Time)

SetTime sets the internal state of time

type BasicQueryClient added in v0.9.6 

type BasicQueryClient struct {
	Enabled       bool     `toml:"enabled"`
	Addresses     []string `toml:"addresses"`
	Database      string   `toml:"database"`
	QueryInterval string   `toml:"query_interval"`
	Concurrency   int      `toml:"concurrency"`
	// contains filtered or unexported fields
}

BasicQueryClient implements the QueryClient interface

func (*BasicQueryClient) BasicReadHandler added in v0.10.0 

func (b *BasicQueryClient) BasicReadHandler(r <-chan response, rt *Timer)

BasicReadHandler handles read responses.

func (*BasicQueryClient) Exec added in v0.9.6 

func (b *BasicQueryClient) Exec(qs <-chan Query, r chan<- response) error

Exec listens to the query channel an executes queries as they come in

func (*BasicQueryClient) Init added in v0.9.6 

func (b *BasicQueryClient) Init() error

Init initializes the InfluxDB client

func (*BasicQueryClient) Query added in v0.9.6 

func (b *BasicQueryClient) Query(cmd Query) (response, error)

Query runs the query

type BroadcastChannel added in v0.10.0 

type BroadcastChannel struct {
	// contains filtered or unexported fields
}

func NewBroadcastChannel added in v0.10.0 

func NewBroadcastChannel() *BroadcastChannel

func (*BroadcastChannel) Broadcast added in v0.10.0 

func (b *BroadcastChannel) Broadcast(r response)

func (*BroadcastChannel) Close added in v0.10.0 

func (b *BroadcastChannel) Close()

func (*BroadcastChannel) Handle added in v0.10.0 

func (b *BroadcastChannel) Handle(rs <-chan response, t *Timer)

func (*BroadcastChannel) Register added in v0.10.0 

func (b *BroadcastChannel) Register(fn responseHandler)

type ConcurrencyLimiter 

type ConcurrencyLimiter struct {
	// contains filtered or unexported fields
}

ConcurrencyLimiter is a go routine safe struct that can be used to ensure that no more than a specifid max number of goroutines are executing.

func NewConcurrencyLimiter 

func NewConcurrencyLimiter(max int) *ConcurrencyLimiter

NewConcurrencyLimiter returns a configured limiter that will ensure that calls to Increment will block if the max is hit.

func (*ConcurrencyLimiter) Decrement 

func (c *ConcurrencyLimiter) Decrement()

Decrement will reduce the count of running goroutines by 1

func (*ConcurrencyLimiter) Increment 

func (c *ConcurrencyLimiter) Increment()

Increment will increase the count of running goroutines by 1. if the number is currently at the max, the call to Increment will block until another goroutine decrements.

type Config 

type Config struct {
	Provision Provision `toml:"provision"`
	Write     Write     `toml:"write"`
	Read      Read      `toml:"read"`
}

Config is a struct for the Stress test configuration

func BasicStress added in v0.9.6 

func BasicStress() (*Config, error)

BasicStress returns a config for a basic stress test.

func DecodeConfig added in v0.9.6 

func DecodeConfig(s string) (*Config, error)

DecodeConfig takes a file path for a toml config file and returns a pointer to a Config Struct.

func DecodeFile 

func DecodeFile(s string) (*Config, error)

DecodeFile takes a file path for a toml config file and returns a pointer to a Config Struct.

func NewConfig 

func NewConfig(s string) (*Config, error)

NewConfig returns a pointer to a Config

type Field added in v0.9.6 

type Field KeyValue

Field is a struct for a field in influxdb.

type Fields added in v0.9.6 

type Fields []Field

Fields is an slice of all the fields for a point.

type InfluxClient added in v0.9.6 

type InfluxClient interface {
	Batch(ps <-chan Point, r chan<- response) error
	// contains filtered or unexported methods
}

InfluxClient is an interface for writing data to the database.

type InfluxClients added in v0.9.6 

type InfluxClients struct {
	Basic BasicClient `toml:"basic"`
}

InfluxClients is a struct that contains the configuration parameters for all implemented InfluxClient's.

type KeyValue added in v0.9.6 

type KeyValue struct {
	Key   string
	Value string
}

KeyValue is an intermediate type that is used to express Tag and Field similarly.

type Pnt added in v0.9.6 

type Pnt struct {
	// contains filtered or unexported fields
}

Pnt is a struct that implements the Point interface.

func (Pnt) Graphite added in v0.9.6 

func (p Pnt) Graphite() []byte

Graphite returns a byte array for a point in graphite format.

func (Pnt) Line added in v0.9.6 

func (p Pnt) Line() []byte

Line returns a byte array for a point in line protocol format.

func (*Pnt) Next added in v0.9.6 

func (p *Pnt) Next(i int, t time.Time)

Next generates very simple points very efficiently. TODO: Take this out

func (Pnt) OpenJSON added in v0.9.6 

func (p Pnt) OpenJSON() []byte

OpenJSON returns a byte array for a point in opentsdb json format

func (Pnt) OpenTelnet added in v0.9.6 

func (p Pnt) OpenTelnet() []byte

OpenTelnet returns a byte array for a point in opentsdb-telnet format

func (*Pnt) Set added in v0.9.6 

func (p *Pnt) Set(b []byte)

Set sets the internal state for a Pnt.

type Point added in v0.9.6 

type Point interface {
	Line() []byte
	Graphite() []byte
	OpenJSON() []byte
	OpenTelnet() []byte
}

Point is an interface that is used to represent the abstract idea of a point in InfluxDB.

type PointGenerator added in v0.9.6 

type PointGenerator interface {
	Generate() (<-chan Point, error)
	Time() time.Time
}

PointGenerator is an interface for generating points.

type PointGenerators added in v0.9.6 

type PointGenerators struct {
	Basic *BasicPointGenerator `toml:"basic"`
}

PointGenerators is a struct that contains the configuration parameters for all implemented PointGenerator's.

type Provision added in v0.9.6 

type Provision struct {
	Basic BasicProvisioner `toml:"basic"`
}

Provision is a struct that contains the configuration parameters for all implemented Provisioner's.

type Provisioner added in v0.9.6 

type Provisioner interface {
	Provision() error
}

Provisioner is an interface that provisions an InfluxDB instance

type Querier added in v0.9.6 

type Querier struct {
	QueryGenerator
	QueryClient
}

Querier queries the database.

func NewQuerier added in v0.9.6 

func NewQuerier(q QueryGenerator, c QueryClient) Querier

NewQuerier returns a Querier.

type Query added in v0.9.6 

type Query string

Query is query

type QueryClient added in v0.9.6 

type QueryClient interface {
	Query(q Query) (response, error)
	Exec(qs <-chan Query, r chan<- response) error
}

QueryClient is an interface that can write a query to an InfluxDB instance.

type QueryClients added in v0.9.6 

type QueryClients struct {
	Basic BasicQueryClient `toml:"basic"`
}

QueryClients is a struct that contains the configuration parameters for all implemented QueryClient's.

type QueryGenerator added in v0.9.6 

type QueryGenerator interface {
	QueryGenerate(f func() time.Time) (<-chan Query, error)
	SetTime(t time.Time)
}

QueryGenerator is an interface that is used to define queries that will be ran on the DB.

type QueryGenerators added in v0.9.6 

type QueryGenerators struct {
	Basic BasicQuery `toml:"basic"`
}

QueryGenerators is a struct that contains the configuration parameters for all implemented QueryGenerator's.

type QueryResponse added in v0.9.6 

type QueryResponse struct {
	Body string
	// contains filtered or unexported fields
}

QueryResponse is a response for a Querier

func (QueryResponse) Success added in v0.9.6 

func (r QueryResponse) Success() bool

Success returns true if the request was successful and false otherwise.

type Read added in v0.9.6 

type Read struct {
	QueryGenerators QueryGenerators `toml:"query_generator"`
	QueryClients    QueryClients    `toml:"query_client"`
}

Read is a struct that contains the configuration parameters for the stress test Reader.

type ResponseTime 

type ResponseTime struct {
	Value int
	Time  time.Time
}

ResponseTime is a struct that contains `Value` `Time` pairing.

func NewResponseTime 

func NewResponseTime(v int) ResponseTime

NewResponseTime returns a new response time with value `v` and time `time.Now()`.

type ResponseTimes 

type ResponseTimes []ResponseTime

ResponseTimes is a slice of response times

func (ResponseTimes) Len 

func (rs ResponseTimes) Len() int

Implements the `Len` method for the sort.Interface type

func (ResponseTimes) Less 

func (rs ResponseTimes) Less(i, j int) bool

Implements the `Less` method for the sort.Interface type

func (ResponseTimes) Swap 

func (rs ResponseTimes) Swap(i, j int)

Implements the `Swap` method for the sort.Interface type

type StdPoint added in v0.9.6 

type StdPoint struct {
	Measurement string
	Tags        Tags
	Fields      Fields
	Timestamp   int64
}

StdPoint represents a point in InfluxDB

func (StdPoint) Graphite added in v0.9.6 

func (p StdPoint) Graphite() []byte

Graphite returns a byte array for a point in graphite-protocol format

func (StdPoint) Line added in v0.9.6 

func (p StdPoint) Line() []byte

Line returns a byte array for a point in line-protocol format

func (StdPoint) OpenJSON added in v0.9.6 

func (p StdPoint) OpenJSON() []byte

OpenJSON returns a byte array for a point in JSON format

func (StdPoint) OpenTelnet added in v0.9.6 

func (p StdPoint) OpenTelnet() []byte

OpenTelnet returns a byte array for a point in OpenTSDB-telnet format

type StressTest added in v0.9.6 

type StressTest struct {
	Provisioner
	Writer
	Querier
}

StressTest is a struct that contains all of the logic required to execute a Stress Test

func NewStressTest added in v0.9.6 

func NewStressTest(p Provisioner, w Writer, r Querier) StressTest

NewStressTest returns an instance of a StressTest

func (*StressTest) Start added in v0.9.6 

func (s *StressTest) Start(wHandle responseHandler, rHandle responseHandler)

Start executes the Stress Test

type Tag added in v0.9.6 

type Tag KeyValue

Tag is a struct for a tag in influxdb.

type Tags added in v0.9.6 

type Tags []Tag

Tags is an slice of all the tags for a point.

type Timer 

type Timer struct {
	// contains filtered or unexported fields
}

Timer is struct that can be used to track elaspsed time

func NewTimer 

func NewTimer() *Timer

NewTimer returns a pointer to a `Timer` struct where the timers `start` field has been set to `time.Now()`

func (*Timer) Elapsed 

func (t *Timer) Elapsed() time.Duration

Elapsed returns the total elapsed time between the `start` and `end` fields on a timer.

func (*Timer) End 

func (t *Timer) End() time.Time

End returns a Timers end field

func (*Timer) Start 

func (t *Timer) Start() time.Time

Start returns a Timers start field

func (*Timer) StartTimer 

func (t *Timer) StartTimer()

StartTimer sets a timers `start` field to the current time

func (*Timer) StopTimer 

func (t *Timer) StopTimer()

StopTimer sets a timers `end` field to the current time

type Write added in v0.9.6 

type Write struct {
	PointGenerators PointGenerators `toml:"point_generator"`
	InfluxClients   InfluxClients   `toml:"influx_client"`
}

Write is a struct that contains the configuration parameters for the stress test Writer.

type WriteResponse added in v0.9.6 

type WriteResponse response

WriteResponse is a response for a Writer

type Writer added in v0.9.6 

type Writer struct {
	PointGenerator
	InfluxClient
}

Writer is a PointGenerator and an InfluxClient.

func NewWriter added in v0.9.6 

func NewWriter(p PointGenerator, i InfluxClient) Writer

NewWriter returns a Writer.

Source Files

View all Source files

Directories

Path Synopsis
v2
statement
stress_client
stressql
stressql/statement

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.