apptest

package
v1.4.0-victorialogs Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 22, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

README

App Integration Tests

The apptest package contains the integration tests for the VictoriaMetrics applications (such as vmstorage, vminsert, and vmselect).

An integration test aims at verifying the behavior of an application as a whole, as apposed to a unit test that verifies the behavior of a building block of an application.

To achieve that an integration test starts an application in a separate process and then issues HTTP requets to it and verifies the responses, examines the metrics the app exposes and/or files it creates, etc.

Note that an object of testing may be not just a single app, but several apps working together. A good example is VictoriaMetrics cluster. An integration test may reproduce an arbitrary cluster configuration and verify how the components work together as a system.

The package provides a collection of helpers to start applications and make queries to them:

  • app.go - contains the generic code for staring an application and should not be used by integration tests directly.
  • {vmstorage,vminsert,etc}.go - build on top of app.go and provide the code for staring a specific application.
  • client.go - provides helper functions for sending HTTP requests to applications.

The integration tests themselves reside in tests/*_test.go files. Apart from having the _test suffix, there are no strict rules of how to name a file, but the name should reflect the prevailing purpose of the tests located in that file. For example, sharding_test.go aims at testing data sharding.

Since integration tests start applications in a separate process, they require the application binary files to be built and put into the bin directory. The build rule used for running integration tests, make integration-test, accounts for that, it builds all application binaries before running the tests. But if you want to run the tests without make, i.e. by executing go test ./app/apptest, you will need to build the binaries first (for example, by executing make all).

Not all binaries can be built from master branch, cluster binaries can be built only from cluster branch. Hence, not all test cases suitable to run in both branches:

  • If test is using binaries from cluster branch, then test name should be prefixed with TestCluster word
  • If test is using binaries from master branch, then test name should be prefixed with TestVmsingle word.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AssertOptions added in v1.97.13 

type AssertOptions struct {
	Msg        string
	Got        func() any
	Want       any
	CmpOpts    []cmp.Option
	DoNotRetry bool
	Retries    int
	Period     time.Duration
	FailNow    bool
}

AssertOptions hold the assertion params, such as got and wanted values as well as the message that should be included into the assertion error message in case of failure.

In VictoriaMetrics (especially the cluster version) the inserted data does not become visible for querying right away. Therefore, the first comparisons may fail. AssertOptions allow to configure how many times the actual result must be retrieved and compared with the expected one and for long to wait between the retries. If these two params (`Retries` and `Period`) are not set, the default values will be used.

If it is known that the data is available, then the retry functionality can be disabled by setting the `DoNotRetry` field.

AssertOptions are used by the TestCase.Assert() method, and this method uses cmp.Diff() from go-cmp package for comparing got and wanted values. AssertOptions, therefore, allows to pass cmp.Options to cmp.Diff() via `CmpOpts` field.

Finally the `FailNow` field controls whether the assertion should fail using `testing.T.Errorf()` or `testing.T.Fatalf()`.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is used for interacting with the apps over the network.

At the moment it only supports HTTP protocol but may be exptended to support RPCs, etc.

func NewClient 

func NewClient() *Client

NewClient creates a new client.

func (*Client) CloseConnections 

func (c *Client) CloseConnections()

CloseConnections closes client connections.

func (*Client) Get 

func (c *Client) Get(t *testing.T, url string, wantStatusCode int) string

Get sends a HTTP GET request. Once the function receives a response, it checks whether the response status code matches the expected one and returns the response body to the caller.

func (*Client) Post 

func (c *Client) Post(t *testing.T, url, contentType string, data []byte, wantStatusCode int) string

Post sends a HTTP POST request. Once the function receives a response, it checks whether the response status code matches the expected one and returns the response body to the caller.

func (*Client) PostForm 

func (c *Client) PostForm(t *testing.T, url string, data url.Values, wantStatusCode int) string

PostForm sends a HTTP POST request containing the POST-form data. Once the function receives a response, it checks whether the response status code matches the expected one and returns the response body to the caller.

type ClusterOptions added in v1.97.15 

type ClusterOptions struct {
	Vmstorage1Instance string
	Vmstorage1Flags    []string
	Vmstorage2Instance string
	Vmstorage2Flags    []string
	VminsertInstance   string
	VminsertFlags      []string
	VmselectInstance   string
	VmselectFlags      []string
}

ClusterOptions holds the params for simple cluster configuration suitable for most tests.

The cluster consists of two vmstorages, one vminsert and one vmselect, no data replication.

Such configuration is suitable for tests that don't verify the cluster-specific behavior (such as sharding, replication, or multilevel vmselect) but instead just need a typical cluster configuration to verify some business logic (such as API surface, or MetricsQL). Such cluster tests usually come paired with corresponding vmsingle tests.

type PrometheusAPIV1QueryResponse added in v1.97.12 

type PrometheusAPIV1QueryResponse struct {
	Status string
	Data   *QueryData
}

PrometheusAPIV1QueryResponse is an inmemory representation of the /prometheus/api/v1/query or /prometheus/api/v1/query_range response.

func NewPrometheusAPIV1QueryResponse added in v1.97.12 

func NewPrometheusAPIV1QueryResponse(t *testing.T, s string) *PrometheusAPIV1QueryResponse

NewPrometheusAPIV1QueryResponse is a test helper function that creates a new instance of PrometheusAPIV1QueryResponse by unmarshalling a json string.

type PrometheusAPIV1SeriesResponse 

type PrometheusAPIV1SeriesResponse struct {
	Status    string
	IsPartial bool
	Data      []map[string]string
	Trace     *Trace
}

PrometheusAPIV1SeriesResponse is an inmemory representation of the /prometheus/api/v1/series response.

func NewPrometheusAPIV1SeriesResponse added in v1.97.12 

func NewPrometheusAPIV1SeriesResponse(t *testing.T, s string) *PrometheusAPIV1SeriesResponse

NewPrometheusAPIV1SeriesResponse is a test helper function that creates a new instance of PrometheusAPIV1SeriesResponse by unmarshalling a json string.

func (*PrometheusAPIV1SeriesResponse) Sort added in v1.97.13 

func (r *PrometheusAPIV1SeriesResponse) Sort() *PrometheusAPIV1SeriesResponse

Sort sorts the response data.

type PrometheusQuerier added in v1.97.12 

type PrometheusQuerier interface {
	PrometheusAPIV1Export(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse
	PrometheusAPIV1Query(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse
	PrometheusAPIV1QueryRange(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse
	PrometheusAPIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *PrometheusAPIV1SeriesResponse
}

PrometheusQuerier contains methods available to Prometheus-like HTTP API for Querying

type PrometheusWriteQuerier added in v1.97.13 

type PrometheusWriteQuerier interface {
	PrometheusWriter
	PrometheusQuerier
	StorageFlusher
}

PrometheusWriteQuerier encompasses the methods for writing, flushing and querying the data.

type PrometheusWriter added in v1.97.12 

type PrometheusWriter interface {
	PrometheusAPIV1Write(t *testing.T, records []pb.TimeSeries, opts QueryOpts)
	PrometheusAPIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts)
}

PrometheusWriter contains methods available to Prometheus-like HTTP API for Writing new data

type QueryData added in v1.97.12 

type QueryData struct {
	ResultType string
	Result     []*QueryResult
}

QueryData holds the query result along with its type.

type QueryOpts added in v1.97.12 

type QueryOpts struct {
	Tenant       string
	Timeout      string
	Start        string
	End          string
	Time         string
	Step         string
	ExtraFilters []string
	ExtraLabels  []string
	Trace        string
}

QueryOpts contains various params used for querying or ingesting data

type QueryResult added in v1.97.12 

type QueryResult struct {
	Metric  map[string]string
	Sample  *Sample   `json:"value"`
	Samples []*Sample `json:"values"`
}

QueryResult holds the metric name (in the form of label name-value collection) and its samples.

Sample or Samples field is set for /prometheus/api/v1/query or /prometheus/api/v1/query_range response respectively.

type Sample added in v1.97.12 

type Sample struct {
	Timestamp int64
	Value     float64
}

Sample is a timeseries value at a given timestamp.

func NewSample added in v1.97.12 

func NewSample(t *testing.T, timeStr string, value float64) *Sample

NewSample is a test helper function that creates a new sample out of time in RFC3339 format and a value.

func (*Sample) UnmarshalJSON added in v1.97.12 

func (s *Sample) UnmarshalJSON(b []byte) error

UnmarshalJSON populates the sample fields from a JSON string.

type ServesMetrics 

type ServesMetrics struct {
	// contains filtered or unexported fields
}

ServesMetrics is used to retrive the app's metrics.

This type is expected to be embdded by the apps that serve metrics.

func (*ServesMetrics) GetIntMetric 

func (app *ServesMetrics) GetIntMetric(t *testing.T, metricName string) int

GetIntMetric retrieves the value of a metric served by an app at /metrics URL. The value is then converted to int.

func (*ServesMetrics) GetMetric 

func (app *ServesMetrics) GetMetric(t *testing.T, metricName string) float64

GetMetric retrieves the value of a metric served by an app at /metrics URL.

func (*ServesMetrics) GetMetricsByPrefix added in v1.97.13 

func (app *ServesMetrics) GetMetricsByPrefix(t *testing.T, prefix string) []float64

GetMetricsByPrefix retrieves the values of all metrics that start with given prefix.

type Stopper added in v1.97.12 

type Stopper interface {
	Stop()
}

Stopper is an interface of objects that needs to be stopped via Stop() call

type StorageFlusher added in v1.97.13 

type StorageFlusher interface {
	ForceFlush(t *testing.T)
}

StorageFlusher defines a method that forces the flushing of data inserted into the storage, so it becomes available for searching immediately.

type TestCase 

type TestCase struct {
	// contains filtered or unexported fields
}

TestCase holds the state and defines clean-up procedure common for all test cases.

func NewTestCase 

func NewTestCase(t *testing.T) *TestCase

NewTestCase creates a new test case.

func (*TestCase) Assert added in v1.97.13 

func (tc *TestCase) Assert(opts *AssertOptions)

Assert compares the actual result with the expected one possibly multiple times in order to account for the fact that the inserted data does not become available for querying right away (especially in cluster version of VictoriaMetrics).

func (*TestCase) Client 

func (tc *TestCase) Client() *Client

Client returns an instance of the client that can be used for interacting with the app(s) under test.

func (*TestCase) Dir 

func (tc *TestCase) Dir() string

Dir returns the directory name that should be used by as the -storageDataDir.

func (*TestCase) ForceFlush added in v1.97.14 

func (tc *TestCase) ForceFlush(apps ...*Vmstorage)

ForceFlush flushes zero or more storages.

func (*TestCase) MustStartCluster added in v1.97.15 

func (tc *TestCase) MustStartCluster(opts *ClusterOptions) PrometheusWriteQuerier

MustStartCluster starts a typical cluster configuration with custom flags.

func (*TestCase) MustStartDefaultCluster added in v1.97.13 

func (tc *TestCase) MustStartDefaultCluster() PrometheusWriteQuerier

MustStartDefaultCluster starts a typical cluster configuration with default flags.

func (*TestCase) MustStartDefaultVmsingle added in v1.97.13 

func (tc *TestCase) MustStartDefaultVmsingle() *Vmsingle

MustStartDefaultVmsingle is a test helper function that starts an instance of vmsingle with defaults suitable for most tests.

func (*TestCase) MustStartVminsert added in v1.97.12 

func (tc *TestCase) MustStartVminsert(instance string, flags []string) *Vminsert

MustStartVminsert is a test helper function that starts an instance of vminsert and fails the test if the app fails to start.

func (*TestCase) MustStartVmselect added in v1.97.12 

func (tc *TestCase) MustStartVmselect(instance string, flags []string) *Vmselect

MustStartVmselect is a test helper function that starts an instance of vmselect and fails the test if the app fails to start.

func (*TestCase) MustStartVmsingle added in v1.97.12 

func (tc *TestCase) MustStartVmsingle(instance string, flags []string) *Vmsingle

MustStartVmsingle is a test helper function that starts an instance of vmsingle and fails the test if the app fails to start.

func (*TestCase) MustStartVmstorage added in v1.97.12 

func (tc *TestCase) MustStartVmstorage(instance string, flags []string) *Vmstorage

MustStartVmstorage is a test helper function that starts an instance of vmstorage and fails the test if the app fails to start.

func (*TestCase) Stop added in v1.97.12 

func (tc *TestCase) Stop()

Stop performs the test case clean up, such as closing all client connections and removing the -storageDataDir directory.

Note that the -storageDataDir is not removed in case of test case failure to allow for further manual debugging.

func (*TestCase) StopApp added in v1.97.14 

func (tc *TestCase) StopApp(instance string)

StopApp stops the app identified by the `instance` name and removes it from the collection of started apps.

func (*TestCase) T added in v1.97.14 

func (tc *TestCase) T() *testing.T

T returns the test state.

type Trace added in v1.97.14 

type Trace struct {
	DurationMsec float64 `json:"duration_msec"`
	Message      string
	Children     []*Trace
}

Trace provides the description and the duration of some unit of work that has been performed during the request processing.

func (*Trace) Contains added in v1.97.14 

func (t *Trace) Contains(s string) int

Contains counts how many trace messages contain substring s.

func (*Trace) String added in v1.97.14 

func (t *Trace) String() string

String returns string representation of the trace.

type Vminsert 

type Vminsert struct {
	*ServesMetrics
	// contains filtered or unexported fields
}

Vminsert holds the state of a vminsert app and provides vminsert-specific functions.

func StartVminsert 

func StartVminsert(instance string, flags []string, cli *Client) (*Vminsert, error)

StartVminsert starts an instance of vminsert with the given flags. It also sets the default flags and populates the app instance state with runtime values extracted from the application log (such as httpListenAddr)

func (*Vminsert) ClusternativeListenAddr added in v1.97.14 

func (app *Vminsert) ClusternativeListenAddr() string

ClusternativeListenAddr returns the address at which the vminsert process is listening for connections from other vminsert apps.

func (Vminsert) Name added in v1.97.14 

func (app Vminsert) Name() string

Name returns the application instance name.

func (*Vminsert) PrometheusAPIV1ImportPrometheus 

func (app *Vminsert) PrometheusAPIV1ImportPrometheus(t *testing.T, records []string, opts QueryOpts)

PrometheusAPIV1ImportPrometheus is a test helper function that inserts a collection of records in Prometheus text exposition format for the given tenant by sending a HTTP POST request to /prometheus/api/v1/import/prometheus vminsert endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1importprometheus

func (*Vminsert) PrometheusAPIV1Write added in v1.97.13 

func (app *Vminsert) PrometheusAPIV1Write(t *testing.T, records []pb.TimeSeries, opts QueryOpts)

PrometheusAPIV1Write is a test helper function that inserts a collection of records in Prometheus remote-write format by sending a HTTP POST request to /prometheus/api/v1/write vminsert endpoint.

func (Vminsert) Stop 

func (app Vminsert) Stop()

stop sends the app process a SIGINT signal and waits until it terminates gracefully.

func (*Vminsert) String 

func (app *Vminsert) String() string

String returns the string representation of the vminsert app state.

type Vmselect 

type Vmselect struct {
	*ServesMetrics
	// contains filtered or unexported fields
}

Vmselect holds the state of a vmselect app and provides vmselect-specific functions.

func StartVmselect 

func StartVmselect(instance string, flags []string, cli *Client) (*Vmselect, error)

StartVmselect starts an instance of vmselect with the given flags. It also sets the default flags and populates the app instance state with runtime values extracted from the application log (such as httpListenAddr)

func (*Vmselect) ClusternativeListenAddr 

func (app *Vmselect) ClusternativeListenAddr() string

ClusternativeListenAddr returns the address at which the vmselect process is listening for connections from other vmselect apps.

func (Vmselect) Name added in v1.97.14 

func (app Vmselect) Name() string

Name returns the application instance name.

func (*Vmselect) PrometheusAPIV1Export added in v1.97.13 

func (app *Vmselect) PrometheusAPIV1Export(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse

PrometheusAPIV1Export is a test helper function that performs the export of raw samples in JSON line format by sending a HTTP POST request to /prometheus/api/v1/export vmselect endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1export

func (*Vmselect) PrometheusAPIV1Query added in v1.97.12 

func (app *Vmselect) PrometheusAPIV1Query(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse

PrometheusAPIV1Query is a test helper function that performs PromQL/MetricsQL instant query by sending a HTTP POST request to /prometheus/api/v1/query vmselect endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1query

func (*Vmselect) PrometheusAPIV1QueryRange added in v1.97.12 

func (app *Vmselect) PrometheusAPIV1QueryRange(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse

PrometheusAPIV1QueryRange is a test helper function that performs PromQL/MetricsQL range query by sending a HTTP POST request to /prometheus/api/v1/query_range vmselect endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1query_range

func (*Vmselect) PrometheusAPIV1Series 

func (app *Vmselect) PrometheusAPIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *PrometheusAPIV1SeriesResponse

PrometheusAPIV1Series sends a query to a /prometheus/api/v1/series endpoint and returns the list of time series that match the query.

See https://docs.victoriametrics.com/url-examples/#apiv1series

func (Vmselect) Stop 

func (app Vmselect) Stop()

stop sends the app process a SIGINT signal and waits until it terminates gracefully.

func (*Vmselect) String 

func (app *Vmselect) String() string

String returns the string representation of the vmselect app state.

type Vmsingle added in v1.97.12 

type Vmsingle struct {
	*ServesMetrics
	// contains filtered or unexported fields
}

Vmsingle holds the state of a vmsingle app and provides vmsingle-specific functions.

func StartVmsingle added in v1.97.12 

func StartVmsingle(instance string, flags []string, cli *Client) (*Vmsingle, error)

StartVmsingle starts an instance of vmsingle with the given flags. It also sets the default flags and populates the app instance state with runtime values extracted from the application log (such as httpListenAddr).

func (*Vmsingle) ForceFlush added in v1.97.12 

func (app *Vmsingle) ForceFlush(t *testing.T)

ForceFlush is a test helper function that forces the flushing of inserted data, so it becomes available for searching immediately.

func (*Vmsingle) InfluxWrite added in v1.97.15 

func (app *Vmsingle) InfluxWrite(t *testing.T, records []string, _ QueryOpts)

InfluxWrite is a test helper function that inserts a collection of records in Influx line format by sending a HTTP POST request to /influx/write vmsingle endpoint.

See https://docs.victoriametrics.com/url-examples/#influxwrite

func (Vmsingle) Name added in v1.97.14 

func (app Vmsingle) Name() string

Name returns the application instance name.

func (*Vmsingle) PrometheusAPIV1Export added in v1.97.13 

func (app *Vmsingle) PrometheusAPIV1Export(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse

PrometheusAPIV1Export is a test helper function that performs the export of raw samples in JSON line format by sending a HTTP POST request to /prometheus/api/v1/export vmsingle endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1export

func (*Vmsingle) PrometheusAPIV1ImportPrometheus added in v1.97.12 

func (app *Vmsingle) PrometheusAPIV1ImportPrometheus(t *testing.T, records []string, _ QueryOpts)

PrometheusAPIV1ImportPrometheus is a test helper function that inserts a collection of records in Prometheus text exposition format by sending a HTTP POST request to /prometheus/api/v1/import/prometheus vmsingle endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1importprometheus

func (*Vmsingle) PrometheusAPIV1Query added in v1.97.12 

func (app *Vmsingle) PrometheusAPIV1Query(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse

PrometheusAPIV1Query is a test helper function that performs PromQL/MetricsQL instant query by sending a HTTP POST request to /prometheus/api/v1/query vmsingle endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1query

func (*Vmsingle) PrometheusAPIV1QueryRange added in v1.97.12 

func (app *Vmsingle) PrometheusAPIV1QueryRange(t *testing.T, query string, opts QueryOpts) *PrometheusAPIV1QueryResponse

PrometheusAPIV1QueryRange is a test helper function that performs PromQL/MetricsQL range query by sending a HTTP POST request to /prometheus/api/v1/query_range vmsingle endpoint.

See https://docs.victoriametrics.com/url-examples/#apiv1query_range

func (*Vmsingle) PrometheusAPIV1Series added in v1.97.12 

func (app *Vmsingle) PrometheusAPIV1Series(t *testing.T, matchQuery string, opts QueryOpts) *PrometheusAPIV1SeriesResponse

PrometheusAPIV1Series sends a query to a /prometheus/api/v1/series endpoint and returns the list of time series that match the query.

See https://docs.victoriametrics.com/url-examples/#apiv1series

func (*Vmsingle) PrometheusAPIV1Write added in v1.97.13 

func (app *Vmsingle) PrometheusAPIV1Write(t *testing.T, records []pb.TimeSeries, _ QueryOpts)

PrometheusAPIV1Write is a test helper function that inserts a collection of records in Prometheus remote-write format by sending a HTTP POST request to /prometheus/api/v1/write vmsingle endpoint.

func (Vmsingle) Stop added in v1.97.12 

func (app Vmsingle) Stop()

stop sends the app process a SIGINT signal and waits until it terminates gracefully.

func (*Vmsingle) String added in v1.97.12 

func (app *Vmsingle) String() string

String returns the string representation of the vmsingle app state.

type Vmstorage 

type Vmstorage struct {
	*ServesMetrics
	// contains filtered or unexported fields
}

Vmstorage holds the state of a vmstorage app and provides vmstorage-specific functions.

func StartVmstorage 

func StartVmstorage(instance string, flags []string, cli *Client) (*Vmstorage, error)

StartVmstorage starts an instance of vmstorage with the given flags. It also sets the default flags and populates the app instance state with runtime values extracted from the application log (such as httpListenAddr)

func (*Vmstorage) ForceFlush added in v1.97.12 

func (app *Vmstorage) ForceFlush(t *testing.T)

ForceFlush is a test helper function that forces the flushing of inserted data, so it becomes available for searching immediately.

func (Vmstorage) Name added in v1.97.14 

func (app Vmstorage) Name() string

Name returns the application instance name.

func (Vmstorage) Stop 

func (app Vmstorage) Stop()

stop sends the app process a SIGINT signal and waits until it terminates gracefully.

func (*Vmstorage) String 

func (app *Vmstorage) String() string

String returns the string representation of the vmstorage app state.

func (*Vmstorage) VminsertAddr 

func (app *Vmstorage) VminsertAddr() string

VminsertAddr returns the address at which the vmstorage process is listening for vminsert connections.

func (*Vmstorage) VmselectAddr 

func (app *Vmstorage) VmselectAddr() string

VmselectAddr returns the address at which the vmstorage process is listening for vmselect connections.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.