tests

package
v1.7.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 26, 2019 License: MIT Imports: 20 Imported by: 0

README

Server Integration Tests

This directory contains integration tests for the database.

To run them using an in-process local server:

go test ./tests

They can also be run against a remote server running in a separate process or machine

URL=http://127.0.0.1:8086 go test -parallel 1 ./tests

When running tests against a remote server, -parallel 1 is currently needed as many of the tests use the same DB and RP names which causes tests to fail when run concurrently.

When adding tests, try to add tests that will always work for remote server usage.

Structure

Currently, the file server_test.go has integration tests for single node scenarios. At some point we'll need to add cluster tests, and may add them in a different file, or rename server_test.go to server_single_node_test.go or something like that.

What is in a test?

Each test is broken apart effectively into the following areas:

  • Write sample data
  • Use cases for table driven test, that include a command (typically a query) and an expected result.

When each test runs it does the following:

  • init: determines if there are any writes and if so, writes them to the in-memory database
  • queries: iterate through each query, executing the command, and comparing the results to the expected result.

Idempotent - Allows for parallel tests

Each test should be idempotent, meaning that its data will not be affected by other tests, or use cases within the table tests themselves. This allows for parallel testing, keeping the test suite total execution time very low.

Basic sample test
// Ensure the server can have a database with multiple measurements.
func TestServer_Query_Multiple_Measurements(t *testing.T) {
    t.Parallel()
    s := OpenServer(NewConfig(), "")
    defer s.Close()

    if err := s.CreateDatabaseAndRetentionPolicy("db0", newRetentionPolicyInfo("rp0", 1, 1*time.Hour)); err != nil {
        t.Fatal(err)
    }

    // Make sure we do writes for measurements that will span across shards
    writes := []string{
        fmt.Sprintf("cpu,host=server01 value=100,core=4 %d", mustParseTime(time.RFC3339Nano, "2000-01-01T00:00:00Z").UnixNano()),
        fmt.Sprintf("cpu1,host=server02 value=50,core=2 %d", mustParseTime(time.RFC3339Nano, "2015-01-01T00:00:00Z").UnixNano()),
    }
    test := NewTest("db0", "rp0")
    test.write = strings.Join(writes, "\n")

    test.addQueries([]*Query{
        &Query{
            name:    "measurement in one shard but not another shouldn't panic server",
            command: `SELECT host,value  FROM db0.rp0.cpu`,
            exp:     `{"results":[{"series":[{"name":"cpu","tags":{"host":"server01"},"columns":["time","value"],"values":[["2000-01-01T00:00:00Z",100]]}]}]}`,
        },
    }...)

    if err := test.init(s); err != nil {
        t.Fatalf("test init failed: %s", err)
    }

    for _, query := range test.queries {
        if query.skip {
            t.Logf("SKIP:: %s", query.name)
            continue
        }
        if err := query.Execute(s); err != nil {
            t.Error(query.Error(err))
        } else if !query.success() {
            t.Error(query.failureMessage())
        }
    }
}

Let's break this down:

In this test, we first tell it to run in parallel with the t.Parallel() call.

We then open a new server with:

s := OpenServer(NewConfig(), "")
defer s.Close()

If needed, we create a database and default retention policy. This is usually needed when inserting and querying data. This is not needed if you are testing commands like CREATE DATABASE, SHOW DIAGNOSTICS, etc.

if err := s.CreateDatabaseAndRetentionPolicy("db0", newRetentionPolicyInfo("rp0", 1, 1*time.Hour)); err != nil {
    t.Fatal(err)
}

Next, set up the write data you need:

writes := []string{
    fmt.Sprintf("cpu,host=server01 value=100,core=4 %d", mustParseTime(time.RFC3339Nano, "2000-01-01T00:00:00Z").UnixNano()),
    fmt.Sprintf("cpu1,host=server02 value=50,core=2 %d", mustParseTime(time.RFC3339Nano, "2015-01-01T00:00:00Z").UnixNano()),
}

Create a new test with the database and retention policy:

test := NewTest("db0", "rp0")

Send in the writes:

test.write = strings.Join(writes, "\n")

Add some queries (the second one is mocked out to show how to add more than one):

test.addQueries([]*Query{
    &Query{
        name:    "measurement in one shard but not another shouldn't panic server",
        command: `SELECT host,value  FROM db0.rp0.cpu`,
        exp:     `{"results":[{"series":[{"name":"cpu","tags":{"host":"server01"},"columns":["time","value"],"values":[["2000-01-01T00:00:00Z",100]]}]}]}`,
    },
    &Query{
        name:    "another test here...",
        command: `Some query command`,
        exp:     `the expected results`,
    },
}...)

The rest of the code is boilerplate execution code. It is purposefully not refactored out to a helper to make sure the test failure reports the proper lines for debugging purposes.

Running the tests

To run the tests:

go test ./cmd/influxd/run -parallel 500 -timeout 10s
Running a specific test
go test ./cmd/influxd/run -parallel 500 -timeout 10s -run TestServer_Query_Fill
Verbose feedback

By default, all logs are silenced when testing. If you pass in the -v flag, the test suite becomes verbose, and enables all logging in the system

go test ./cmd/influxd/run -parallel 500 -timeout 10s -run TestServer_Query_Fill -v

Documentation

Overview

This package is a set of convenience helpers and structs to make integration testing easier

Index

Constants

This section is empty.

Variables

View Source
var LosAngeles = mustParseLocation("America/Los_Angeles")

Functions

func MustReadAll

func MustReadAll(r io.Reader) []byte

MustReadAll reads r. Panic on error.

func NewRetentionPolicySpec added in v1.5.0

func NewRetentionPolicySpec(name string, rf int, duration time.Duration) *meta.RetentionPolicySpec

form a correct retention policy given name, replication factor and duration

func RemoteEnabled

func RemoteEnabled() bool

Types

type Config added in v1.5.0

type Config struct {
	*run.Config
	// contains filtered or unexported fields
}

Config is a test wrapper around a run.Config. It also contains a root temp directory, making cleanup easier.

func NewConfig

func NewConfig() *Config

NewConfig returns the default config with temporary paths.

type LocalServer

type LocalServer struct {
	*run.Server

	Config *Config
	// contains filtered or unexported fields
}

LocalServer is a Server that is running in-process and can be accessed directly

func (*LocalServer) Close

func (s *LocalServer) Close()

Close shuts down the server and removes all temporary paths.

func (*LocalServer) Closed

func (s *LocalServer) Closed() bool

func (*LocalServer) CreateDatabase

func (s *LocalServer) CreateDatabase(db string) (*meta.DatabaseInfo, error)

func (*LocalServer) CreateDatabaseAndRetentionPolicy

func (s *LocalServer) CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error

CreateDatabaseAndRetentionPolicy will create the database and retention policy.

func (*LocalServer) CreateSubscription

func (s *LocalServer) CreateSubscription(database, rp, name, mode string, destinations []string) error

func (*LocalServer) DropDatabase

func (s *LocalServer) DropDatabase(db string) error

func (LocalServer) HTTPGet

func (s LocalServer) HTTPGet(url string) (results string, err error)

HTTPGet makes an HTTP GET request to the server and returns the response.

func (LocalServer) HTTPPost

func (s LocalServer) HTTPPost(url string, content []byte) (results string, err error)

HTTPPost makes an HTTP POST request to the server and returns the response.

func (LocalServer) MustQuery

func (s LocalServer) MustQuery(query string) string

MustQuery executes a query against the server and returns the results.

func (LocalServer) MustQueryWithParams

func (s LocalServer) MustQueryWithParams(query string, values url.Values) string

MustQueryWithParams executes a query against the server and returns the results.

func (LocalServer) MustWrite

func (s LocalServer) MustWrite(db, rp, body string, params url.Values) string

MustWrite executes a write to the server. Panic on error.

func (*LocalServer) Open added in v1.5.0

func (s *LocalServer) Open() error

Open opens the server. If running this test on a 32-bit platform it reduces the size of series files so that they can all be addressable in the process.

func (LocalServer) Query

func (s LocalServer) Query(query string) (results string, err error)

Query executes a query against the server and returns the results.

func (LocalServer) QueryWithParams

func (s LocalServer) QueryWithParams(query string, values url.Values) (results string, err error)

Query executes a query against the server and returns the results.

func (*LocalServer) Reset

func (s *LocalServer) Reset() error

func (*LocalServer) URL

func (s *LocalServer) URL() string

URL returns the base URL for the httpd endpoint.

func (LocalServer) Write

func (s LocalServer) Write(db, rp, body string, params url.Values) (results string, err error)

Write executes a write against the server and returns the results.

func (*LocalServer) WritePoints

func (s *LocalServer) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error

type Query

type Query struct {
	// contains filtered or unexported fields
}

func (*Query) Error

func (q *Query) Error(err error) string

func (*Query) Execute

func (q *Query) Execute(s Server) (err error)

Execute runs the command and returns an err if it fails

type RemoteServer

type RemoteServer struct {
	// contains filtered or unexported fields
}

RemoteServer is a Server that is accessed remotely via the HTTP API

func (*RemoteServer) Close

func (s *RemoteServer) Close()

func (*RemoteServer) Closed

func (s *RemoteServer) Closed() bool

func (*RemoteServer) CreateDatabase

func (s *RemoteServer) CreateDatabase(db string) (*meta.DatabaseInfo, error)

func (*RemoteServer) CreateDatabaseAndRetentionPolicy

func (s *RemoteServer) CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error

func (*RemoteServer) CreateSubscription

func (s *RemoteServer) CreateSubscription(database, rp, name, mode string, destinations []string) error

func (*RemoteServer) DropDatabase

func (s *RemoteServer) DropDatabase(db string) error

func (RemoteServer) HTTPGet

func (s RemoteServer) HTTPGet(url string) (results string, err error)

HTTPGet makes an HTTP GET request to the server and returns the response.

func (RemoteServer) HTTPPost

func (s RemoteServer) HTTPPost(url string, content []byte) (results string, err error)

HTTPPost makes an HTTP POST request to the server and returns the response.

func (RemoteServer) MustQuery

func (s RemoteServer) MustQuery(query string) string

MustQuery executes a query against the server and returns the results.

func (RemoteServer) MustQueryWithParams

func (s RemoteServer) MustQueryWithParams(query string, values url.Values) string

MustQueryWithParams executes a query against the server and returns the results.

func (RemoteServer) MustWrite

func (s RemoteServer) MustWrite(db, rp, body string, params url.Values) string

MustWrite executes a write to the server. Panic on error.

func (*RemoteServer) Open

func (s *RemoteServer) Open() error

func (RemoteServer) Query

func (s RemoteServer) Query(query string) (results string, err error)

Query executes a query against the server and returns the results.

func (RemoteServer) QueryWithParams

func (s RemoteServer) QueryWithParams(query string, values url.Values) (results string, err error)

Query executes a query against the server and returns the results.

func (*RemoteServer) Reset

func (s *RemoteServer) Reset() error

Reset attempts to remove all database state by dropping everything

func (*RemoteServer) SetLogOutput

func (s *RemoteServer) SetLogOutput(w io.Writer)

func (*RemoteServer) URL

func (s *RemoteServer) URL() string

func (RemoteServer) Write

func (s RemoteServer) Write(db, rp, body string, params url.Values) (results string, err error)

Write executes a write against the server and returns the results.

func (*RemoteServer) WritePoints

func (s *RemoteServer) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error

type Server

type Server interface {
	URL() string
	Open() error
	SetLogOutput(w io.Writer)
	Close()
	Closed() bool

	CreateDatabase(db string) (*meta.DatabaseInfo, error)
	CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error
	CreateSubscription(database, rp, name, mode string, destinations []string) error
	DropDatabase(db string) error
	Reset() error

	Query(query string) (results string, err error)
	QueryWithParams(query string, values url.Values) (results string, err error)

	Write(db, rp, body string, params url.Values) (results string, err error)
	MustWrite(db, rp, body string, params url.Values) string
	WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error
}

Server represents a test wrapper for run.Server.

func NewServer

func NewServer(c *Config) Server

NewServer returns a new instance of Server.

func OpenDefaultServer

func OpenDefaultServer(c *Config) Server

OpenDefaultServer opens a test server with a default database & retention policy.

func OpenServer

func OpenServer(c *Config) Server

OpenServer opens a test server.

func OpenServerWithVersion

func OpenServerWithVersion(c *Config, version string) Server

OpenServerWithVersion opens a test server with a specific version.

type Test

type Test struct {
	// contains filtered or unexported fields
}

func NewTest

func NewTest(db, rp string) Test

type Tests

type Tests map[string]Test

type Write

type Write struct {
	// contains filtered or unexported fields
}

type WriteError

type WriteError struct {
	// contains filtered or unexported fields
}

func (WriteError) Body

func (wr WriteError) Body() string

func (WriteError) Error

func (wr WriteError) Error() string

func (WriteError) StatusCode

func (wr WriteError) StatusCode() int

type Writes

type Writes []*Write

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL