tests

package
v1.7.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 26, 2019 License: MIT Imports: 20 Imported by: 0

README

Server Integration Tests

This directory contains integration tests for the database.

To run them using an in-process local server:

go test ./tests

They can also be run against a remote server running in a separate process or machine

URL=http://127.0.0.1:8086 go test -parallel 1 ./tests

When running tests against a remote server, -parallel 1 is currently needed as many of the tests use the same DB and RP names which causes tests to fail when run concurrently.

When adding tests, try to add tests that will always work for remote server usage.

Structure

Currently, the file server_test.go has integration tests for single node scenarios. At some point we'll need to add cluster tests, and may add them in a different file, or rename server_test.go to server_single_node_test.go or something like that.

What is in a test?

Each test is broken apart effectively into the following areas:

  • Write sample data
  • Use cases for table driven test, that include a command (typically a query) and an expected result.

When each test runs it does the following:

  • init: determines if there are any writes and if so, writes them to the in-memory database
  • queries: iterate through each query, executing the command, and comparing the results to the expected result.

Idempotent - Allows for parallel tests

Each test should be idempotent, meaning that its data will not be affected by other tests, or use cases within the table tests themselves. This allows for parallel testing, keeping the test suite total execution time very low.

Basic sample test
// Ensure the server can have a database with multiple measurements.
func TestServer_Query_Multiple_Measurements(t *testing.T) {
    t.Parallel()
    s := OpenServer(NewConfig(), "")
    defer s.Close()

    if err := s.CreateDatabaseAndRetentionPolicy("db0", newRetentionPolicyInfo("rp0", 1, 1*time.Hour)); err != nil {
        t.Fatal(err)
    }

    // Make sure we do writes for measurements that will span across shards
    writes := []string{
        fmt.Sprintf("cpu,host=server01 value=100,core=4 %d", mustParseTime(time.RFC3339Nano, "2000-01-01T00:00:00Z").UnixNano()),
        fmt.Sprintf("cpu1,host=server02 value=50,core=2 %d", mustParseTime(time.RFC3339Nano, "2015-01-01T00:00:00Z").UnixNano()),
    }
    test := NewTest("db0", "rp0")
    test.write = strings.Join(writes, "\n")

    test.addQueries([]*Query{
        &Query{
            name:    "measurement in one shard but not another shouldn't panic server",
            command: `SELECT host,value  FROM db0.rp0.cpu`,
            exp:     `{"results":[{"series":[{"name":"cpu","tags":{"host":"server01"},"columns":["time","value"],"values":[["2000-01-01T00:00:00Z",100]]}]}]}`,
        },
    }...)

    if err := test.init(s); err != nil {
        t.Fatalf("test init failed: %s", err)
    }

    for _, query := range test.queries {
        if query.skip {
            t.Logf("SKIP:: %s", query.name)
            continue
        }
        if err := query.Execute(s); err != nil {
            t.Error(query.Error(err))
        } else if !query.success() {
            t.Error(query.failureMessage())
        }
    }
}

Let's break this down:

In this test, we first tell it to run in parallel with the t.Parallel() call.

We then open a new server with:

s := OpenServer(NewConfig(), "")
defer s.Close()

If needed, we create a database and default retention policy. This is usually needed when inserting and querying data. This is not needed if you are testing commands like CREATE DATABASE, SHOW DIAGNOSTICS, etc.

if err := s.CreateDatabaseAndRetentionPolicy("db0", newRetentionPolicyInfo("rp0", 1, 1*time.Hour)); err != nil {
    t.Fatal(err)
}

Next, set up the write data you need:

writes := []string{
    fmt.Sprintf("cpu,host=server01 value=100,core=4 %d", mustParseTime(time.RFC3339Nano, "2000-01-01T00:00:00Z").UnixNano()),
    fmt.Sprintf("cpu1,host=server02 value=50,core=2 %d", mustParseTime(time.RFC3339Nano, "2015-01-01T00:00:00Z").UnixNano()),
}

Create a new test with the database and retention policy:

test := NewTest("db0", "rp0")

Send in the writes:

test.write = strings.Join(writes, "\n")

Add some queries (the second one is mocked out to show how to add more than one):

test.addQueries([]*Query{
    &Query{
        name:    "measurement in one shard but not another shouldn't panic server",
        command: `SELECT host,value  FROM db0.rp0.cpu`,
        exp:     `{"results":[{"series":[{"name":"cpu","tags":{"host":"server01"},"columns":["time","value"],"values":[["2000-01-01T00:00:00Z",100]]}]}]}`,
    },
    &Query{
        name:    "another test here...",
        command: `Some query command`,
        exp:     `the expected results`,
    },
}...)

The rest of the code is boilerplate execution code. It is purposefully not refactored out to a helper to make sure the test failure reports the proper lines for debugging purposes.

Running the tests

To run the tests:

go test ./cmd/influxd/run -parallel 500 -timeout 10s
Running a specific test
go test ./cmd/influxd/run -parallel 500 -timeout 10s -run TestServer_Query_Fill
Verbose feedback

By default, all logs are silenced when testing. If you pass in the -v flag, the test suite becomes verbose, and enables all logging in the system

go test ./cmd/influxd/run -parallel 500 -timeout 10s -run TestServer_Query_Fill -v

Documentation

Overview

This package is a set of convenience helpers and structs to make integration testing easier

Index

Constants

This section is empty.

Variables

View Source 
var LosAngeles = mustParseLocation("America/Los_Angeles")

Functions

func MustReadAll 

func MustReadAll(r io.Reader) []byte

MustReadAll reads r. Panic on error.

func NewRetentionPolicySpec added in v1.5.0 

func NewRetentionPolicySpec(name string, rf int, duration time.Duration) *meta.RetentionPolicySpec

form a correct retention policy given name, replication factor and duration

func RemoteEnabled 

func RemoteEnabled() bool

Types

type Config added in v1.5.0 

type Config struct {
	*run.Config
	// contains filtered or unexported fields
}

Config is a test wrapper around a run.Config. It also contains a root temp directory, making cleanup easier.

func NewConfig 

func NewConfig() *Config

NewConfig returns the default config with temporary paths.

type LocalServer 

type LocalServer struct {
	*run.Server

	Config *Config
	// contains filtered or unexported fields
}

LocalServer is a Server that is running in-process and can be accessed directly

func (*LocalServer) Close 

func (s *LocalServer) Close()

Close shuts down the server and removes all temporary paths.

func (*LocalServer) Closed 

func (s *LocalServer) Closed() bool

func (*LocalServer) CreateDatabase 

func (s *LocalServer) CreateDatabase(db string) (*meta.DatabaseInfo, error)

func (*LocalServer) CreateDatabaseAndRetentionPolicy 

func (s *LocalServer) CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error

CreateDatabaseAndRetentionPolicy will create the database and retention policy.

func (*LocalServer) CreateSubscription 

func (s *LocalServer) CreateSubscription(database, rp, name, mode string, destinations []string) error

func (*LocalServer) DropDatabase 

func (s *LocalServer) DropDatabase(db string) error

func (LocalServer) HTTPGet 

func (s LocalServer) HTTPGet(url string) (results string, err error)

HTTPGet makes an HTTP GET request to the server and returns the response.

func (LocalServer) HTTPPost 

func (s LocalServer) HTTPPost(url string, content []byte) (results string, err error)

HTTPPost makes an HTTP POST request to the server and returns the response.

func (LocalServer) MustQuery 

func (s LocalServer) MustQuery(query string) string

MustQuery executes a query against the server and returns the results.

func (LocalServer) MustQueryWithParams 

func (s LocalServer) MustQueryWithParams(query string, values url.Values) string

MustQueryWithParams executes a query against the server and returns the results.

func (LocalServer) MustWrite 

func (s LocalServer) MustWrite(db, rp, body string, params url.Values) string

MustWrite executes a write to the server. Panic on error.

func (*LocalServer) Open added in v1.5.0 

func (s *LocalServer) Open() error

Open opens the server. If running this test on a 32-bit platform it reduces the size of series files so that they can all be addressable in the process.

func (LocalServer) Query 

func (s LocalServer) Query(query string) (results string, err error)

Query executes a query against the server and returns the results.

func (LocalServer) QueryWithParams 

func (s LocalServer) QueryWithParams(query string, values url.Values) (results string, err error)

Query executes a query against the server and returns the results.

func (*LocalServer) Reset 

func (s *LocalServer) Reset() error

func (*LocalServer) URL 

func (s *LocalServer) URL() string

URL returns the base URL for the httpd endpoint.

func (LocalServer) Write 

func (s LocalServer) Write(db, rp, body string, params url.Values) (results string, err error)

Write executes a write against the server and returns the results.

func (*LocalServer) WritePoints 

func (s *LocalServer) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error

type Query 

type Query struct {
	// contains filtered or unexported fields
}

func (*Query) Error 

func (q *Query) Error(err error) string

func (*Query) Execute 

func (q *Query) Execute(s Server) (err error)

Execute runs the command and returns an err if it fails

type RemoteServer 

type RemoteServer struct {
	// contains filtered or unexported fields
}

RemoteServer is a Server that is accessed remotely via the HTTP API

func (*RemoteServer) Close 

func (s *RemoteServer) Close()

func (*RemoteServer) Closed 

func (s *RemoteServer) Closed() bool

func (*RemoteServer) CreateDatabase 

func (s *RemoteServer) CreateDatabase(db string) (*meta.DatabaseInfo, error)

func (*RemoteServer) CreateDatabaseAndRetentionPolicy 

func (s *RemoteServer) CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error

func (*RemoteServer) CreateSubscription 

func (s *RemoteServer) CreateSubscription(database, rp, name, mode string, destinations []string) error

func (*RemoteServer) DropDatabase 

func (s *RemoteServer) DropDatabase(db string) error

func (RemoteServer) HTTPGet 

func (s RemoteServer) HTTPGet(url string) (results string, err error)

HTTPGet makes an HTTP GET request to the server and returns the response.

func (RemoteServer) HTTPPost 

func (s RemoteServer) HTTPPost(url string, content []byte) (results string, err error)

HTTPPost makes an HTTP POST request to the server and returns the response.

func (RemoteServer) MustQuery 

func (s RemoteServer) MustQuery(query string) string

MustQuery executes a query against the server and returns the results.

func (RemoteServer) MustQueryWithParams 

func (s RemoteServer) MustQueryWithParams(query string, values url.Values) string

MustQueryWithParams executes a query against the server and returns the results.

func (RemoteServer) MustWrite 

func (s RemoteServer) MustWrite(db, rp, body string, params url.Values) string

MustWrite executes a write to the server. Panic on error.

func (*RemoteServer) Open 

func (s *RemoteServer) Open() error

func (RemoteServer) Query 

func (s RemoteServer) Query(query string) (results string, err error)

Query executes a query against the server and returns the results.

func (RemoteServer) QueryWithParams 

func (s RemoteServer) QueryWithParams(query string, values url.Values) (results string, err error)

Query executes a query against the server and returns the results.

func (*RemoteServer) Reset 

func (s *RemoteServer) Reset() error

Reset attempts to remove all database state by dropping everything

func (*RemoteServer) SetLogOutput 

func (s *RemoteServer) SetLogOutput(w io.Writer)

func (*RemoteServer) URL 

func (s *RemoteServer) URL() string

func (RemoteServer) Write 

func (s RemoteServer) Write(db, rp, body string, params url.Values) (results string, err error)

Write executes a write against the server and returns the results.

func (*RemoteServer) WritePoints 

func (s *RemoteServer) WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error

type Server 

type Server interface {
	URL() string
	Open() error
	SetLogOutput(w io.Writer)
	Close()
	Closed() bool

	CreateDatabase(db string) (*meta.DatabaseInfo, error)
	CreateDatabaseAndRetentionPolicy(db string, rp *meta.RetentionPolicySpec, makeDefault bool) error
	CreateSubscription(database, rp, name, mode string, destinations []string) error
	DropDatabase(db string) error
	Reset() error

	Query(query string) (results string, err error)
	QueryWithParams(query string, values url.Values) (results string, err error)

	Write(db, rp, body string, params url.Values) (results string, err error)
	MustWrite(db, rp, body string, params url.Values) string
	WritePoints(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, user meta.User, points []models.Point) error
}

Server represents a test wrapper for run.Server.

func NewServer 

func NewServer(c *Config) Server

NewServer returns a new instance of Server.

func OpenDefaultServer 

func OpenDefaultServer(c *Config) Server

OpenDefaultServer opens a test server with a default database & retention policy.

func OpenServer 

func OpenServer(c *Config) Server

OpenServer opens a test server.

func OpenServerWithVersion 

func OpenServerWithVersion(c *Config, version string) Server

OpenServerWithVersion opens a test server with a specific version.

type Test 

type Test struct {
	// contains filtered or unexported fields
}

func NewTest 

func NewTest(db, rp string) Test

type Tests 

type Tests map[string]Test

type Write 

type Write struct {
	// contains filtered or unexported fields
}

type WriteError 

type WriteError struct {
	// contains filtered or unexported fields
}

func (WriteError) Body 

func (wr WriteError) Body() string

func (WriteError) Error 

func (wr WriteError) Error() string

func (WriteError) StatusCode 

func (wr WriteError) StatusCode() int

type Writes 

type Writes []*Write

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.