confort

package module
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 24, 2022 License: MIT Imports: 31 Imported by: 0

README

Unit test with Docker containers in Go

Go Reference coverage

This project aims to use Docker containers in parallelized tests efficiently.
Package confort provides test utilities to start Docker containers and generate unique identifiers.

Features

1. Use Docker containers with go test

If you want to run a unit test which depends on Docker container, all you need to do is declare the container in the test code with confort.Confort and run go test.
The desired container will be started in the test. Also, the test code can reuse existing containers.

2. Share containers on parallelized tests

In some cases, starting a Docker container requires a certain amount of computing resources and time.
Sometimes there are several unit tests that requires the same Docker container. confort.Confort and confort command enables us to share it and make testing more efficient. And we can choose shared locking not only exclusive.

3. Avoid conflict between tests by using unique identifier generator

To efficiently use containers simultaneously from parallelized tests, it is effective to make the resource name created on the container unique for each test (e.g., database name or realm name). unique.Unique helps generating unique names.

Unit Test Example

Single package test
func TestExample(t *testing.T) {
    ctx := context.Background()
    
    // CFT_NAMESPACE=your_ci_id
    cft := confort.New(t, ctx,
        confort.WithNamespace("fallback-namespace", false),
    )
    
    // start container
    db := cft.Run(t, ctx, &confort.ContainerParams{
        Name:  "db",
        Image: "postgres:14.4-alpine3.16",
        Env: map[string]string{
            "POSTGRES_USER":     dbUser,
            "POSTGRES_PASSWORD": dbPassword,
        },
        ExposedPorts: []string{"5432/tcp"},
        Waiter:       wait.Healthy(),
    },
        // pull image if not exists
        confort.WithPullOptions(&types.ImagePullOptions{}, os.Stderr),
        // enable health check
        confort.WithContainerConfig(func(config *container.Config) {
            config.Healthcheck = &container.HealthConfig{
                Test:     []string{"CMD-SHELL", "pg_isready"},
                Interval: 5 * time.Second,
                Timeout:  3 * time.Second,
            }
        }),
    )
    
    // use container exclusively. the container will be released after the test finished
    // UseShared is also available
    ports := db.UseExclusive(t, ctx)
    addr := ports.HostPort("5432/tcp")
    // connect PostgreSQL using `addr`
	
    uniq := unique.String(12)
    schema := uniq.Must(t)
    // create a schema using `schema` as its name
}
Parallelized test with confort command
func TestExample(t *testing.T) {
    ctx := context.Background()

    // Connect beacon server using an address from `.confort.lock` or CFT_BEACON_ADDR.
    // The function does not fail even if the beacon server is not enabled. But beacon.Enabled == false.
    beacon := confort.ConnectBeacon(t, ctx)

    // CFT_NAMESPACE=your_ci_id
    cft := confort.New(t, ctx,
        confort.WithNamespace("fallback-namespace", false),
        // Following line enables an integration with `confort` command.
        // Exclusion control is performed through the beacon server.
        confort.WithBeacon(beacon),
    )

    // ...

    unique := unique.String(12,
        // Following line enables an integration with `confort` command. 
        // This stores the values created across the entire test
        // and helps create unique one.
        unique.WithBeacon(beacon, "schema"), 
    )
    // ...
}

Run test

Unit test of a package
$ go test .
Unit tests of all packages recursively
$ confort test -- ./...

confort test command launches beacon server that helps exclusion control of containers in parallelized tests and run test. Command line arguments after "--" are passed to go test. The go command is appropriately selected according to "go.mod" using gocmd.
After tests finished, all created resources will be removed (removal policy is configurable with option "-policy").

In your CI script

Short example of .gitlab-ci.yml:

variables:
  CONFORT: github.com/daichitakahashi/confort/cmd/confort
  CFT_NAMESPACE: $CI_JOB_ID # use job id to avoid conflict with other tests
test:
  script:
    - go run $CONFORT start & # launch beacon server
    - go test ./... # run test using beacon server
  after_script:
    - go run $CONFORT stop # cleanup created Docker resources and stop beacon server safely

Off course, you can also use confort test command.

Detail of confort command

The functionality of this package consists of Go package confort and command confort. These are communicating with each other in gRPC protocol, and each version should be matched.

To avoid version mismatches, "go run" is recommended instead of "go install".

confort test

Start the beacon server and execute tests.
After the tests are finished, the beacon server will be stopped automatically.
If you want to use options of "go test", put them after "--".

There are following options.

-go=<go version>

Specify go version that runs tests. "-go=mod" enables to use go version written in your go.mod.

-go-mode=<mode>

Specify detection mode of -go option. Default value is "fallback".

  • "exact" finds go command that has the exact same version as given in "-go"
  • "latest" finds the latest go command that has the same major version as given in "-go"
  • "fallback" behaves like "latest", but if no command was found, fallbacks to "go" command
-namespace=<namespace>

Specify the namespace(prefix) of docker resources created by confort.Confort. The value is set as CFT_NAMESPACE.

-policy=<policy>

Specify resource handling policy. The value is set as CFT_RESOURCE_POLICY. Default value is "reuse".

  • With "error", the existing same resource(network and container) makes test failed
  • With "reuse", tests reuse resources if already exist
  • "reusable" is similar to "reuse", but created resources will not be removed after the tests finished
  • "takeover" is also similar to "reuse", but reused resources will be removed after the tests
confort start

Start the beacon server and output its endpoint to the lock file(".confort.lock"). If the lock file already exists, this command fails.
See the document of confort.ConnectBeacon.

There is a following option.

-lock-file=<filename>

Specify the user-defined filename of the lock file. Default value is ".confort.lock".
With this option, to tell the endpoint to the test code, you have to set file name as environment variable CFT_LOCKFILE. If CFT_LOCKFILE is already set, the command uses the value as default.

confort stop

Stop the beacon server started by confort start command.
The target server address will be read from lock file(".confort.lock"), and the lock file will be removed. If "confort start" has accompanied by "-lock-file" option, this command requires the same.

There is a following option.

-lock-file=<filename>

Specify the user-defined filename of the lock file. It is the same as the -lock-file option of confort start.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Backend 

type Backend interface {
	Namespace(ctx context.Context, namespace string) (Namespace, error)
	BuildImage(ctx context.Context, buildContext io.Reader, buildOptions types.ImageBuildOptions, force bool, buildOut io.Writer) error
}

type BuildOption 

type BuildOption interface {
	option.Interface
	// contains filtered or unexported methods
}

func WithBuildOutput 

func WithBuildOutput(dst io.Writer) BuildOption

WithBuildOutput sets dst that the output during build will be written.

func WithForceBuild 

func WithForceBuild() BuildOption

WithForceBuild forces to build an image even if it already exists.

func WithImageBuildOptions 

func WithImageBuildOptions(f func(option *types.ImageBuildOptions)) BuildOption

WithImageBuildOptions modifies the configuration of build. The argument `option` already contains required values, according to Build.

type BuildParams added in v0.3.0 

type BuildParams struct {
	Image      string
	Dockerfile string
	ContextDir string
	BuildArgs  map[string]*string
	Platform   string
}

type Confort 

type Confort struct {
	// contains filtered or unexported fields
}

func New 

func New(tb testing.TB, ctx context.Context, opts ...NewOption) *Confort

New creates Confort instance which is an interface of controlling containers. Confort creates docker resources like a network and containers. Also, it provides an exclusion control of container usage.

If you want to control the same containers across parallelized tests, enable integration with the beacon server by using `confort` command and WithBeacon option.

func (*Confort) Build 

func (cft *Confort) Build(tb testing.TB, ctx context.Context, b *BuildParams, opts ...BuildOption)

Build creates new image from given Dockerfile and context directory.

When same name image already exists, it doesn't perform building. WithForceBuild enables us to build image on every call of Build.

func (*Confort) LazyRun 

func (cft *Confort) LazyRun(tb testing.TB, ctx context.Context, c *ContainerParams, opts ...RunOption) *Container

LazyRun creates container but doesn't start. When container is required by UseShared or UseExclusive, the container starts.

If container is already created/started by other test or process, LazyRun just store container info. It makes no error.

func (*Confort) Namespace 

func (cft *Confort) Namespace() string

Namespace returns namespace associated with cft.

func (*Confort) Network added in v0.2.0 

func (cft *Confort) Network() *types.NetworkResource

Network returns docker network representation associated with Confort.

func (*Confort) Run 

func (cft *Confort) Run(tb testing.TB, ctx context.Context, c *ContainerParams, opts ...RunOption) *Container

Run starts container with given parameters. If container already exists and not started, it starts. It reuses already started container and its endpoint information.

When container is already existing and connected to another network, Run and other methods let the container connect to this network and create alias. For now, without specifying host port, container loses the port binding occasionally. If you want to use port binding and use a container with several network, and encounter such trouble, give it a try.

type Connection 

type Connection struct {
	Conn *grpc.ClientConn
	// contains filtered or unexported fields
}

func ConnectBeacon 

func ConnectBeacon(tb testing.TB, ctx context.Context) *Connection

ConnectBeacon tries to connect beacon server and returns its result. The address of server will be read from CFT_BEACON_ADDR or lock file specified as CFT_LOCKFILE.

With `confort test` command

This command starts beacon server and sets the address as CFT_BEACON_ADDR automatically.

With `confort start` command

This command starts beacon server and creates a lock file that contains the address. The default filename is ".confort.lock" and you don't need to set the file name as CFT_LOCKFILE. If you set a custom filename with "-lock-file" option, also you have to set the file name as CFT_LOCKFILE, or you can set address that read from lock file as CFT_BEACON_ADDR.

func (*Connection) Enabled 

func (c *Connection) Enabled() bool

type Container 

type Container struct {
	// contains filtered or unexported fields
}

Container represents a created container and its controller.

func (*Container) Alias added in v0.3.0 

func (c *Container) Alias() string

Alias returns a host name of the container. The alias is valid only in a docker network created in New or attached by Confort.Run.

func (*Container) ID added in v0.3.0 

func (c *Container) ID() string

ID returns its container id.

func (*Container) Name added in v0.3.0 

func (c *Container) Name() string

Name returns an actual name of the container.

func (*Container) Use added in v0.3.0 

func (c *Container) Use(tb testing.TB, ctx context.Context, exclusive bool, opts ...UseOption) Ports

Use acquires a lock for using the container and returns its endpoint. If exclusive is true, it requires to use the container exclusively. When other tests have already acquired an exclusive or shared lock for the container, it blocks until all previous locks are released.

func (*Container) UseExclusive added in v0.3.0 

func (c *Container) UseExclusive(tb testing.TB, ctx context.Context, opts ...UseOption) Ports

UseExclusive acquires an exclusive lock for using the container explicitly and returns its endpoint.

func (*Container) UseShared added in v0.3.0 

func (c *Container) UseShared(tb testing.TB, ctx context.Context, opts ...UseOption) Ports

UseShared acquires a shared lock for using the container explicitly and returns its endpoint.

type ContainerParams added in v0.3.0 

type ContainerParams struct {
	Name         string
	Image        string
	Env          map[string]string
	Cmd          []string
	Entrypoint   []string
	ExposedPorts []string
	Waiter       *wait.Waiter
}

type InitFunc 

type InitFunc func(ctx context.Context, ports Ports) error

type Namespace 

type Namespace interface {
	Namespace() string
	Network() *types.NetworkResource

	CreateContainer(ctx context.Context, name string, container *container.Config, host *container.HostConfig,
		network *network.NetworkingConfig, configConsistency bool,
		wait *wait.Waiter, pullOptions *types.ImagePullOptions, pullOut io.Writer) (string, error)
	StartContainer(ctx context.Context, name string) (Ports, error)
	Release(ctx context.Context) error
}

type NewOption 

type NewOption interface {
	option.Interface
	// contains filtered or unexported methods
}

func WithBeacon 

func WithBeacon(conn *Connection) NewOption

WithBeacon configures Confort to integrate with a starting beacon server. The beacon server is started by the "confort" command. Use Connection object given from ConnectBeacon as the argument conn.

For detail, see ConnectBeacon and "confort help".

func WithClientOptions 

func WithClientOptions(opts ...client.Opt) NewOption

WithClientOptions sets options for Docker API client. Default option is client.FromEnv. For detail, see client.NewClientWithOpts.

func WithDefaultTimeout 

func WithDefaultTimeout(d time.Duration) NewOption

WithDefaultTimeout sets the default timeout for each request to the Docker API and beacon server. The default value of the "default timeout" is 1 min. If default timeout is 0, Confort doesn't apply any timeout for ctx.

If a timeout has already been set to ctx, the default timeout is not applied.

func WithNamespace 

func WithNamespace(namespace string, force bool) NewOption

WithNamespace specifies namespace of Confort. Default namespace is the value of the CFT_NAMESPACE environment variable. The "confort test" command has "-namespace" option that overrides the variable. If force is true, the value of the argument namespace takes precedence.

If neither CFT_NAMESPACE nor WithNamespace is set, New fails.

func WithResourcePolicy 

func WithResourcePolicy(s ResourcePolicy) NewOption

WithResourcePolicy overrides the policy for handling Docker resources that already exist, such as containers and networks. By default, ResourcePolicyReuse or the value of the CFT_RESOURCE_POLICY environment variable, if set, is used. The "confort test" command has "-policy" option that overrides the variable.

func WithTerminateFunc added in v0.2.0 

func WithTerminateFunc(f *func()) NewOption

WithTerminateFunc extracts the function to release all resources created by Confort. With this option, the responsibility for termination is passed to the user.

type Ports 

type Ports nat.PortMap

func (Ports) Binding 

func (p Ports) Binding(port nat.Port) (b nat.PortBinding)

Binding returns the first value associated with the given container port. If there are no values associated with the port, Binding returns zero value. To access multiple values, use the nat.PortMap directly.

func (Ports) HostPort added in v0.2.0 

func (p Ports) HostPort(port nat.Port) string

HostPort returns "host:port" style string of the first value associated with the given container port. If there are no values associated with the port, HostPort returns empty string.

func (Ports) URL added in v0.2.0 

func (p Ports) URL(port nat.Port, scheme string) string

URL returns "scheme://host:port" style string of the first value associated with the given container port. If there are no values associated with the port, URL returns empty string. And if scheme is empty, use "http" as a default scheme.

type ResourcePolicy 

type ResourcePolicy string
const (
	ResourcePolicyError    ResourcePolicy = beaconutil.ResourcePolicyError
	ResourcePolicyReuse    ResourcePolicy = beaconutil.ResourcePolicyReuse
	ResourcePolicyReusable ResourcePolicy = beaconutil.ResourcePolicyReusable
	ResourcePolicyTakeOver ResourcePolicy = beaconutil.ResourcePolicyTakeOver
)

type RunOption 

type RunOption interface {
	option.Interface
	// contains filtered or unexported methods
}

func WithConfigConsistency 

func WithConfigConsistency(check bool) RunOption

WithConfigConsistency enables/disables the test checking consistency of configurations. By default, this test is enabled. NOTICE: This is quite experimental feature.

func WithContainerConfig 

func WithContainerConfig(f func(config *container.Config)) RunOption

WithContainerConfig modifies the configuration of container. The argument `config` already contains required values to create container, apply your values with care.

func WithHostConfig 

func WithHostConfig(f func(config *container.HostConfig)) RunOption

WithHostConfig modifies the configuration of container from host side. The argument `config` already contains required values to create container, apply your values with care.

func WithNetworkingConfig 

func WithNetworkingConfig(f func(config *network.NetworkingConfig)) RunOption

WithNetworkingConfig modifies the configuration of network. The argument `config` already contains required values to connecting to bridge network, and a container cannot join multi-networks on container creation.

func WithPullOptions 

func WithPullOptions(opts *types.ImagePullOptions, out io.Writer) RunOption

WithPullOptions enables to pull image that not exists. For example, if you want to use an image hosted in private repository, you have to fill RegistryAuth field.

The output will be written to `out`. If nil, io.Discard will be used.

type UseOption 

type UseOption interface {
	option.Interface
	// contains filtered or unexported methods
}

func WithInitFunc 

func WithInitFunc(init InitFunc) UseOption

WithInitFunc sets initializer to set up container using the given port. The init will be performed only once per container, executed with an exclusive lock. If you use a container with Confort.UseShared, the lock state is downgraded to the shared lock after init.

The returned error makes the acquired lock released and testing.TB fail. After that, you can attempt to use the container and init again.

func WithReleaseFunc 

func WithReleaseFunc(f *func()) UseOption

WithReleaseFunc extracts the function to release lock of the container. With this option, the responsibility for releasing the container is passed to the user.

Source Files

View all Source files

Directories

Path Synopsis
cmd
confort
integrationtest
container
database
internal
beaconutil
cmd
exclusion
mock
proto
beacon

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.