dockerutil

package
v0.0.0-...-afa323b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2024 License: Apache-2.0, MIT Imports: 33 Imported by: 19

README

dockerutil

This package is for creating and controlling docker containers for testing runsc, gVisor's docker/kubernetes binary. A simple test may look like:

 func TestSuperCool(t *testing.T) {
   ctx := context.Background()
   c := dockerutil.MakeContainer(ctx, t)
   got, err := c.Run(ctx, dockerutil.RunOpts{
     Image: "basic/alpine"
   }, "echo", "super cool")
   if err != nil {
      t.Fatalf("err was not nil: %v", err)
   }
   want := "super cool"
   if !strings.Contains(got, want){
     t.Fatalf("want: %s, got: %s", want, got)
   }
 }

For further examples, see many of our end to end tests elsewhere in the repo, such as those in //test/e2e or benchmarks at //test/benchmarks.

dockerutil uses the "official" docker golang api, which is very powerful. dockerutil is a thin wrapper around this API, allowing desired new use cases to be easily implemented.

Profiling

dockerutil is capable of generating profiles. Currently, the only option is to use pprof profiles generated by runsc debug. The profiler will generate Block, CPU, Heap, Goroutine, and Mutex profiles. To generate profiles:

  • Install runsc with the --profile flag: make configure RUNTIME=myrunsc ARGS="--profile" Also add other flags with ARGS like --platform=kvm.
  • Restart docker: sudo service docker restart

To run and generate CPU profiles run:

make sudo TARGETS=//path/to:target \
  ARGS="--runtime=myrunsc -test.v -test.bench=. --pprof-cpu" OPTIONS="-c opt"

Profiles would be at: /tmp/profile/myrunsc/CONTAINERNAME/cpu.pprof

Container name in most tests and benchmarks in gVisor is usually the test name and some random characters like so: BenchmarkABSL-CleanCache-JF2J2ZYF3U7SL47QAA727CSJI3C4ZAW2

Profiling requires root as runsc debug inspects running containers in /var/run among other things.

Writing for Profiling

The below shows an example of using profiles with dockerutil.

func TestSuperCool(t *testing.T){
  ctx := context.Background()
  // profiled and using runtime from dockerutil.runtime flag
  profiled := MakeContainer()

  // not profiled and using runtime runc
  native := MakeNativeContainer()

  err := profiled.Spawn(ctx, RunOpts{
    Image: "some/image",
  }, "sleep", "100000")
  // profiling has begun here
  ...
  expensive setup that I don't want to profile.
  ...
  profiled.RestartProfiles()
  // profiled activity
}

In the above example, profiled would be profiled and native would not. The call to RestartProfiles() restarts the clock on profiling. This is useful if the main activity being tested is done with docker exec or container.Spawn() followed by one or more container.Exec() calls.

Documentation

Overview

Package dockerutil is a collection of utility functions.

Package dockerutil provides utility functions for GPU tests.

Index

Constants

View Source 
const (
	// AllGPUCapabilities is the environment variable that enables all NVIDIA
	// GPU capabilities within a container.
	AllGPUCapabilities = "NVIDIA_DRIVER_CAPABILITIES=all"

	// DefaultGPUCapabilities is the environment variable that enables default
	// NVIDIA GPU capabilities within a container.
	DefaultGPUCapabilities = "NVIDIA_DRIVER_CAPABILITIES=compute,utility"
)

Variables

View Source 
var ErrNoIP = errors.New("no IP available")

ErrNoIP indicates that no IP address is available.

Functions

func EnsureDockerExperimentalEnabled 

func EnsureDockerExperimentalEnabled()

EnsureDockerExperimentalEnabled ensures that Docker has experimental features enabled.

func EnsureSupportedDockerVersion 

func EnsureSupportedDockerVersion()

EnsureSupportedDockerVersion checks if correct docker is installed.

This logs directly to stderr, as it is typically called from a Main wrapper.

func IsGVisorRuntime 

func IsGVisorRuntime(ctx context.Context, t *testing.T) (bool, error)

IsGVisorRuntime returns whether the default container runtime used by `dockerutil` is gVisor-based or not.

func NumGPU 

func NumGPU() int

NumGPU crudely estimates the number of NVIDIA GPUs on the host.

func PrintDockerConfig 

func PrintDockerConfig()

PrintDockerConfig prints the whole Docker configuration file to the log.

func Runtime 

func Runtime() string

Runtime returns the value of the flag runtime.

func RuntimeArgs 

func RuntimeArgs() ([]string, error)

RuntimeArgs returns the arguments for the current runtime.

func RuntimePath 

func RuntimePath() (string, error)

RuntimePath returns the binary path for the current runtime.

func Save 

func Save(logger testutil.Logger, image string, w io.Writer) error

Save exports a container image to the given Writer.

Note that the writer should be actively consuming the output, otherwise it is not guaranteed that the Save will make any progress and the call may stall indefinitely.

This is called by criutil in order to import imports.

func UsingSystemdCgroup 

func UsingSystemdCgroup() (bool, error)

UsingSystemdCgroup returns true if the docker configuration has the native.cgroupdriver=systemd option set in "exec-opts", or if the system is using cgroupv2, in which case systemd is the default driver.

Types

type Container 

type Container struct {
	Name string
	// contains filtered or unexported fields
}

Container represents a Docker Container allowing user to configure and control as one would with the 'docker' client. Container is backed by the official golang docker API. See: https://pkg.go.dev/github.com/docker/docker.

func MakeContainer 

func MakeContainer(ctx context.Context, logger testutil.Logger) *Container

MakeContainer constructs a suitable Container object.

The runtime used is determined by the runtime flag.

Containers will check flags for profiling requests.

func MakeContainerWithRuntime 

func MakeContainerWithRuntime(ctx context.Context, logger testutil.Logger, suffix string) *Container

MakeContainerWithRuntime is like MakeContainer, but allows for a runtime to be specified by suffix.

func MakeNativeContainer 

func MakeNativeContainer(ctx context.Context, logger testutil.Logger) *Container

MakeNativeContainer constructs a suitable Container object.

The runtime used will be the system default.

Native containers aren't profiled.

func (*Container) Checkpoint 

func (c *Container) Checkpoint(ctx context.Context, name string) error

Checkpoint is analogous to 'docker checkpoint'.

func (*Container) CheckpointResume 

func (c *Container) CheckpointResume(ctx context.Context, name string) error

CheckpointResume is analogous to 'docker checkpoint'.

func (*Container) CleanUp 

func (c *Container) CleanUp(ctx context.Context)

CleanUp kills and deletes the container (best effort).

func (*Container) ConfigsFrom 

func (c *Container) ConfigsFrom(ctx context.Context, r RunOpts, args ...string) (*container.Config, *container.HostConfig, *network.NetworkingConfig, error)

ConfigsFrom returns container configs from RunOpts and args. The caller should call 'CreateFrom' and Start.

func (*Container) CopyFiles 

func (c *Container) CopyFiles(opts *RunOpts, target string, sources ...string)

CopyFiles copies in and mounts the given files. They are always ReadOnly.

func (*Container) Create 

func (c *Container) Create(ctx context.Context, r RunOpts, args ...string) error

Create is analogous to 'docker create'.

func (*Container) CreateFrom 

func (c *Container) CreateFrom(ctx context.Context, profileImage string, conf *container.Config, hostconf *container.HostConfig, netconf *network.NetworkingConfig) error

CreateFrom creates a container from the given configs.

func (*Container) Exec 

func (c *Container) Exec(ctx context.Context, opts ExecOpts, args ...string) (string, error)

Exec creates a process inside the container. If the process exits with a non-zero error code, the error will be of type `ExecError`.

func (*Container) ExecProcess 

func (c *Container) ExecProcess(ctx context.Context, opts ExecOpts, args ...string) (Process, error)

ExecProcess creates a process inside the container and returns a process struct for the caller to use.

func (*Container) FindIP 

func (c *Container) FindIP(ctx context.Context, ipv6 bool) (net.IP, error)

FindIP returns the IP address of the container.

func (*Container) FindPort 

func (c *Container) FindPort(ctx context.Context, sandboxPort int) (int, error)

FindPort returns the host port that is mapped to 'sandboxPort'.

func (*Container) ID 

func (c *Container) ID() string

ID returns the container id.

func (*Container) Kill 

func (c *Container) Kill(ctx context.Context) error

Kill kills the container.

func (*Container) Logs 

func (c *Container) Logs(ctx context.Context) (string, error)

Logs is analogous 'docker logs'. 

func (c *Container) MakeLink(target string) string

MakeLink formats a link to add to a RunOpts.

func (*Container) OutputStreams 

func (c *Container) OutputStreams(ctx context.Context) (string, string, error)

OutputStreams gets the container's stdout and stderr streams separately.

func (*Container) Pause 

func (c *Container) Pause(ctx context.Context) error

Pause is analogous to 'docker pause'.

func (*Container) Remove 

func (c *Container) Remove(ctx context.Context) error

Remove is analogous to 'docker rm'.

func (*Container) Restore 

func (c *Container) Restore(ctx context.Context, name string) error

Restore is analogous to 'docker start --checkpoint [name]'.

func (*Container) RootDirectory 

func (c *Container) RootDirectory() (string, error)

RootDirectory returns an educated guess about the container's root directory.

func (*Container) Run 

func (c *Container) Run(ctx context.Context, r RunOpts, args ...string) (string, error)

Run is analogous to 'docker run'.

func (*Container) SandboxPid 

func (c *Container) SandboxPid(ctx context.Context) (int, error)

SandboxPid returns the container's pid.

func (*Container) Spawn 

func (c *Container) Spawn(ctx context.Context, r RunOpts, args ...string) error

Spawn is analogous to 'docker run -d'.

func (*Container) SpawnProcess 

func (c *Container) SpawnProcess(ctx context.Context, r RunOpts, args ...string) (Process, error)

SpawnProcess is analogous to 'docker run -it'. It returns a process which represents the root process.

func (*Container) Start 

func (c *Container) Start(ctx context.Context) error

Start is analogous to 'docker start'.

func (*Container) Stats 

func (c *Container) Stats(ctx context.Context) (*types.StatsJSON, error)

Stats returns a snapshot of container stats similar to `docker stats`.

func (*Container) Status 

func (c *Container) Status(ctx context.Context) (types.ContainerState, error)

Status inspects the container returns its status.

func (*Container) Stop 

func (c *Container) Stop(ctx context.Context) error

Stop is analogous to 'docker stop'.

func (*Container) Unpause 

func (c *Container) Unpause(ctx context.Context) error

Unpause is analogous to 'docker unpause'.

func (*Container) Wait 

func (c *Container) Wait(ctx context.Context) error

Wait waits for the container to exit.

func (*Container) WaitForOutput 

func (c *Container) WaitForOutput(ctx context.Context, pattern string, timeout time.Duration) (string, error)

WaitForOutput searches container logs for pattern and returns or timesout.

func (*Container) WaitForOutputSubmatch 

func (c *Container) WaitForOutputSubmatch(ctx context.Context, pattern string, timeout time.Duration) ([]string, error)

WaitForOutputSubmatch searches container logs for the given pattern or times out. It returns any regexp submatches as well.

func (*Container) WaitTimeout 

func (c *Container) WaitTimeout(ctx context.Context, timeout time.Duration) error

WaitTimeout waits for the container to exit with a timeout.

type ContainerPool 

type ContainerPool struct {
	// contains filtered or unexported fields
}

ContainerPool represents a pool of reusable containers. Callers may request a container from the pool, and must release it back when they are done with it.

This is useful for large tests which can `exec` individual test cases inside the same set of reusable containers, to avoid the cost of creating and destroying containers for each test.

It also supports reserving the whole pool ("exclusive"), which locks out all other callers from getting any other container from the pool. This is useful for tests where running in parallel may induce unexpected errors which running serially would not cause. This allows the test to first try to run in parallel, and then re-run failing tests exclusively to make sure their failure is not due to parallel execution.

func NewContainerPool 

func NewContainerPool(containers []*Container) *ContainerPool

NewContainerPool returns a new ContainerPool holding the given set of containers.

func (*ContainerPool) CleanUp 

func (cp *ContainerPool) CleanUp(ctx context.Context)

CleanUp waits for all containers to be back into the pool, and cleans up each container as soon as it gets back in the pool.

func (*ContainerPool) Get 

func (cp *ContainerPool) Get(ctx context.Context) (*Container, func(), error)

Get returns a free container and a function to release it back to the pool.

func (*ContainerPool) GetExclusive 

func (cp *ContainerPool) GetExclusive(ctx context.Context) (*Container, func(), error)

GetExclusive ensures all pooled containers are in the pool, reserves all of them, and returns one of them, along with a function to release them all back to the pool.

func (*ContainerPool) SetContainerLabel 

func (cp *ContainerPool) SetContainerLabel(c *Container, label string)

SetContainerLabel sets the label for a container. This is printed in `cp.String`, and wiped every time the container changes state. It is useful for the ContainerPool user to track what each container is doing.

func (*ContainerPool) String 

func (cp *ContainerPool) String() string

String returns a string representation of the pool. It includes the state of each container, and their user-set label.

func (*ContainerPool) Utilization 

func (cp *ContainerPool) Utilization() float64

Utilization returns the utilization of the pool. This is the ratio of the cumulative duration containers have been in a productive state (reserved) divided by the maximum potential productive container-duration since the first container was reserved.

type ExecError 

type ExecError struct {
	ExitStatus int
}

ExecError is returned when a process terminated with a non-zero exit status. It implements `error`.

func (*ExecError) Error 

func (ee *ExecError) Error() string

Error implements `error.Error`.

type ExecOpts 

type ExecOpts struct {
	// Env are additional environment variables.
	Env []string

	// Privileged enables privileged mode.
	Privileged bool

	// User is the user to use.
	User string

	// Enables Tty and stdin for the created process.
	UseTTY bool

	// WorkDir is the working directory of the process.
	WorkDir string

	// NoWrap, if true, indicates that no command-line wrapping may be performed
	// on the command line being exec'd. Otherwise, settings are inherited from
	// the container.
	NoWrap bool
}

ExecOpts holds arguments for Exec calls.

type Network 

type Network struct {
	Name string

	Subnet *net.IPNet
	// contains filtered or unexported fields
}

Network is a docker network.

func NewNetwork 

func NewNetwork(ctx context.Context, logger testutil.Logger) *Network

NewNetwork sets up the struct for a Docker network. Names of networks will be unique.

func (*Network) Cleanup 

func (n *Network) Cleanup(ctx context.Context) error

Cleanup cleans up the docker network.

func (*Network) Connect 

func (n *Network) Connect(ctx context.Context, container *Container, ipv4, ipv6 string) error

Connect is analogous to 'docker network connect' with the arguments provided.

func (*Network) Create 

func (n *Network) Create(ctx context.Context) error

Create is analogous to 'docker network create'.

func (*Network) Inspect 

func (n *Network) Inspect(ctx context.Context) (types.NetworkResource, error)

Inspect returns this network's info.

type Process 

type Process struct {
	// contains filtered or unexported fields
}

Process represents a containerized process.

func (*Process) ExitCode 

func (p *Process) ExitCode(ctx context.Context) (int, error)

ExitCode returns the process's exit code.

func (*Process) IsRunning 

func (p *Process) IsRunning(ctx context.Context) (bool, error)

IsRunning checks if the process is running.

func (*Process) Logs 

func (p *Process) Logs() (string, error)

Logs returns combined stdout/stderr from the process.

func (*Process) Read 

func (p *Process) Read() (string, string, error)

Read returns process's stdout and stderr.

func (*Process) WaitExitStatus 

func (p *Process) WaitExitStatus(ctx context.Context) (int, error)

WaitExitStatus until process completes and returns exit status.

func (*Process) Write 

func (p *Process) Write(timeout time.Duration, buf []byte) (int, error)

Write writes buf to the process's stdin.

type RunOpts 

type RunOpts struct {
	// Image is the image relative to images/. This will be mangled
	// appropriately, to ensure that only first-party images are used.
	Image string

	// Memory is the memory limit in bytes.
	Memory int

	// Cpus in which to allow execution. ("0", "1", "0-2").
	CpusetCpus string

	// Ports are the ports to be allocated.
	Ports []int

	// WorkDir sets the working directory.
	WorkDir string

	// ReadOnly sets the read-only flag.
	ReadOnly bool

	// Env are additional environment variables.
	Env []string

	// User is the user to use.
	User string

	// Optional argv to override the ENTRYPOINT specified in the image.
	Entrypoint []string

	// Privileged enables privileged mode.
	Privileged bool

	// Sets network mode for the container. See container.NetworkMode for types. Several options will
	// not work w/ gVisor. For example, you can't set the "sandbox" network option for gVisor using
	// this handle.
	NetworkMode string

	// CapAdd are the extra set of capabilities to add.
	CapAdd []string

	// CapDrop are the extra set of capabilities to drop.
	CapDrop []string

	// Mounts is the list of directories/files to be mounted inside the container.
	Mounts []mount.Mount

	// Links is the list of containers to be connected to the container.
	Links []string

	// DeviceRequests are device requests on the container itself.
	DeviceRequests []container.DeviceRequest

	Devices []container.DeviceMapping

	// SecurityOpts are security options to set on the container.
	SecurityOpts []string
	// contains filtered or unexported fields
}

RunOpts are options for running a container.

func GPURunOpts 

func GPURunOpts(sniffGPUOpts SniffGPUOpts) (RunOpts, error)

GPURunOpts returns Docker run options with GPU support enabled.

type SniffGPUOpts 

type SniffGPUOpts struct {
	// If set, explains why the sniffer should be disabled for this test.
	// If unset or empty, the sniffer is enabled.
	DisableSnifferReason string

	// If true, the test will not fail even when the workload calls incompatible
	// ioctls. Useful for debugging.
	// TODO(b/340955577): Should be converted to a flag and removed from this
	// struct once all GPU tests have no incompatible ioctls.
	AllowIncompatibleIoctl bool

	// The set of GPU capabilities exposed to the container.
	// If unset, defaults to `DefaultGPUCapabilities`.
	Capabilities string
	// contains filtered or unexported fields
}

SniffGPUOpts dictates options to sniffer GPU workloads.

func (*SniffGPUOpts) GPUCapabilities 

func (sgo *SniffGPUOpts) GPUCapabilities() string

GPUCapabilities returns the set of GPU capabilities meant to be exposed to the container.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.