exec

README

Exec plugin

The exec plugin runs a given binary or executable and determines whether the test passed based on criteria specified by options. The executables are run using a transport that abstracts the mechanism of launching the process.

Parameters

Top-level parameter is called bag and it has a single element with the actual plugin parameter values.

• bin: specifies the path and args for the executable
• transport: specifies the protocol to use and options for that specific protocol
• constraints: currently just has time_quota, the maximum duration any local process can run in the context of the Contest server (more details below per transport). Value of 0 means infinite quota.
• ocp_output (default: false): if this is true, the output of the process is parsed as per OCP Testing & Validation specification in order to decide whether it passed or not. When set to false, the test is considered passed if exit code was 0.

Transport options

Proto local:

• this has no specific options, just runs the given binary on the same server as the Contest process. The binary must exist on that filesystem.
• the top level time_quota applies here and when it is exceeded, the Contest server kills the process

Of note is the fact that the Contest server binary must be built (or run) with tag unsafe for this to be available. By default a stub implementation is used that just errors out. The consideration here is that arbitrary code can be run on the Contest server machine using this transport therefore making the machine insecure.

Proto ssh:

• host: ip/hostname that has the ssh server to connect to
• port (default: 22): ssh server port
• user: ssh user to use on connect
• password (default: empty): ssh password to use; if empty, password auth is not considered
• identity_file (default: empty): ssh private key to use as identity; if empty, pubkey auth is not considered
• send_binary (default: false): if true, the bin.path parameter specifies a file in the Contest filesystem that is going to be copied to the target machine thru the ssh channel (in /tmp) before starting execution. The default false means that the bin.path option specifies an existing remote file on the target machine and no transfers take place.
• async (default: omit): see below

For long running remote jobs (for now, just ssh based), the async option specifies the remote agent that will be used to monitor the job. For convenience, an implementation that satisfies the agent protocol for this transport is provided in cmds/exec_agent. To use it go build the exec_agent binary and specify the binary path in the agent key. This binary must be present on the same filesystem Contest is running on. The time_quota (default: 0) option inside async gets passed onto the agent when it launches a job. If this is exceeded, the remote agent just kills the controlled process and exits. This is useful in the case of network partitions or Contest server errors because it enforces that resources are eventually released on the target machines. The default means infinite quota, so it is recommended to set it to some appropriate value.

The top level time_quota applies to individual ssh operations. So when async is omitted, the live ssh connection (and process running remotely) gets killed when the quota is exceeded. For async processes, the individual start, poll, etc operations are monitored for execution time, making this option less relevant in this case.

Examples

The following launches /home/test/exec_bin on the same machine as the Contest server and runs it for at most 20 seconds per target with the only argument being the target FQDN. The result is determined from the exit code.

"Steps": [
    {
        "name": "exec",
        "label": "label",
        "parameters": {
            "bag": [
                {
                    "bin": {
                        "path": "/home/test/exec_bin",
                        "args": [
                            "{{.FQDN}}"
                        ]
                    },
                    "transport": {
                        "proto": "local"
                    },
                    "constraints": {
                        "time_quota": "20s"
                    }
                }
            ]
        }
    }
]

The following launches the file /packages/exec_bin residing on the test target thru ssh, which was authenticated using pubkey method. The process and connection are killed after 20s (and test fails) if that quota is exceeded. The binary produces OCP T&V output, which is parsed by Contest to decide the pass/fail result.

"Steps": [
    {
        "name": "exec",
        "label": "label",
        "parameters": {
            "bag": [
                {
                    "bin": {
                        "path": "/packages/exec_bin"
                    },
                    "transport": {
                        "proto": "ssh",
                        "options": {
                            "host": "{{.FQDN}}",
                            "user": "test",
                            "identity_file": "/home/test/.ssh/id_rsa",
                        }
                    },
                    "constraints": {
                        "time_quota": "20s"
                    },
                    "ocp_output": true
                }
            ]
        }
    }
]

The following config copies the /home/test/exec_bin binary to the target machine (in /tmp/<random_uuid_name>), along with a copy of the agent taken from /home/test/contest/cmds/exec_agent/exec_agent. The agent is then started (which starts the remote binary itself) then the ssh connection is terminated. Contest then periodically establishes new ssh connections to poll the outputs and state of the remote agent-controlled process. When the agent goes over the 20s quota, it kills the process (regardless of anything Contest might be doing). The test result is parsed from the OCP T&V output.

"Steps": [
    {
        "name": "exec",
        "label": "label",
        "parameters": {
            "bag": [
                {
                    "bin": {
                        "path": "/home/test/exec_bin"
                    },
                    "transport": {
                        "proto": "ssh",
                        "options": {
                            "host": "{{.FQDN}}",
                            "user": "test",
                            "identity_file": "/home/test/.ssh/id_rsa",
                            "send_binary": true,
                            "async": {
                                "agent": "/home/test/contest/cmds/exec_agent/exec_agent",
                                "time_quota": "20s"
                            }
                        }
                    },
                    "ocp_output": true
                }
            ]
        }
    }
]

Documentation

Index

• Constants
• Variables
• func Load() (string, test.TestStepFactory, []event.Name)
• func New() test.TestStep
• type Log
• type OCPEventParser
  • func NewOCPEventParser(target *target.Target, ev testevent.Emitter) *OCPEventParser
  • func (ep *OCPEventParser) Error() error
  • func (ep *OCPEventParser) Parse(ctx xcontext.Context, root *OCPRoot) error
• type OCPRoot
• type OCPState
  • func (s OCPState) Error() error
• type Result
• type RunArtifact
• type RunEnd
• type RunStart
• type Severity
• type Status
• type StepArtifact
• type StepEnd
• type StepStart
• type TargetRunner
  • func NewTargetRunner(ts *TestStep, ev testevent.Emitter) *TargetRunner
  • func (r *TargetRunner) Run(ctx xcontext.Context, target *target.Target) error
• type TestStep
  • func (ts TestStep) Name() string
  • func (ts *TestStep) Run(ctx xcontext.Context, ch test.TestStepChannels, params test.TestStepParameters, ...) (json.RawMessage, error)
  • func (ts *TestStep) ValidateParameters(_ xcontext.Context, stepParams test.TestStepParameters) error

Constants

const (
	TestStartEvent = event.Name("TestStart")
	TestEndEvent   = event.Name("TestEnd")
	TestLogEvent   = event.Name("TestLog")

	StepStartEvent = event.Name("StepStart")
	StepEndEvent   = event.Name("StepEnd")
	StepLogEvent   = event.Name("StepLog")
)

events that we may emit during the plugin's lifecycle

const (
	SeverityInfo    = Severity("INFO")
	SeverityDebug   = Severity("DEBUG")
	SeverityWarning = Severity("WARNING")
	SeverityError   = Severity("ERROR")
	SeverityFatal   = Severity("FATAL")
)

const (
	StatusUnkown   = Status("UNKNOWN")
	StatusComplete = Status("COMPLETE")
	StatusError    = Status("ERROR")
	StatusSkipped  = Status("SKIPPED")
)

const (
	ResultPass = Result("PASS")
	ResultFail = Result("FAIL")
	ResultNA   = Result("NOT_APPLICABLE")
)

Variables

var Events = []event.Name{
	TestStartEvent, TestEndEvent,
	TestLogEvent,
	StepStartEvent, StepEndEvent,
	StepLogEvent,
}

Events defines the events that a TestStep is allow to emit. Emitting an event that is not registered here will cause the plugin to terminate with an error.

var Name = "Exec"

Name is the name used to look this plugin up.

Functions

func Load

func Load() (string, test.TestStepFactory, []event.Name)

Load returns the name, factory and events which are needed to register the step.

func New

func New() test.TestStep

New initializes and returns a new exec step.

Types

type Log

type Log struct {
	Severity Severity `json:"severity,omitempty"`
	Text     string   `json:"text,omitempty"`
}

TODO: these should just be temporary until the go:generate tool TODO: this should also mean refactoring all the parser code

type OCPEventParser

type OCPEventParser struct {
	// contains filtered or unexported fields
}

func NewOCPEventParser

func NewOCPEventParser(target *target.Target, ev testevent.Emitter) *OCPEventParser

func (*OCPEventParser) Error

func (ep *OCPEventParser) Error() error

func (*OCPEventParser) Parse

func (ep *OCPEventParser) Parse(ctx xcontext.Context, root *OCPRoot) error

type OCPRoot

type OCPRoot struct {
	SequenceNumber int           `json:"sequenceNumber"`
	Timestamp      string        `json:"timestamp"`
	RunArtifact    *RunArtifact  `json:"testRunArtifact,omitempty"`
	StepArtifact   *StepArtifact `json:"testStepArtifact,omitempty"`
}

type OCPState

type OCPState struct {
	RunEnd *RunEnd
}

TODO: check if there can be multiple runs in the same output

func (OCPState) Error

func (s OCPState) Error() error

type Result

type Result string

type RunArtifact

type RunArtifact struct {
	RunStart *RunStart `json:"testRunStart,omitempty"`
	RunEnd   *RunEnd   `json:"testRunEnd,omitempty"`
	Log      *Log      `json:"log,omitempty"`
}

type RunEnd

type RunEnd struct {
	Name   string `json:"name,omitempty"`
	Status Status `json:"status,omitempty"`
	Result Result `json:"result,omitempty"`
}

type RunStart

type RunStart struct {
	Name    string `json:"name,omitempty"`
	Version string `json:"version,omitempty"`
}

type Severity

type Severity string

type Status

type Status string

type StepArtifact

type StepArtifact struct {
	StepId string `json:"testStepId,omitempty"`

	StepStart *StepStart `json:"testStepStart,omitempty"`
	StepEnd   *StepEnd   `json:"testStepEnd,omitempty"`
	Log       *Log       `json:"log,omitempty"`
}

type StepEnd

type StepEnd struct {
	Name   string `json:"name,omitempty"`
	Status Status `json:"status,omitempty"`
}

type StepStart

type StepStart struct {
	Name string `json:"name,omitempty"`
}

type TargetRunner

type TargetRunner struct {
	// contains filtered or unexported fields
}

func NewTargetRunner

func NewTargetRunner(ts *TestStep, ev testevent.Emitter) *TargetRunner

func (*TargetRunner) Run

func (r *TargetRunner) Run(ctx xcontext.Context, target *target.Target) error

type TestStep

type TestStep struct {
	// contains filtered or unexported fields
}

TestStep implementation for the exec plugin

func (TestStep) Name

func (ts TestStep) Name() string

Name returns the name of the Step

func (*TestStep) Run

func (ts *TestStep) Run(ctx xcontext.Context, ch test.TestStepChannels, params test.TestStepParameters, ev testevent.Emitter, resumeState json.RawMessage) (json.RawMessage, error)

Run executes the step.

func (*TestStep) ValidateParameters

func (ts *TestStep) ValidateParameters(_ xcontext.Context, stepParams test.TestStepParameters) error

ValidateParameters validates the parameters associated to the step