audit

README

Summary Engineering Notes

Summary is intended for development use to enable systematic probe logging.

These logs are designed to be one-per-probe. Each test may have multiple probes, and multiple probes will likely share similar probe types. Probes will overwrite within each test, however, to ensure that only one probe type is logged per test.

State

A new State context is created each time probr is run, and is readily accessible anywhere in the code via audit.State.

SummaryStateStruct.LogProbeMeta

Adding entries to the an Probe's meta data requires the name of the test and a key-value pair to be inserted.

n := "name-of-the-current-test"
k = "arbitrary_key_name"
v = "string_value"
audit.State.LogProbeMeta(n, k, v)

SummaryStateStruct.LogPodName

The names of all pods should be tracked, so users may identify whether Probr is the source of any unexpected pods in their cluster.

if pd != nil {
  s.PodName = pd.GetObjectMeta().GetName()
  audit.State.LogPodName(s.PodName)
}

SummaryStateStruct.GetProbeLog

Many summaries will be made directly to probes. In order to do so, the probe must first be retrieved by name. In the example below, the probe is being stored alongside other test context information for easy access during execution.

	ctx.BeforeScenario(func(s *godog.Scenario) {
		ps.name = s.Name
		ps.probe = audit.State.GetProbeLog(NAME)
		coreengine.LogScenarioStart(s)
	})

SummaryStateStruct.ProbeComplete

After an probe has finished running every scenario, we should audit the final outcome of the probe.

s, o, err := g.Handler(g.Data)
audit.State.ProbeComplete(t.ProbeDescriptor.Name)

SummaryStateStruct.SetProbrStatus

After all probes have completed, we should set the final probr status. This step may not always be relevant, as it may be possible to nest it within other methods such as PrintSummary. This should be reevaluated after more feedback has been gathered regarding how Probr is being used.

audit.State.SetProbrStatus()

SummaryStateStruct.PrintSummary

Instead of logging the final status of Probr within a particular loglevel and formatting it using log, PrintSummary simply formats the status into JSON and prints it to the command line. This is currently the very last thing our CLI tool does prior to exiting.

audit.State.PrintSummary()
os.Exit(s)

Probes

Probe.CountPodCreated and Probe.CountPodDestroyed

Whenever Probr will create or destroy a pod, these counters should be called to update the probe accordingly.

Probe.AuditScenarioStep

This function should be used every time a step in a scenario completes. AuditScenarioStep will automatically form the name of the step from the name of the function that called it. The name value provided will establish which scenario the step is a part of. The error (or nil) provided will dictate whether the test passes or fails.

The description and payload values are arbitrary and are used only to assist auditors in their evaluation. A nil error will be recorded as a successful step.

s.audit.AuditScenarioStep( "description string", payloadObject, err) 

Documentation

Index

• Variables
• type Probe
  • func (e *Probe) CountPodCreated(podName string)
  • func (e *Probe) CountPodDestroyed()
  • func (e *Probe) InitializeAuditor(name string, tags []*messages.Pickle_PickleTag) *ScenarioAudit
• type ProbeAudit
  • func (e *ProbeAudit) Write()
• type ScenarioAudit
  • func (p *ScenarioAudit) AuditScenarioStep(stepName, description string, payload interface{}, err error)

Variables

var State summaryState

State holds the values for all probe and scenario audits throughout the runtime

Types

type Probe

type Probe struct {
	Meta               map[string]interface{}
	PodsCreated        int
	PodsDestroyed      int
	ScenariosAttempted int
	ScenariosSucceeded int
	ScenariosFailed    int
	Result             string
	// contains filtered or unexported fields
}

Probe is passed through various functions to audit the probe's progress

func (*Probe) CountPodCreated

func (e *Probe) CountPodCreated(podName string)

CountPodCreated increments pods_created for probe

func (*Probe) CountPodDestroyed

func (e *Probe) CountPodDestroyed()

CountPodDestroyed increments pods_destroyed for probe

func (*Probe) InitializeAuditor

func (e *Probe) InitializeAuditor(name string, tags []*messages.Pickle_PickleTag) *ScenarioAudit

InitializeAuditor creates a new audit entry for the specified scenario

type ProbeAudit

type ProbeAudit struct {
	Name               string
	PodsDestroyed      *int
	ScenariosAttempted *int
	ScenariosSucceeded *int
	ScenariosFailed    *int
	Result             *string
	Scenarios          map[int]*ScenarioAudit
	// contains filtered or unexported fields
}

ProbeAudit is used to hold all information related to probe execution

func (*ProbeAudit) Write

func (e *ProbeAudit) Write()

type ScenarioAudit

type ScenarioAudit struct {
	Name   string
	Result string // Passed / Failed / Given Not Met
	Tags   []string
	Steps  map[int]*stepAudit
}

ScenarioAudit is used by scenario states to audit progress through each step

func (*ScenarioAudit) AuditScenarioStep

func (p *ScenarioAudit) AuditScenarioStep(stepName, description string, payload interface{}, err error)

AuditScenarioStep sets description, payload, and pass/fail based on err parameter. This function should be deferred to catch panic behavior, otherwise the audit will not be logged on panic