Documentation ¶
Overview ¶
Package testreporter contains a Reporter for collecting detailed test reports.
While you could probably get at all of the exposed information in reports by examining the output of "go test", the generated report is intended to collect the information in one machine-readable file.
Proper use of the reporter requires some slight changes to how you normally write tests. If you do any of these "the normal way", the tests will still operate fine; you will just miss some detail in the external report.
First, the reporter instance must be initialized and Closed. The cmd/interchaintest package does this in a MainTest function, similar to this:
func TestMain(m *testing.M) { f, _ := os.Create("/tmp/report.json") reporter := testreporter.NewReporter(f) code := m.Run() _ = reporter.Close() os.Exit(code) }
Next, every test that needs to be tracked must call TrackTest. If you omit the call to TrackTest, then the test's start and end time, and skip/fail status, will not be reported.
var reporter *testreporter.Reporter // Initialized somehow. func TestFoo(t *testing.T) { reporter.TrackTest(t) // Normal test usage continues... }
Calling TrackTest tracks the test's start and finish time, including whether the test was skipped or failed.
Parallel tests should not call t.Parallel directly, but instead should use TrackParallel. This will track the time the test paused waiting for parallel execution and when parallel execution resumes. If you omit the call to TrackParallel, then at worst you have a misleading test duration.
func TestFooParallel(t *testing.T) { reporter.TrackTest(t) reporter.TrackParallel(t) // Normal test usage continues... }
If a test needs to be skipped, the TrackSkip method will track the skip reason. Like the other Track methods, calling t.Skip directly will still cause the test to be skipped, and the reporter will note that the test was skipped, but the reporter would not track the specific skip reason.
func TestFooSkip(t *testing.T) { if someReason() { reporter.TrackSkip(t, "skipping due to %s", whySkipped()) } }
Lastly, and perhaps most importantly, the reporter is designed to integrate with testify's require and assert packages. Plain "go test" runs simply have a stream of log lines and a failure/skip state. But if you connect the reporter with a require or assert instance, any failed assertions are stored as error messages in the report.
func TestBar(t *testing.T) { reporter.TrackTest(t) req := require.New(reporter.TestifyT(t)) t.Log("About to test Bar()") // Goes to "go test" output, but not included in report. // If this fails, the report includes a "TestErrorMessage" entry in the report. req.NoError(Bar(), "failure executing Bar()") }
If you use a plain require.NoError(t, err) call, the report will note that the test failed, but the report will not include the error line.
Index ¶
- type BeginSuiteMessage
- type BeginTestMessage
- type ContinueTestMessage
- type FinishSuiteMessage
- type FinishTestMessage
- type Message
- type PauseTestMessage
- type RelayerExecMessage
- type RelayerExecReporter
- type Reporter
- type T
- type TestErrorMessage
- type TestSkipMessage
- type TestifyReporter
- type TestifyT
- type WrappedMessage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BeginSuiteMessage ¶
BeginSuiteMessage indicates when the Reporter was initialized, which should correlate with the beginning of a TestMain function, or an init function in a normal test suite.
type BeginTestMessage ¶
BeginTestMessage indicates the beginning of a single test. If the test uses t.Parallel (via (*Reporter).TrackParallel), the reporter will also track a PauseTestMessage and a ContinueTestMessage.
type ContinueTestMessage ¶
ContinueTestMessage indicates that a test has resumed execution after a call to t.Parallel.
type FinishSuiteMessage ¶
FinishSuiteMessage indicates the time the test suite has finished.
type FinishTestMessage ¶
FinishTestMessage is tracked at the end of a single test.
type Message ¶
type Message interface {
// contains filtered or unexported methods
}
Message is the sentinel interface to all testreporter messages.
type PauseTestMessage ¶
PauseTestMessage indicates that a test is entering parallel mode and waiting for its turn to continue execution.
type RelayerExecMessage ¶
type RelayerExecMessage struct { Name string // Test name, but "Name" for consistency. StartedAt, FinishedAt time.Time ContainerName string `json:",omitempty"` Command []string Stdout, Stderr string ExitCode int Error string `json:",omitempty"` }
RelayerExecMessage is the result of executing a relayer command. This message is populated through the RelayerExecReporter type, which is returned by the Reporter's RelayerExecReporter method.
type RelayerExecReporter ¶
type RelayerExecReporter struct {
// contains filtered or unexported fields
}
RelayerExecReporter provides one method that satisfies the ibc.RelayerExecReporter interface. Instances of RelayerExecReporter must be retrieved through (*Reporter).RelayerExecReporter.
func (*RelayerExecReporter) TrackRelayerExec ¶
func (r *RelayerExecReporter) TrackRelayerExec( containerName string, command []string, stdout, stderr string, exitCode int, startedAt, finishedAt time.Time, err error, )
TrackRelayerExec tracks the execution of an individual relayer command.
type Reporter ¶
type Reporter struct {
// contains filtered or unexported fields
}
func NewNopReporter ¶
func NewNopReporter() *Reporter
NewNopReporter returns a reporter that does not write anywhere.
func NewReporter ¶
func NewReporter(w io.WriteCloser) *Reporter
func (*Reporter) Close ¶
Close closes the reporter and blocks until its results are flushed to the underlying writer.
func (*Reporter) RelayerExecReporter ¶
func (r *Reporter) RelayerExecReporter(t T) *RelayerExecReporter
RelayerExecReporter returns a RelayerExecReporter associated with t.
func (*Reporter) TestifyT ¶
func (r *Reporter) TestifyT(t TestifyT) *TestifyReporter
TestifyT returns a TestifyReporter which will track logged errors in test. Typically you will use this with the New method on the require or assert package:
req := require.New(reporter.TestifyT(t)) // ... req.NoError(err, "failed to foo the bar")
func (*Reporter) TrackParallel ¶
TrackParallel tracks when the pause begins for a parallel test and when it continues to resume.
type T ¶
type T interface { Name() string Cleanup(func()) Skip(...any) Parallel() Failed() bool Skipped() bool }
T is a subset of testing.TB, representing only the methods required by the reporter.
type TestErrorMessage ¶
TestErrorMessage is tracked when a Reporter's TestifyT().Errorf method is called. This is the intended usage of a Reporter with require:
req := require.New(rep.TestifyT(t)) req.NoError(foo())
If req.NoError fails, then rep will track a TestErrorMessage.
type TestSkipMessage ¶
TestSkipMessage is tracked when a Reporter's TrackSkip method is called. This allows the report to track the reason a test was skipped.
type TestifyReporter ¶
type TestifyReporter struct {
// contains filtered or unexported fields
}
TestifyReporter wraps a Reporter to satisfy the testify/require.TestingT interface. This allows the reporter to track logged errors.
func (*TestifyReporter) Errorf ¶
func (r *TestifyReporter) Errorf(format string, args ...any)
Errorf records the error message in r's Reporter and then passes through to r's underlying TestifyT.
func (*TestifyReporter) FailNow ¶
func (r *TestifyReporter) FailNow()
FailNow passes through to r's TestifyT. It does not need to log another message because r's Reporter should be tracking the test already.
type WrappedMessage ¶
WrappedMessage wraps a Message with an outer Type field so that decoders can determine the underlying message's type.
func JSONMessage ¶
func JSONMessage(m Message) WrappedMessage
JSONMessage produces a WrappedMessage so that a stream of Message values can be distinguishedu by a top-level "Type" key.
func (*WrappedMessage) UnmarshalJSON ¶
func (m *WrappedMessage) UnmarshalJSON(b []byte) error