truth

Documentation

Overview

Package truth implements an extensible, simple, assertion library for Go with minimal dependencies.

Why have an assertion library at all?

While it is recommended to not use assertion libraries for Go tests, we have found that approach to be lacking in the following ways:

1. Writing good error messages is difficult; when an assertion fails, the error message needs to indicate why. As the error messages get more complex, you may be tempted to write helpers... at which point you've built an assertion library.
2. If you refuse to put the helpers in a library, you now have one or more custom functions per package, which is worse.
3. If you ignore the problem, you may be tempted to write shorter error messages (or fewer tests), which makes test failures more cumbersome to debug.

So, for our applications, we've found them to be helpful tools to write high quality tests. This library is NOT a requirement, but a tool. If it gets in the way, don't use it (or, perhaps better, improve it).

Why not X?

At the time of writing, Go generics were relatively new, and no assertion libraries had adopted them in a meaningful way to make compilers do the legwork to make sure all the types lined up correctly when possible.

One of the really bad things about early assertion libraries was that they were almost always forced to use `any` for inputs, and extensive amounts of "reflect" based code.

This made the APIs of such assertion libraries more difficult to grok for readers, and meant that a large class of assertion failures only showed up at runtime, which was unfortunate.

While this library does still do some runtime type reflection (to convert from `actual any` to `T` for the given Comparison in assert.Loosely and check.Loosely), this conversion is done in exactly one place (this package), and does not require each comparison to reimplement this. In addition, the default symbols assert.That and check.That do no dynamic type inference at all, which should hopefully encourage test authors to follow this stricter style by default.

Why now, and why this style?

At the time this library was written, our codebase had a large amount of testing code written with `github.com/smartystreets/goconvey/convey` which is a "BDD style" testing framework (sort of). We liked the assertion syntax well enough to emulate it here; in convey assertions looked like:

So(actualValue, ShouldResemble, expectedValue)
So(somePointer, ShouldBeNil)
So(aString, ShouldBeOneOf, "a", "b", "c")

However, this framework had the problem that assertion functions are difficult to document (since their signature is always `func(any, ...any) string`), had an extra implementation burden for implementing custom checkers (every implementation, even small helpers inside of a test package, had to do type-casting on the expected arguments, ensure that the right number of expected values were passed, etc.).

Further, the return type of `string` is also dissatisfyingly untyped... there were global variables which could manipulate the goconvey assertions so that they returned encoded JSON instead of a plain string message, but this was even more surprising and dissatisfying.

For goconvey assertions, you also had to also use the controversial "Convey" suite syntax (see the sister library `ftt` adjacent to `truth`, which implements the test layout/format without the "BDD style" flavoring). This `assert` library has no such restriction and works directly with anything implementing testing.TB.

This library is a way for us to provide high quality assertion replacements for the So assertions of goconvey.

Usage

import (
  "testing"

  "go.chromium.org/luci/common/testing/truth/assert"

  // check makes non-fatal assertions (t.Fail()), in opposition to assert
  // which makes fatal assertions (t.FailNow()).
  "go.chromium.org/luci/common/testing/truth/check"

  // Optional; these are a collection of useful, common, comparisons, but
  // which are not required.
  "go.chromium.org/luci/common/testing/truth/should"

  // Optional; this library will let you write your own comparison functions
  // "go.chromium.org/luci/common/testing/truth/comparison"
)

func TestSomething(t *testing.T) {
   // This check will fail, but doesn't immediately stop the test.
   check.That(t, 100, should.Equal(200))

   // Checks that `someFunction` returns an `int` with the value `100`.
   // The type of someFunction() is enforced at compile time.
   assert.That(t, someFunction(), should.Equal(100))

   // Checks that `someFunction` returns some value lossesly assignable to
   // `int8` which equals 100.
   assert.Loosely(t, someOtherFunction(), should.Equal[int8](100))

   // Checks that `someFunction` returns some value assignable to
   // `*someStruct` which is populated in the same way.
   //
   // NOTE: should.Resemble correctly handles comparisons between protobufs
   // and types containing protobufs, by default, using the excellent
   // `github.com/google/go-cmp/cmp` library under the hood for comparisons.
   assert.That(t, someFunctionReturningStruct(), should.Resemble(&someStruct{
     ...
   }))
}

Example (Basic_usage)

// For the example we disable colorization, verbosity and fullpaths to make
// the Output stable.
defer disableColorization()()
defer disableVerbosity()()
defer disableFullpath()()

// NOTE: in a real test, you would use `*testing.T` or similar - not something
// like FakeTB. This will also show filename:lineno in the Output below, but
// this is omitted for this example.
t := new(fakeTB)

check.That(t, 100, should.Equal(100)) // OK - no output
check.That(t, 100, should.Equal(102)) // Fails!

// Does not compile:
// check.That(t, uint8(8), should.Equal(8))
// check.That(t, 100, should.Equal("hello"))

check.Loosely(t, 100, should.Equal(100))    // OK
check.Loosely(t, uint8(8), should.Equal(8)) // OK, Compiles

check.Loosely(t, 100, should.Equal(144))     // Fails!
check.Loosely(t, 100, should.Equal("hello")) // Fails!

// Long outputs automatically get diffed.
veryLongA := strings.Repeat("X", 1000) + "arg" + strings.Repeat("X", 1000)
veryLongB := strings.Repeat("X", 1000) + "B" + strings.Repeat("X", 1000)
check.That(t, veryLongA, should.Equal(veryLongB))

Output:

--- FAIL: FakeTestName (0.00s)
    filename.go:NN: check.That should.Equal[int] FAILED
        Actual: 100
        Expected: 102
    filename.go:NN: check.Loosely should.Equal[int] FAILED
        Actual: 100
        Expected: 144
    filename.go:NN: check.Loosely comparison.Func[string].CastCompare FAILED
        ActualType: int
    filename.go:NN: check.That should.Equal[string] FAILED
        Actual [verbose value len=2005 (pass -v to see)]
        Expected [verbose value len=2003 (pass -v to see)]
        Diff: \
              strings.Join({
              	... // 872 identical bytes
              	"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
              	"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
            - 	"arg",
            + 	"B",
              	"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
              	"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
              	... // 872 identical bytes
              }, "")

Example (Helpers_with_line_context)

// For the example we disable colorization, verbosity and fullpaths to make
// the Output stable.
defer disableColorization()()
defer disableVerbosity()()
defer disableFullpath()()

// NOTE: in a real test, you would use `*testing.T` or similar - not something
// like FakeTB. This will also show filename:lineno in the Output below, but
// this is omitted for this example.
t := new(fakeTB)

helper := func(expect int) {
	// tell go testing that it should skip this function when finding
	// the filename.go:NN value to print next to the error message.
	t.Helper()

	check.That(t, 10, should.Equal(expect), truth.LineContext()) // line 98
}

helper(10) // line 100

helper(20) // line 102

Output:

--- FAIL: FakeTestName (0.00s)
    filename.go:NN: check.That should.Equal[int] FAILED
        (at example_test.go:98)
        Actual: 10
        Expected: 20

Index

• Variables
• func ApplyAllOptions(summary *failure.Summary, opts []Option) *failure.Summary
• func Report(t TestingTB, name string, failure *failure.Summary)
• type Option
  • func Explain(format string, args ...any) Option
  • func LineContext(skipFrames ...int) Option
  • func ModifySummary(cb func(*failure.Summary)) Option
• type TestingTB

Examples

• Package (Basic_usage)
• Package (Helpers_with_line_context)

Variables

var Colorize bool

Colorize indicates that the truth library should colorize its output (used to highlight diffs produced by some comparison functions, such as should.Resemble).

By default this is true when Stdout is connected to a terminal.

You may override this in your TestMain function.

var FullSourceContextFilenames bool

FullSourceContextFilenames indicates that the truth library should print out full file paths for any SourceContexts in failure.Summaries.

By default, this is true when the '-test.fullpath' flag is passed to the binary (this is what gets set on the binary when you run `go test -fullpath`).

You may override this in your TestMain function.

var Verbose bool

Verbose indicates that the truth library should always render verbose Findings in comparison Failures.

See the example test for a quick "how to use this".

By default this is true when the '-test.v' flag is passed to the binary (this is what gets set on the binary when you run `go test -v`).

You may override this in your TestMain function.

Functions

func ApplyAllOptions

func ApplyAllOptions(summary *failure.Summary, opts []Option) *failure.Summary

ApplyAllOptions applies all Options to the given failure.Summary.

This is a no-op if `summary` is nil.

This is a fairly low-level function - typically options will be applied for you as part of {check,assert}.{That,Loosely}.

For convenience, returns `summary`.

func Report

func Report(t TestingTB, name string, failure *failure.Summary)

Report checks that there is no failure (i.e. `failure` is nil).

If failure is not nil, this will Log the error with t.Log, using `name` as a prefix.

Note: The recommended way to use the truth library is to import the `assert` and/or `check` sub-packages, which will call into this function. Direct use of truth.Report should be very rare.

Types

type Option

type Option interface {
	// contains filtered or unexported methods
}

Option is an interface to allow test authors to modify the behavior of {assert,check}.{That,Loosely} invocations.

A nil Option has no effect.

Known Options:

• truth.LineContext
• truth.SummaryModifier

func Explain

func Explain(format string, args ...any) Option

Explain is an Option which adds a new finding "Explaination" to the *failure.Summary of the failing comparison.

Example:

truth.Assert(t, wingie[i].cool(), should.BeTrue, truth.Explain(
  "wingie number %d was not cool"))

func LineContext

func LineContext(skipFrames ...int) Option

LineContext returns an Option which adds an "at" SourceContext with one frame containing the filename and line number of the frame calling LineContext, plus skipFrames[0] (if provided).

Example:

check.That(t, something, should.Equal(100), truth.LineContext())

You usually will not need this, but it's very useful when writing a helper function for tests (e.g. using t.Helper()) to let you add the location of the specific assert inside of the helper function along side the 'top most' frame location, as computed directly by the Go testing library.

Example:

func TestThing(t *testing.T) {
  myHelper := func(actual, expected myType) {
    t.Helper()  // makes Go 'testing' package skip this to find original call.

    // We add LineContext to these comparisons so that if they fail, we
    // will see the file:line within this helper function.
    check.That(t, actual.field1, should.Equal(expected.field1).LineContext())
    check.That(t, actual.field2, should.Equal(expected.field2).LineContext())
  }
  // ...
  myHelper(a, expected)
}

In this example, the test will output something like:

--- FAIL: FakeTestName (0.00s)
    example_test.go:XX: Check should.Equal[int] FAILED
       (at example_test.go:YY)
       Actual: 10
       Expected: 20

Where XX is the line of the myHelper call, and YY is the line of the actual should.Equal check inside of the helper function.

func ModifySummary

func ModifySummary(cb func(*failure.Summary)) Option

ModifySummary is an Option which modifies the *failure.Summary of a failing comparison.

Example:

truth.Assert(t, 100, should.Equal(1), truth.ModifySummary(func(s *failure.Summary) {
  s.Findings = append(s.Findings, &failure.Finding{Name: "Extra finding!"})
})

In this example, if 100 does not equal 1, this callback will be invoked on the *failure.Summary generated by should.Equal.

This will never be invoked with a nil *Summary.

type TestingTB

type TestingTB interface {
	Helper()
	Log(args ...any)
	Fail()
	FailNow()
}

TestingTB is a subset of the stdlib testing.TB type.

This allows this library to avoid directly importing "testing".