bisect

Documentation

Overview

Bisect finds changes responsible for causing a failure. A typical use is to identify the source locations in a program that are miscompiled by a given compiler optimization.

Usage:

bisect [flags] [var=value...] command [arguments...]

Bisect operates on a target command line – the target – that can be run with various changes individually enabled or disabled. With none of the changes enabled, the target is known to succeed (exit with exit code zero). With all the changes enabled, the target is known to fail (exit any other way). Bisect repeats the target with different sets of changes enabled, using binary search to find (non-overlapping) minimal change sets that provoke the failure.

The target must cooperate with bisect by accepting a change pattern and then enabling and reporting the changes that match that pattern. The change pattern is passed to the target by substituting it anywhere the string PATTERN appears in the environment values or the command arguments. For each change that matches the pattern, the target must enable that change and also print one or more “match lines” (to standard output or standard error) describing the change. The golang.org/x/tools/core/bisect package provides functions to help targets implement this protocol. We plan to publish that package in a non-core location after finalizing its API.

Bisect starts by running the target with no changes enabled and then with all changes enabled. It expects the former to succeed and the latter to fail, and then it will search for the minimal set of changes that must be enabled to provoke the failure. If the situation is reversed – the target fails with no changes enabled and succeeds with all changes enabled – then bisect automatically runs in reverse as well, searching for the minimal set of changes that must be disabled to provoke the failure.

Bisect prints tracing logs to standard error and the minimal change sets to standard output.

Command Line Flags

Bisect supports the following command-line flags:

-max=M

Stop after finding M minimal change sets. The default is no maximum, meaning to run until all changes that provoke a failure have been identified.

-maxset=S

Disallow change sets larger than S elements. The default is no maximum.

-timeout=D

If the target runs for longer than duration D, stop the target and interpret that as a failure. The default is no timeout.

-count=N

Run each trial N times (default 2), checking for consistency.

-v

Print verbose output, showing each run and its match lines.

In addition to these general flags, bisect supports a few “shortcut” flags that make it more convenient to use with specific targets.

-compile=<rewrite>

This flag is equivalent to adding an environment variable “GOCOMPILEDEBUG=<rewrite>hash=PATTERN”, which, as discussed in more detail in the example below, allows bisect to identify the specific source locations where the compiler rewrite causes the target to fail.

-godebug=<name>=<value>

This flag is equivalent to adding an environment variable “GODEBUG=<name>=<value>#PATTERN”, which allows bisect to identify the specific call stacks where the changed GODEBUG setting value causes the target to fail.

Example

The Go compiler provides support for enabling or disabling certain rewrites and optimizations to allow bisect to identify specific source locations where the rewrite causes the program to fail. For example, to bisect a failure caused by the new loop variable semantics:

bisect go test -gcflags=all=-d=loopvarhash=PATTERN

The -gcflags=all= instructs the go command to pass the -d=... to the Go compiler when compiling all packages. Bisect varies PATTERN to determine the minimal set of changes needed to reproduce the failure.

The go command also checks the GOCOMPILEDEBUG environment variable for flags to pass to the compiler, so the above command is equivalent to:

bisect GOCOMPILEDEBUG=loopvarhash=PATTERN go test

Finally, as mentioned earlier, the -compile flag allows shortening this command further:

bisect -compile=loopvar go test

Defeating Build Caches

Build systems cache build results, to avoid repeating the same compilations over and over. When using a cached build result, the go command (correctly) reprints the cached standard output and standard error associated with that command invocation. (This makes commands like 'go build -gcflags=-S' for printing an assembly listing work reliably.)

Unfortunately, most build systems, including Bazel, are not as careful as the go command about reprinting compiler output. If the compiler is what prints match lines, a build system that suppresses compiler output when using cached compiler results will confuse bisect. To defeat such build caches, bisect replaces the literal text “RANDOM” in environment values and command arguments with a random 64-bit value during each invocation. The Go compiler conveniently accepts a -d=ignore=... debug flag that ignores its argument, so to run the previous example using Bazel, the invocation is:

bazel test --define=gc_goopts=-d=loopvarhash=PATTERN,unused=RANDOM //path/to:test