coverbee

README

CoverBee

Code coverage collection tool for eBPF programs. CoverBee can instrument already compiled eBPF programs by giving it an ELF file. This allows for coverage testing without modifying the existing toolchain.

Installation

go install github.com/dylandreimerink/coverbee/cmd/coverbee@latest

Usage CLI

First, instrument and load the programs in a ELF file by using coverbee load. All programs will be pinned in the directory specified by --prog-pin-dir. If --map-pin-dir is specified, all maps with with pinning = LIBBPF_PIN_BY_NAME set will be pinned in the given map. If --map-pin-dir is not specified, a pin location for the cover-map must be specified with --covermap-pin. The block-list will be written as JSON to a location specified by --block-list this file contains translation data and must be passed to coverbee cover afterwards.

Instrument all programs in the given ELF file and load them into the kernel

Usage:
  coverbee load {--elf=ELF path} {--prog-pin-dir=path to dir} {--map-pin-dir=path to dir | --covermap-pin=path to covermap} {--block-list=path to blocklist} [flags]

Flags:
      --block-list string     Path where the block-list is stored (contains coverage data to source code mapping, needed when reading from cover map)
      --covermap-pin string   Path to pin for the covermap (created by coverbee containing coverage information)
      --elf string            Path to the ELF file containing the programs
  -h, --help                  help for load
      --log string            Path for ultra-verbose log output
      --map-pin-dir string    Path to the directory containing map pins
      --prog-pin-dir string   Path the directory where the loaded programs will be pinned
      --prog-type string      Explicitly set the program type

Then attach the programs or test them with BPF_TEST_RUN.

Once done, to inspect the coverage call coverbee cover, pass it the same --map-pin-dir/--covermap-pin and --block-list as was used for coverbee load. Specify a path for the output with --output which is html by default but can also be set to output go-cover for use with other tools by setting --format go-cover

Collect coverage data and output to file

Usage:
  coverbee cover {--map-pin-dir=path to dir | --covermap-pin=path to covermap} {--block-list=path to blocklist} {--output=path to report output} [flags]

Flags:
      --block-list string     Path where the block-list is stored (contains coverage data to source code mapping, needed when reading from cover map)
      --covermap-pin string   Path to pin for the covermap (created by coverbee containing coverage information)
      --format string         Output format (options: html, go-cover) (default "html")
  -h, --help                  help for cover
      --map-pin-dir string    Path to the directory containing map pins
      --output string         Path to the coverage output

Don't forget to clean up the programs by detaching and/or removing the pins.

Usage as library

1. Load the ELF file using cilium/ebpf
2. Perform normal setup(except for loading the programs, maps can be pre-loaded)
3. Call coverbee.InstrumentAndLoadCollection instead of using ebpf.NewCollectionWithOptions
4. Attach the program or run tests
5. Convert the CFG gotten in step 3 to a block-list with coverbee.CFGToBlockList
6. Get the coverbee_covermap from the collection and apply its contents to the block-list with coverbee.ApplyCoverMapToBlockList
7. Convert the block-list into a go-cover or HTML report file with coverbee.BlockListToGoCover or coverbee.BlockListToHTML respectively

How does CoverBee work

CoverBee instruments existing compiled eBPF programs in ELF format and load them into the kernel. This instrumentation will increment numbers in a eBPF map when certain parts of the program are ran. CoverBee uses the kernel verifier logs to find out which registers and stack slots are not used by the program, and uses these for the instrumentation code.

The contents of the cover-map are be mapped back to the source file via the block-list. This block-list is constructed from the control flow graph of the programs and the BTF.ext line information. Then a modified version of go tool cover is used to create HTML reports.

Limitations / Requirements

• CoverBee requires up to 3 stack slots (24 bytes) available on the stack, programs close to the limit might not pass the verifier once instrumented.
• CoverBee adds instructions to the programs, programs close to the instruction or complexity limit of the kernel might not pass the verifier once instrumented.
• CoverBee used BTF.ext information to convert instructions to coverage information, ELF files without BTF will not work
• CoverBee requires the source code of the programs to pre present at the same location as at compile time and to contain the same contents. BTF.ext contains line and column offsets to absolute filepaths, changes in path or file contents between compilation and coverage testing might result in invalid or non-working coverage reports.
• CoverBee currently loads programs without BTF information, programs relying on BTF to be present will fail the verifier.
• CoverBee will add a map named coverbee_covermap to the collection, so this name can't be used by the program itself.

Documentation

Index

• func ApplyCoverMapToBlockList(coverMap *ebpf.Map, blockList [][]CoverBlock) error
• func BlockListToGoCover(blockList [][]CoverBlock, out io.Writer, mode string)
• func BlockListToHTML(blockList [][]CoverBlock, out io.Writer, mode string) error
• func CFGToBlockList(cfg []*ProgBlock) [][]CoverBlock
• func HtmlOutput(profiles []*cover.Profile, out io.Writer) error
• type CoverBlock
  • func (cb CoverBlock) String() string
• type ProgBlock
  • func InstrumentAndLoadCollection(coll *ebpf.CollectionSpec, opts ebpf.CollectionOptions, logWriter io.Writer) (*ebpf.Collection, []*ProgBlock, error)
  • func ProgramBlocks(prog asm.Instructions) []*ProgBlock

Functions

func ApplyCoverMapToBlockList

func ApplyCoverMapToBlockList(coverMap *ebpf.Map, blockList [][]CoverBlock) error

ApplyCoverMapToBlockList reads from the coverage map and applies the counts inside the map to the block list. The blocklist can be iterated after this to create a go-cover coverage file.

func BlockListToGoCover

func BlockListToGoCover(blockList [][]CoverBlock, out io.Writer, mode string)

BlockListToGoCover convert a block-list into a go-cover file which can be interpreted by `go tool cover`. `mode` value can be `set` or `count`, see `go tool cover -h` for details.

func BlockListToHTML

func BlockListToHTML(blockList [][]CoverBlock, out io.Writer, mode string) error

BlockListToHTML converts a block-list into a HTML coverage report.

func CFGToBlockList

func CFGToBlockList(cfg []*ProgBlock) [][]CoverBlock

CFGToBlockList convert a CFG to a "BlockList", the outer slice indexed by BlockID which maps to an inner slice, each element of which is a reference to a specific block of code inside a source file. Thus the resulting block list can be used to translate blockID's into the pieces of source code to apply coverage mapping.

func HtmlOutput

func HtmlOutput(profiles []*cover.Profile, out io.Writer) error

HtmlOutput generates an HTML page from profile data. coverage report is written to the out writer.

Types

type CoverBlock

type CoverBlock struct {
	Filename     string
	ProfileBlock cover.ProfileBlock
}

CoverBlock wraps the ProfileBlock, adding a filename so each CoverBlock can be turned into a line on a go coverage file.

func (CoverBlock) String

func (cb CoverBlock) String() string

type ProgBlock

type ProgBlock struct {
	Index int
	// The current block of code
	Block asm.Instructions

	// The next block of we don't branch
	NoBranch *ProgBlock
	// The next block if we do branch
	Branch *ProgBlock
}

func InstrumentAndLoadCollection

func InstrumentAndLoadCollection(
	coll *ebpf.CollectionSpec,
	opts ebpf.CollectionOptions,
	logWriter io.Writer,
) (*ebpf.Collection, []*ProgBlock, error)

InstrumentAndLoadCollection adds instrumentation instructions to all programs contained within the given collection. This "instrumentation" is nothing more than incrementing a 16-bit number within a map value, at an index unique to the location within the program(Block ID/index). After updating the program, it is loaded into the kernel, the loaded collection and a list of program blocks is returned. The index of the returned program blocks matches the index of blocks in the coverage map.

Steps of the function:

1. Load the original programs and collect the verbose verifier log
2. Parse the verifier log, which tells us which registers and stack slots are occupied at any given time.
3. Convert the program into a CFG(Control Flow Graph)
4. At the start of each program and bpf-to-bpf function, load the cover-map's index 0 and store the map value in a available slot on the stack.
5. At the start of each block, load an offset into the cover-map value, increment it, write it back. This requires 2 registers which can be clobbered. If only 1 or no registers are unused, store the register values to the stack and restore values afterwards.
6. Move symbols of the original code to the instrumented code so jumps and functions calls first pass by the instrumentation.
7. Load all modified program into the kernel.

func ProgramBlocks

func ProgramBlocks(prog asm.Instructions) []*ProgBlock

ProgramBlocks takes a list of instructions and converts it into a a CFG(Control Flow Graph). Which works as follows:

1. Construct a translation map from RawOffsets to the instructions(since index within the slice doesn't account for LDIMM64 instructions which use two instructions).
2. Apply a label to every jump target and set that label as a reference in the branching instruction. This does two things. First, it makes it easy to find all block boundaries since each block has a function name or jump label. The second is that cilium/ebpf will recalculate the offsets of the jumps based on the symbols when loading, so we can easily add instructions to blocks without fear of breaking offsets.
3. Loop over all instructions, creating a block at each branching instruction or symbol/jump label.
4. Build a translation map from symbol/jump label to block.
5. Loop over all blocks, using the map from step 4 to link blocks together on the branching and non-branching edges.