go-fuzz

module
v0.0.0-...-bfb31ba Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 10, 2015 License: Apache-2.0

README

go-fuzz: randomized testing for Go

Go-fuzz is a coverage-guided fuzzing solution for testing of Go packages. Fuzzing is mainly applicable to packages that parse complex inputs (both text and binary), and is especially useful for hardening of systems that parse inputs from potentially malicious users (e.g. anything accepted over a network).

Usage

First, you need to write a test function of the form:

func Fuzz(data []byte) int

Data is a random input generated by go-fuzz, note that in most cases it is invalid. The function must return 1 if the input is interesting in some way (for example, it was parsed successfully, that is, it is lexically correct, go-fuzz will give more priority to such inputs); -1 if the input must not be added to corpus even if gives new coverage; and 0 otherwise; other values are reserved for future use. In its basic form the Fuzz function just parses the input, and go-fuzz ensures that it does not panic, crash the program, allocate insane amount of memory nor hang. Fuzz function can also do application-level checks, which will make testing more efficient (discover more bugs). For example, Fuzz function can serialize all inputs that were successfully deserialized, thus ensuring that serialization can handle everything deserialization can produce. Or, Fuzz function can deserialize-serialize-deserialize-serialize and check that results of first and second serialization are equal. Or, Fuzz function can feed the input into two different implementations (e.g. dumb and optimized) and check that the output is equal. To communicate application-level bugs Fuzz function should panic (os.Exit(1) will work too, but panic message contains more info). Note that Fuzz function should not output to stdout/stderr, it will slow down fuzzing and nobody will see the output anyway. The exception is printing info about a bug just before panicking.

Here is an example of a simple Fuzz function for image/png package:

package png

import (
	"bytes"
	"image/png"
)

func Fuzz(data []byte) int {
	png.Decode(bytes.NewReader(data))
	return 0
}

A more useful Fuzz function would look like:

func Fuzz(data []byte) int {
	img, err := png.Decode(bytes.NewReader(data))
	if err != nil {
		if img != nil {
			panic("img != nil on error")
		}
		return 0
	}
	var w bytes.Buffer
	err = png.Encode(&w, img)
	if err != nil {
		panic(err)
	}
	return 1
}

The second step is collection of initial input corpus. Ideally, files in the corpus are as small as possible and as diverse as possible. You can use inputs used by unit tests and/or generate them. For example, for an image decoding package you can encode several small bitmaps (black, random noise, white with few non-white pixels) with different levels of compressions and use that as the initial corpus. Go-fuzz will deduplicate and minimize the inputs. So throwing in a thousand of inputs is fine, diversity is more important.

Put the initial corpus into the workdir/corpus directory (in our case examples/png/corpus). Go-fuzz will add own inputs to the corpus directory. Consider committing the generated inputs to your source control system, this will allow you to restart go-fuzz without losing previous work.

Examples directory contains a bunch of examples of test functions and initial input corpuses for various packages.

The next step is to get go-fuzz:

$ go get github.com/dvyukov/go-fuzz/go-fuzz
$ go get github.com/dvyukov/go-fuzz/go-fuzz-build

Then, build the test program with necessary instrumentation:

$ go-fuzz-build github.com/dvyukov/go-fuzz/examples/png

This will produce png-fuzz.zip archive.

Now we are ready to go:

$ go-fuzz -bin=./png-fuzz.zip -workdir=examples/png

Go-fuzz will generate and test various inputs in an infinite loop. Workdir is used to store persistent data like current corpus and crashers, it allows fuzzer to continue after restart. Discovered bad inputs are stored in workdir/crashers dir; where file without a suffix contains binary input, file with .quoted suffix contains quoted input that can be directly copied into a reproducer program or a test, file with .output suffix contains output of the test on this input. Every few seconds go-fuzz prints logs of the form:

2015/04/25 12:39:53 slaves: 500, corpus: 186 (42s ago), crashers: 3,
     restarts: 1/8027, execs: 12009519 (121224/sec), cover: 2746, uptime: 1m39s

Where slaves means number of tests running in parallel (set with -procs flag). corpus is current number of interesting inputs the fuzzer has discovered, time in brackets says when the last interesting input was discovered. crashers is number of discovered bugs (check out workdir/crashers dir). restarts is the rate with which the fuzzer restarts test processes. The rate should be close to 1/10000 (which is the planned restart rate); if it is considerably higher than 1/10000, consider fixing already discovered bugs which lead to frequent restarts. execs is total number of test executions, and the number in brackets is the average speed of test executions. cover is number of bits set in a hashed coverage bitmap, if this number grows fuzzer uncovers new lines of code; size of the bitmap is 64K; ideally cover value should be less than ~5000, otherwise fuzzer can miss new interesting inputs due to hash collisions. And finally uptime is uptime of the process. This same information is also served via http (see the -http flag).

Random Notes

go-fuzz-build builds the program with gofuzz build tag, this allows to put the Fuzz function implementation directly into the tested package, but exclude it from normal builds with // +build gofuzz directive.

If your inputs contain a checksum, it can make sense to append/update the checksum in the Fuzz function. The chances that go-fuzz will generate the correct checksum are very low, so most work will be in vain otherwise.

Go-fuzz can utilize several machines. To do this, start master process separately:

$ go-fuzz -workdir=examples/png -master=127.0.0.1:8745

It will manage persistent corpus and crashers and coordinate work of slave processes. Then run one or more slave processes as:

$ go-fuzz -bin=./png-fuzz.zip -slave=127.0.0.1:8745 -procs=10

External Articles

Credits and technical details

Go-fuzz fuzzing logic is heavily based on american fuzzy lop, so refer to AFL readme if you are interested in technical details. AFL is written and maintained by Michal Zalewski. Some of the mutations employed by go-fuzz are inspired by work done by Mateusz Jurczyk, Gynvael Coldwind and Felix Gröbert.

Trophies

If you find some bugs with go-fuzz and are comfortable with sharing them, I would like to add them to this list. Please either send a pull request for README.md (preferable) or file an issue. If the source code is closed, you can say just "found N bugs in project X". Thank you.

Directories

Path Synopsis
examples
aes
asm
asn1
bmp
bson
bzip2
csv
elf
elliptic
flag
flatbuffers
flatbuffers/mygame
flate
fmt
freetype
fuzz
gif
goast
gob
gofmt
gopacket
gorillamux
gotypes
gzip
html
htmltemplate
http2
httpreq
httpresp
httpserver
idna
jpeg
json
jsonrpc
lzw
macho
mail
mime
multipart
newparser
nss
parser
path
pem
png
png/gen
protobuf
protobuf/gen
protobuf/pb
Package example is a generated protocol buffer package.
 Package example is a generated protocol buffer package.
regexp
smtp
snappy
sqlparser
stdhtml
strings
suffixrray
tar
test
testcover
texttemplate
tiff
time
tls
tlsclient
trace
truetype
ttf
url
webdav
webp
websocketclient
websocketserver
x509
xml
zip
zlib
internal/writerset
Package writerset implements a mechanism to add and remove writers from a construct similar to io.MultiWriter.
 Package writerset implements a mechanism to add and remove writers from a construct similar to io.MultiWriter.
versifier
Package versifier recognizes internal structure of random (currently text-only) data and allows to generate data of a similar structure (for some very weak definition of "similar").
 Package versifier recognizes internal structure of random (currently text-only) data and allows to generate data of a similar structure (for some very weak definition of "similar").

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.