benchmarkjunit

command
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 3, 2023 License: Apache-2.0 Imports: 14 Imported by: 0

README

benchmarkjunit

benchmarkjunit is a go utility that runs go testing package benchmarks with a command like go test -v -bench=. <packages> and translates the output into JUnit XML.

Usage Examples

Run go run ./pkg/benchmarkjunit --help to see all the available flags.

go run ./pkg/benchmarkjunit k8s.io/test-infra/experiment/dummybenchmarks

go run ./pkg/benchmarkjunit ./experiment/dummybenchmarks/...

go run ./pkg/benchmarkjunit \
-o junit_benchmark.xml -l testlog.txt \
--test-arg=-benchmem --bench=Core --pass-on-error \
./experiment/dummybenchmarks/...

With bazel:

bazel run //pkg/benchmarkjunit -- k8s.io/test-infra/experiment/dummybenchmarks
Running as a ProwJob

benchmarkjunit can be used from a ProwJob to generate JUnit results for consumption in Testgrid, Spyglass, Gubernator, etc. See the prowjob_example.yaml file.

Demo Job

The ci-test-infra-benchmark-demo periodic ProwJob provides a live example of how to use benchmarkjunit and what the results look like. Examples:

Translation Example:

The following is the go test benchmark output and the corresponding JUnit XML for the experiment/dummybenchmarks package. This was generated by running go run ./pkg/benchmarkjunit ./experiment/dummybenchmarks -l testlog.txt.

Benchmark output:
goos: linux
goarch: amd64
pkg: k8s.io/test-infra/experiment/dummybenchmarks
BenchmarkCoreSimple-12            	 5000000	       231 ns/op
BenchmarkCoreAllocsAndBytes-12    	10000000	       232 ns/op	  85.84 MB/s	     112 B/op	       1 allocs/op
BenchmarkCoreParallel-12          	50000000	        28.9 ns/op
BenchmarkCoreLog-12               	10000000	       224 ns/op
--- BENCH: BenchmarkCoreLog-12
    dummy_test.go:53: About to DoTheThing() x1.
    dummy_test.go:53: About to DoTheThing() x100.
    dummy_test.go:53: About to DoTheThing() x10000.
    dummy_test.go:53: About to DoTheThing() x1000000.
    dummy_test.go:53: About to DoTheThing() x10000000.
--- SKIP: BenchmarkCoreSkip
    dummy_test.go:60: This Benchmark is skipped.
--- SKIP: BenchmarkCoreSkipNow
--- FAIL: BenchmarkCoreError
    dummy_test.go:68: Early Benchmark error.
    dummy_test.go:53: About to DoTheThing() x1.
--- FAIL: BenchmarkCoreFatal
    dummy_test.go:73: This Benchmark failed.
--- FAIL: BenchmarkCoreFailNow
BenchmarkCoreNestedShallow/simple-12         	 5000000	       236 ns/op
BenchmarkCoreNestedShallow/parallel-12       	50000000	        28.9 ns/op
Benchmark-12                                 	10000000	       226 ns/op
BenchmarkReportAllocs-12                     	10000000	       234 ns/op	     112 B/op	       1 allocs/op
BenchmarkSetBytes-12                         	 5000000	       240 ns/op	  83.00 MB/s
BenchmarkNestedDeep/1_simple-12              	 5000000	       228 ns/op
BenchmarkNestedDeep/1_parallel-12            	50000000	        30.5 ns/op
BenchmarkNestedDeep/3A_simple-12             	10000000	       225 ns/op
BenchmarkNestedDeep/3A-12                    	       1	2453493929 ns/op
BenchmarkNestedDeep/3B_parallel-12           	50000000	        33.1 ns/op
BenchmarkNestedDeep/3B-12                    	       1	1697677606 ns/op
BenchmarkNestedDeep/2-12                     	       1	4152090400 ns/op
BenchmarkNestedDeep/1-12                     	       1	7125087855 ns/op
FAIL
exit status 1
FAIL	k8s.io/test-infra/experiment/dummybenchmarks	25.419s
JUnit XML:
<testsuites>
    <testsuite name="k8s.io/test-infra/experiment/dummybenchmarks" time="25.419" failures="3" tests="20">
        <testcase name="BenchmarkCoreSimple-12" time="1.155" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="5000000"></property>
                <property name="avg op duration (ns/op)" value="231"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreAllocsAndBytes-12" time="2.32" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="10000000"></property>
                <property name="avg op duration (ns/op)" value="232"></property>
                <property name="MB/s" value="85.84"></property>
                <property name="alloced B/op" value="112"></property>
                <property name="allocs/op" value="1"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreParallel-12" time="1.445" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="50000000"></property>
                <property name="avg op duration (ns/op)" value="28.9"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreLog-12" time="2.24" classname="dummybenchmarks">
            <system-out>dummy_test.go:53: About to DoTheThing() x1.&#xA;dummy_test.go:53: About to DoTheThing() x100.&#xA;dummy_test.go:53: About to DoTheThing() x10000.&#xA;dummy_test.go:53: About to DoTheThing() x1000000.&#xA;dummy_test.go:53: About to DoTheThing() x10000000.&#xA;</system-out>
            <properties>
                <property name="op count" value="10000000"></property>
                <property name="avg op duration (ns/op)" value="224"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreSkip" time="0" classname="dummybenchmarks">
            <skipped>dummy_test.go:60: This Benchmark is skipped.&#xA;</skipped>
        </testcase>
        <testcase name="BenchmarkCoreSkipNow" time="0" classname="dummybenchmarks">
            <skipped></skipped>
        </testcase>
        <testcase name="BenchmarkCoreError" time="0" classname="dummybenchmarks">
            <failure>dummy_test.go:68: Early Benchmark error.&#xA;dummy_test.go:53: About to DoTheThing() x1.&#xA;</failure>
            <properties>
                <property name="categorized_fail" value="dummy_test.go:68: Early Benchmark error.&#xA;dummy_test.go:53: About to DoTheThing() x1.&#xA;"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreFatal" time="0" classname="dummybenchmarks">
            <failure>dummy_test.go:73: This Benchmark failed.&#xA;</failure>
            <properties>
                <property name="categorized_fail" value="dummy_test.go:73: This Benchmark failed.&#xA;"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreFailNow" time="0" classname="dummybenchmarks">
            <failure></failure>
        </testcase>
        <testcase name="BenchmarkCoreNestedShallow/simple-12" time="1.18" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="5000000"></property>
                <property name="avg op duration (ns/op)" value="236"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkCoreNestedShallow/parallel-12" time="1.445" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="50000000"></property>
                <property name="avg op duration (ns/op)" value="28.9"></property>
            </properties>
        </testcase>
        <testcase name="Benchmark-12" time="2.26" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="10000000"></property>
                <property name="avg op duration (ns/op)" value="226"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkReportAllocs-12" time="2.34" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="10000000"></property>
                <property name="avg op duration (ns/op)" value="234"></property>
                <property name="alloced B/op" value="112"></property>
                <property name="allocs/op" value="1"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkSetBytes-12" time="1.2" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="5000000"></property>
                <property name="avg op duration (ns/op)" value="240"></property>
                <property name="MB/s" value="83.00"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/1_simple-12" time="1.14" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="5000000"></property>
                <property name="avg op duration (ns/op)" value="228"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/1_parallel-12" time="1.525" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="50000000"></property>
                <property name="avg op duration (ns/op)" value="30.5"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/3A_simple-12" time="2.25" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="10000000"></property>
                <property name="avg op duration (ns/op)" value="225"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/3A-12" time="2.453493929" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="1"></property>
                <property name="avg op duration (ns/op)" value="2453493929"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/3B_parallel-12" time="1.655" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="50000000"></property>
                <property name="avg op duration (ns/op)" value="33.1"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/3B-12" time="1.697677606" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="1"></property>
                <property name="avg op duration (ns/op)" value="1697677606"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/2-12" time="4.1520904" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="1"></property>
                <property name="avg op duration (ns/op)" value="4152090400"></property>
            </properties>
        </testcase>
        <testcase name="BenchmarkNestedDeep/1-12" time="7.125087855" classname="dummybenchmarks">
            <properties>
                <property name="op count" value="1"></property>
                <property name="avg op duration (ns/op)" value="7125087855"></property>
            </properties>
        </testcase>
    </testsuite>
</testsuites>

Documentation

The Go Gopher

There is no documentation for this package.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.