addrundata

README

The addrundata Tool

The addrundata tool keeps all the metadata.textproto files up to date in the featureprofiles repo. These files contain rundata that identify each test, and the rundata will find its way into the test XML output when a functional test is run with the -xml flag. The rundata allow us to track the test result.

There are two modes of operation:

• Check mode: go run ./tools/addrundata

  When run without a flag, this checks the integrity of rundata in the repo. This is used by the "Rundata Check" pull request check. If the check fails, we will not be able to track the test result from those tests with outdated rundata.

• Fix mode: go run ./tools/addrundata --fix

  This will update any outdated rundata, to be run by the author of a pull request if the "Rundata Check" fails.

An example metadata.textproto looks like this:

# proto-file: proto/metadata.proto
# proto-message: Metadata

uuid: "bf60afdc-7130-4bef-a23c-39783c7f2bb3"
plan_id: "XX-1.1"
description: "Foo Functional Test"

Both plan_id and description are sourced from the top-level heading in README.md:

# XX-1.1: Foo Functional Test

## Summary

One line summary of what foo functional test does.

But the uuid is uniquely generated for each test. The addrundata tool takes care of the UUID generation. Both the ate_tests and otg_tests variants of the same test must have the same rundata.

Documentation

Overview

Command addrundata adds or updates rundata reporting to all tests in the source code, based on each of their README.md.

Tests are found under feature/${feature}/${subfeature}/${testkind}/${testname} where testkind is one of "ate_tests", "otg_tests", or simply "tests". If an ATE test is present, it should have the same rundata as the OTG test.

The rundata is stored in the metadata.textproto file in the test package. Other test files are left unchanged.

Test plan ID and the description are extracted from the README.md, whereas the UUID is randomly assigned. Existing UUID assignments are honored. ATE and OTG versions of the same test must have the same UUID.