tests/

README

Tests

The tests directory contains integration tests that require a pre-seeded database.

Each tests/ contains a seed.txt file at tests//seed.txt, which is used by the docker seeddb container to setup the test database.

The tests can be run using tests//run.sh.

Search Tests

The tests/search/scripts directory contains test scripts *.txt that are run using tests/search/run.sh.

These scripts follow the format:

Test name A
search query A
Expected result A1
Expected result A2

Test name B
search query B
Expected result B1
Expected result B2

...

Each test set is separated by a newline. Comments (lines that being with a #), are ignored.

The imported by counts for a dataset can also be updated in search/importedby.txt. The file has the format:

<package-path>, <imported-by-count>

It is expected that the modules for these packages are in tests/search/seed.txt.

Symbol History API Tests

The tests/api/scripts directory contains tests that are run using tests/api/run.sh.

The tests compare API data for a module tests/api/testdata with the API history on the frontend versions page. The frontend that is used for comparison must have GO_DISCOVERY_SERVE_STATS set so that the /<path>?tab=versions&content=json endpoint is available.

Adding a package

Data in tests/api/testdata is generated by running:

go run tests/api/main.go generate [module path]:[package path suffix]

For example, go run tests/api/main.go generate github.com/julieqiu/api-demo:pkg for package github.com/julieqiu/api-demo/pkg.

This data can now be used to check the API history on a frontend by running:

go run tests/api/main.go compare [module path]:[package path suffix]

Screentest

The screentest/ directory contains visual diff tests for pages on pkg.go.dev. Tests are written for the screentest tool from golang.org/x/website. See the command docs for more information.

Running the tests

In order to run the tests, run this command from the root of the repository:

./tests/screentest/run.sh <env>

./tests/screentest/run.sh ci sets up a series of docker containers that run a postgres database, frontend, and headless chrome, and runs the screentest using headless chrome.

Tests in ./tests/screentest/testcases.ci.txt run only for CI and local runs of the screentest. These tests may rely on mocked data, like the vuln page tests. Tests in ./tests/screentest/testcases.txt run for all environments and are suitable for tests with data that is consistent between environments or static pages.

Understanding Test Failures

If the tests fail with diffs, a file with the diffs will be created in the testdata directory that show the cause of the failure.

Use ./devtools/docker/compose.sh logs to get detailed logs of the docker containers.

Troubleshooting

If docker fails with "connect: cannot assign requested address" during run.sh, try running it again. It will eventually succeed. Do not pass the '-rm' flag so that prior state is preserved. You can also try reducing the value of cmdgonetlimit in devtools/docker/compose.yaml.