README
¶
Gateway Conformance
A set of GO and HTTP tools for testing implementation compliance with https://specs.ipfs.tech
About
Gateway Conformance test suite is a set of tools for testing implementation
compliance with a subset of IPFS Specifications. The
test suite is implementation and language-agnostic. Point gateway conformance test at HTTP endpoint and specify which tests should run.
IPFS Shipyard uses it for ensuring specification compliance of the boxo/gateway library included in Kubo, the most popular IPFS implementation,
that powers various public gateways, IPFS Desktop, and Brave.
Some scenarios in which you may find this project helpful:
- You are building an product that relies on in-house IPFS Gateway and want to ensure HTTP interface is implemented correctly
- You are building an IPFS implementation and want to leverage existing HTTP test fixtures to tell if you are handling edge cases correctly
- You want to test if a trustless retrieval endpoint supports partial CARs from IPIP-402
- You want to confirm a commercial service provider implemented content-addressing correctly
Usage
The gateway-conformance can be run as a standalone binary, a Docker image, or a part of Github Action.
Some of the tests require the tested gateway to be able to resolve specific fixtures CIDs or IPNS records.
Two high level commands exist:
- test (test runner with ability to specify a subset of tests to run)
- extract-fixtures (allowing for custom provisioning of how test vectors are loaded into tested runtime)
CLI
$ # Install the gateway-conformance binary
$ go install github.com/ipfs/gateway-conformance/cmd/gateway-conformance@latest
$ # run subdomain-gateway tests against endpoint at http://localhost:8080 output as JSON
$ gateway-conformance test --gateway-url http://localhost:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 30m
If you are looking for TLDR, see examples.
Docker
Prebuilt image at ghcr.io/ipfs/gateway-conformance can be used for both test and extract-fixtures commands:
$ # extract fixtures to ./fixtures directory
$ docker run -v "${PWD}:/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance:vA.B.C extract-fixtures --directory fixtures --merged false
$ # run subdomain-gateway tests against endpoint at http://localhost:8080
$ docker run --network host -v "${PWD}:/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance:vA.B.C test --gateway-url http://localhost:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 30m
NOTE: replace vA.B.C with a semantic version version you want to test against
Github Action
Common operations are possible via reusable GitHub actions:
ipfs/gateway-conformance/.github/actions/testipfs/gateway-conformance/.github/actions/extract-fixtures
To learn how to integrate them in the CI of your project, see real world examples in:
kubo/../gateway-conformance.yml(fixtures imported into tested node)boxo/../gateway-conformance.yml(fixtures imported into a sidecar kubo node that is peered with tested library)rainbow/../gateway-conformance.yml(fixtures imported into a kubo node that acts as a delegated block backend)
Web Dashboard
Conformance test suite output can be plain text or JSON, which in turn can be represented as a web dashboard which aggregates results from many test runs and renders them on a static website.
The Implementation Dashboard instance at conformance.ipfs.tech is a view that showcases some of well known and complete implementations of IPFS Gateways in the ecosystem.
Learn more at /docs/web-dashboard.md
Commands
See test and extract-fixtures documentation at /docs/commands.md
Examples
Want to test mature specs, while disabling specific specs?
Or only test a specific spec (like trustless gateway), while disabling a sub-part of it (only blocks and CARS, no IPNS)?
See /docs/examples.md
Releases
The main branch may contain tests for features and IPIPs which are not yet
supported by stable releases of IPFS implementations.
Due to this, implementations SHOULD test themselves against a stable release of this test suite instead.
See /releases for the list of available releases.
Development
Want to improve the conformance test suite itself?
See documentation at /docs/development.md
Test DSL Syntax
Interested in write a new test case?
Test cases are written in Domain Specific Language (DLS) based on Golang.
More details at /docs/test-dsl-syntax.md
License
This project is dual-licensed under Apache 2.0 and MIT terms:
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
Directories