protoc-gen-go-aip-test

README

protoc-gen-go-aip-test

Generate test suites for protobuf services implementing standard AIP methods.

The generated test suites are based on guidance for standard methods, and experience from implementing these methods in practice. See Suites for a list of the generated tests.

Experimental: This plugin is experimental, and breaking changes with regard to the generated tests suites should be expected.

Usage

Step 1: Declare a service with AIP standard methods

service FreightService {
  // Get a shipper.
  // See: https://google.aip.dev/131 (Standard methods: Get).
  rpc GetShipper(GetShipperRequest) returns (Shipper) {
    option (google.api.http) = {
      get: "/v1/{name=shippers/*}"
    };
    option (google.api.method_signature) = "name";
  }

  // ...
}

Step 2: Install the generator

Either install using go install:

go install github.com/einride/protoc-gen-go-aip-test@latest

Or download a prebuilt binary from releases and put it in your PATH.

The generator can also be built from source using Go.

Step 3: Generate test suites

Include the plugin in protoc invocation

protoc
  --go-aip-test_out=[OUTPUT DIR] \
  --go-aip-test_opt=module=[OUTPUT MODULE] \
  [.proto files ...]

This can also be done via a buf generate template. See buf.gen.yaml for an example.

Step 4: Run tests

There are two alternative ways of bootstrapping the tests.

Alternative 1:

Instantiate the generated test suites and call the methods you want to test.

package example

func Test_FreightService(t *testing.T) {
	t.Skip("this is just an example, the service is not implemented.")
	// setup server before test
	server := examplefreightv1.UnimplementedFreightServiceServer{}
	// setup test suite
	suite := examplefreightv1.FreightServiceTestSuite{
		T:      t,
		Server: server,
	}

	// run tests for each resource in the service
	ctx := context.Background()
	suite.TestShipper(ctx, examplefreightv1.ShipperTestSuiteConfig{
		// Create should return a resource which is valid to create, i.e.
		// all required fields set.
		Create: func() *examplefreightv1.Shipper {
			return &examplefreightv1.Shipper{
				DisplayName:    "Example shipper",
				BillingAccount: "billingAccounts/12345",
			}
		},
		// Update should return a resource which is valid to update, i.e.
		// all required fields set.
		Update: func() *examplefreightv1.Shipper {
			return &examplefreightv1.Shipper{
				DisplayName:    "Updated example shipper",
				BillingAccount: "billingAccounts/54321",
			}
		},
	})
}

Alternative 2:

Implement the generated configure provider interface (FreightServiceTestSuiteConfigProvider) and pass the implementation to TestServices to start the tests.

A benefit of using TestServices (over alternative 1) is that as new services or resources are added to the API the test code won't compile until the required inputs are also added (or explicitly ignored). This makes it harder to forget to add the test implementations for new services/resources.

package example

import "testing"

func Test_FreightService(t *testing.T) {
	// Even though no implementation exists, the tests will pass but be skipped.
	examplefreightv1.TestServices(t, &aipTests{})
}

type aipTests struct{}

var _ examplefreightv1.FreightServiceTestSuiteConfigProvider = &aipTests{}

func (a aipTests) FreightServiceShipper(_ *testing.T) *examplefreightv1.FreightServiceShipperTestSuiteConfig {
	// Returns nil to indicate that it's not ready to be tested.
	return nil
}

func (a aipTests) FreightServiceSite(_ *testing.T) *examplefreightv1.FreightServiceSiteTestSuiteConfig {
	// Returns nil to indicate that it's not ready to be tested.
	return nil
}

Skipping tests

There may be multiple reasons for an API to deviate from the guidance for standard methods (for examples see AIP-200). This plugin supports skipping individual or groups of tests using the Skip field generated for each test suite config.

Each test are compared, using strings.Contains, against a list of skipped test patterns. The full name of each test will follow the format [resource]/[method type]/[test_name].

Sample skips:

• "Get/invalid_name" skips the "invalid name" test for Get standard method.
• "Get" skips all tests for a Get standard method.

Suites

Create

Name	Description	Only if
missing parent	Method should fail with InvalidArgument if no parent is provided.	Generated only if all are true: has Create method resource has a parent
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	Generated only if all are true: has Create method resource has a parent
create time	Field create_time should be populated when the resource is created.	Generated only if all are true: has Create method Create method does not return long-running operation has field 'create_time'
persisted	The created resource should be persisted and reachable with Get.	Generated only if all are true: has Create method Create method does not return long-running operation has Get method
user settable id	If method support user settable IDs, when set the resource should be returned with the provided ID.	Generated only if all are true: has Create method Create method does not return long-running operation has user settable ID
invalid user settable id	Method should fail with InvalidArgument if the user settable id doesn't conform to RFC-1034, see doc.	Generated only if all are true: has Create method Create method does not return long-running operation has user settable ID
invalid user settable id - uuid	Method should fail with InvalidArgument if the user settable ID appears to be a UUID, see doc.	Generated only if all are true: has Create method Create method does not return long-running operation has user settable ID
already exists	If method support user settable IDs and the same ID is reused the method should return AlreadyExists.	Generated only if all are true: has Create method Create method does not return long-running operation has user settable ID
required fields	The method should fail with InvalidArgument if the resource has any required fields and they are not provided.	Generated only if all are true: has Create method resource has any required fields
resource references	The method should fail with InvalidArgument if the resource has any resource references and they are invalid.	Generated only if all are true: has Create method resource has any mutable resource references
etag populated	Field etag should be populated when the resource is created.	Generated only if all are true: has Create method Create method does not return long-running operation has field 'etag'

Get

Name	Description	Only if
missing name	Method should fail with InvalidArgument if no name is provided.	Generated only if all are true: has Get method
invalid name	Method should fail with InvalidArgument if the provided name is not valid.	Generated only if all are true: has Get method
exists	Resource should be returned without errors if it exists.	Generated only if all are true: has Get method
not found	Method should fail with NotFound if the resource does not exist.	Generated only if all are true: has Get method
only wildcards	Method should fail with InvalidArgument if the provided name only contains wildcards ('-')	Generated only if all are true: has Get method
soft-deleted	A soft-deleted resource should be returned without errors.	Generated only if all are true: has Get method has Delete method has field 'delete_time'

BatchGet

Name	Description	Only if
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	Generated only if all are true: resource has a parent has BatchGet method is not alternative batch request message
names missing	Method should fail with InvalidArgument if no names are provided.	Generated only if all are true: has BatchGet method is not alternative batch request message
invalid names	Method should fail with InvalidArgument if a provided name is not valid.	Generated only if all are true: has BatchGet method is not alternative batch request message
wildcard name	Method should fail with InvalidArgument if a provided name only contains wildcards (-)	Generated only if all are true: has BatchGet method is not alternative batch request message
all exists	Resources should be returned without errors if they exist.	Generated only if all are true: has BatchGet method is not alternative batch request message
atomic	The method must be atomic; it must fail for all resources or succeed for all resources (no partial success).	Generated only if all are true: has BatchGet method is not alternative batch request message
parent mismatch	If a caller sets the "parent", and the parent collection in the name of any resource being retrieved does not match, the request must fail.	Generated only if all are true: resource has a parent has BatchGet method is not alternative batch request message
ordered	The order of resources in the response must be the same as the names in the request.	Generated only if all are true: has BatchGet method is not alternative batch request message
duplicate names	If a caller provides duplicate names, the service should return duplicate resources.	Generated only if all are true: has BatchGet method is not alternative batch request message

Update

Name	Description	Only if
missing name	Method should fail with InvalidArgument if no name is provided.	Generated only if all are true: has Update method
invalid name	Method should fail with InvalidArgument if provided name is not valid.	Generated only if all are true: has Update method
update time	Field update_time should be updated when the resource is updated.	Generated only if all are true: has Create method Create method does not return long-running operation has Update method Update method does not return long-running operation has field 'update_time'
persisted	The updated resource should be persisted and reachable with Get.	Generated only if all are true: has Update method Update method does not return long-running operation has Get method
preserve create_time	The field create_time should be preserved when a '*'-update mask is used.	Generated only if all are true: has Update method Update method does not return long-running operation has field 'create_time' resource has any required fields
etag mismatch	Method should fail with Aborted if the supplied etag doesnt match the current etag value.	Generated only if all are true: has Update method has field 'etag'
etag updated	Field etag should have a new value when the resource is successfully updated.	Generated only if all are true: has Update method has field 'etag'
not found	Method should fail with NotFound if the resource does not exist.	Generated only if all are true: has Update method
invalid update mask	The method should fail with InvalidArgument if the update_mask is invalid.	Generated only if all are true: has Update method Update method has update_mask
required fields	Method should fail with InvalidArgument if any required field is missing when called with '*' update_mask.	Generated only if all are true: has Update method resource has any required fields

List

Name	Description	Only if
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	Generated only if all are true: has List method resource has a parent
invalid page token	Method should fail with InvalidArgument is provided page token is not valid.	Generated only if all are true: has List method
negative page size	Method should fail with InvalidArgument is provided page size is negative.	Generated only if all are true: has List method
isolation	If parent is provided the method must only return resources under that parent.	Generated only if all are true: resource has a parent has List method resource has a parent
last page	If there are no more resources, next_page_token should not be set.	Generated only if all are true: resource has a parent has List method resource has a parent
more pages	If there are more resources, next_page_token should be set.	Generated only if all are true: resource has a parent has List method resource has a parent
one by one	Listing resource one by one should eventually return all resources.	Generated only if all are true: resource has a parent has List method resource has a parent
deleted	Method should not return deleted resources.	Generated only if all are true: resource has a parent has List method has Delete method resource has a parent

Search

Name	Description	Only if
invalid parent	Method should fail with InvalidArgument if provided parent is invalid.	Generated only if all are true: has Search method resource has a parent
invalid page token	Method should fail with InvalidArgument is provided page token is not valid.	Generated only if all are true: has Search method
negative page size	Method should fail with InvalidArgument is provided page size is negative.	Generated only if all are true: has Search method
isolation	If parent is provided the method must only return resources under that parent.	Generated only if all are true: resource has a parent has Search method resource has a parent
last page	If there are no more resources, next_page_token should not be set.	Generated only if all are true: resource has a parent has Search method resource has a parent
more pages	If there are more resources, next_page_token should be set.	Generated only if all are true: resource has a parent has Search method resource has a parent
one by one	Searching resource one by one should eventually return all resources.	Generated only if all are true: resource has a parent has Search method resource has a parent
deleted	Method should not return deleted resources.	Generated only if all are true: resource has a parent has Search method has Delete method resource has a parent

Delete

Name	Description	Only if
missing name	Method should fail with InvalidArgument if no name is provided.	Generated only if all are true: has Delete method
invalid name	Method should fail with InvalidArgument if the provided name is not valid.	Generated only if all are true: has Delete method
exists	Resource should be deleted without errors if it exists.	Generated only if all are true: has Delete method
not found	Method should fail with NotFound if the resource does not exist.	Generated only if all are true: has Delete method
already deleted	Method should fail with NotFound if the resource was already deleted. This also applies to soft-deletion.	Generated only if all are true: has Delete method
only wildcards	Method should fail with InvalidArgument if the provided name only contains wildcards ('-')	Generated only if all are true: has Delete method
etag mismatch	Method should fail with Aborted if the supplied etag doesnt match the current etag value.	Generated only if all are true: has Delete method request has etag field has field 'etag'