api

package
v3.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 29, 2024 License: MIT Imports: 29 Imported by: 0

README

API Design

We are using a documentation driven process.

The API is defined using OpenAPI v2 in indexer.oas2.yml.

Updating REST API

The Makefile will install our fork of oapi-codegen, use make oapi-codegen to install it directly.

  1. Document your changes by editing indexer.oas2.yml
  2. Regenerate the endpoints by running generate.sh. The sources at generated/ will be updated.
  3. Update the implementation in handlers.go. It is sometimes useful to consult generated/routes.go to make sure the handler properly implements ServerInterface.

What codegen tool is used?

We found that oapi-codegen produced the cleanest code, and had an easy to work with codebase. There is an algorand fork of this project which contains a couple modifications that were needed to properly support our needs.

Specifically, uint64 types aren't strictly supported by OpenAPI. So we added a type-mapping feature to oapi-codegen.

Why do we have indexer.oas2.yml and indexer.oas3.yml?

We chose to maintain V2 and V3 versions of the spec because OpenAPI v3 doesn't seem to be widely supported. Some tools worked better with V3 and others with V2, so having both available has been useful. To reduce developer burdon, the v2 specfile is automatically converted v3 using converter.swagger.io.

Fixtures Test

What is a Fixtures Test?

Currently (September 2022) fixtures_test.go is a library that allows testing Indexer's router to verify that endpoints accept parameters and respond as expected, and guard against future regressions. app_boxes_fixtures_test.go is an example fixtures test and is the creator of the fixture boxes.json.

A fixtures test

  1. is defined by a go-slice called a Seed Fixture e.g. var boxSeedFixture which contains request information for making HTTP requests against an Indexer server
  2. iterates through the slice, making each of the defined requests and generating a Live Fixture
  3. reads a Saved Fixture from a json file e.g. boxes.json
  4. persists the Live Fixture to a json file not in source control
  5. asserts that the Saved Fixture is equal to the Live Fixture

In reality, because we always want to save the Live Fixture before making assertions that could fail the test and pre-empt saving, steps (3) and (4) happen in the opposite order.

What's the purpose of a Fixtures Test?

A fixtures test should allow one to quickly stand up an end-to-end test to validate that Indexer endpoints are working as expected. After Indexer's state is programmatically set up, it's easy to add new requests and verify that the responses look exactly as expected. Once you're satisfied that the responses are correct, it's easy to freeze the test and guard against future regressions.

What does a Fixtures Test Function Look Like?

func TestBoxes shows the basic structure of a fixtures test.

  1. setupIdbAndReturnShutdownFunc() is called to set up the Indexer database

    • this isn't expected to require modification

  2. setupLiveBoxes() is used to prepare the local ledger and process blocks in order to bring Indexer into a particular state

    • this will always depend on what the test is trying to achieve
    • in this case, an app was used to create and modify a set of boxes which are then queried against
    • it is conceivable that instead of bringing Indexer into a particular state, the responses from the DB or even the handler may be mocked, so we could have had setupLiveBoxesMocker() instead of setupLiveBoxes()

  3. setupLiveServerAndReturnShutdownFunc() is used to bring up an instance of a real Indexer.

    • this shouldn't need to be modified; however, if running in parallel and making assertions that conflict with other tests, you may need to localize the variable fixtestListenAddr and run on a separate port
    • if running a mock server instead, a different setup function would be needed

  4. validateLiveVsSaved() runs steps (1) through (5) defined in the previous section

    • this is designed to be generic and ought not require much modification going forward
Which Endpoints are Currently Testable in a Fixtures Test?

Endpoints defined in proverRoutes are testable.

Currently (September 2022) these are:

  • /v2/accounts
  • /v2/applications
  • /v2/applications/:application-id
  • /v2/applications/:application-id/box
  • /v2/applications/:application-id/boxes
How to Introduce a New Fixtures Test for an Already Testable Endpoint?

To set up a new test for endpoints defined above one needs to:

1. Define a new Seed Fixture

For example, consider

var boxSeedFixture = fixture{
	File:   "boxes.json",
	Owner:  "TestBoxes",
	Frozen: true,
	Cases: []testCase{
		// /v2/accounts - 1 case
		{
			Name: "What are all the accounts?",
			Request: requestInfo{
				Path:   "/v2/accounts",
				Params: []param{},
			},
		},
        ...

A seed fixture is a struct with fields

  • File (required) - the name in test_resources where the fixture is read from (and written to with an _ prefix)

  • Owner (recommended) - a name to define which test "owns" the seed

  • Frozen (required) - set true when you need to run assertions of the Live Fixture vs. the Saved Fixture. For tests to pass, it needs to be set true.

  • Cases - the slice of testCases. Each of these has the fields:

    • Name (required) - an identifier for the test case
    • Request (required) - a requestInfo struct specifying:
      • Path (required) - the path to be queried
      • Params (required but may be empty) - the slice of parameters (strings name and value) to be appended to the path
2. Define a new Indexer State Setup Function

There are many examples of setting up state that can be emulated. For example:

How to Make a New Endpoint Testable by Fixtures Tests?

There are 2 steps:

  1. Implement a new function witness generator aka prover function of type func(responseInfo) (interface{}, *string) as examplified in this section. Such a function is supposed to parse an Indexer response's body into a generated model. Currently, all provers are boilerplate, and with generics, it's expected that this step will no longer be necessary (this POC shows how it would be done with generics).
  2. Define a new route in the proverRoutes struct. This is a tree structure which is traversed by splitting a path using / and eventually reaching a leaf which consists of a prover as defined in #1.

For example, to enable the endpoint GET /v2/applications/{application-id}/logs for fixtures test, one need only define a logsProof witness generator and have it mapped in proverRoutes under:

proverRoutes.parts["v2"].parts["applications"].parts[":application-id"].parts["logs"] = logsProof
How to Fix a Fixtures Test?

Supposing that there was a breaking change upstream, and a fixtures test is now failing. The following approach should work most of the time:

  1. Run the broken test generating a temporary fixture file in the fixturesDirectory (currently test_resources) with a name the same as the json fixture except begining with _ (e.g. _boxes.json vs. boxes.json).
  2. Observe the diff between the temporary fixture and the saved fixture. If the diff is acceptable, then simply copy the temporary fixture over the saved fixture.
  3. If the diff isn't acceptable, then make any necessary changes to the setup and seed and repeat steps 1 and 2.

Documentation

Index

Constants

View Source 
const (
	ErrNoBoxesFound    = "no application boxes found"
	ErrWrongAppidFound = "the wrong application-id was found, please contact us, this shouldn't happen"
	ErrWrongBoxFound   = "a box with an unexpected name was found, please contact us, this shouldn't happen"
	ErrNoAccountsFound = "no accounts found for address"

	ErrMultipleBoxes        = "multiple application boxes found for this app id and box name, please contact us, this shouldn't happen"
	ErrFailedLookingUpBoxes = "failed while looking up application boxes"

	ErrResultLimitReached = "Result limit exceeded"

	ErrBoxNotFound = "box not found"
)

constant error messages.

Variables

View Source 
var ErrVerifyFailedEndpoint error = fmt.Errorf("endpoint is disabled")

ErrVerifyFailedEndpoint an error that signifies that the entire endpoint is disabled

Functions

func Serve 

func Serve(ctx context.Context, serveAddr string, db idb.IndexerDb, dataError func() error, log *log.Logger, options ExtraOptions)

Serve starts an http server for the indexer API. This call blocks.

func Verify 

func Verify(dm *DisabledMap, nameOfHandlerFunc string, ctx echo.Context, log DisabledParameterErrorReporter) error

Verify returns nil if the function can continue (i.e. the parameters are valid and disabled parameters are not supplied), otherwise VerifyFailedEndpoint if the endpoint failed and VerifyFailedParameter if a disabled parameter was provided.

Types

type DisabledMap 

type DisabledMap struct {
	// Key -> Function Name/Operation ID
	Data map[string]*EndpointConfig
}

DisabledMap a type that holds a map of disabled types The key for a disabled map is the handler function name

func MakeDisabledMap 

func MakeDisabledMap() *DisabledMap

MakeDisabledMap creates a new empty disabled map

func MakeDisabledMapFromOA3 

func MakeDisabledMapFromOA3(swag *openapi3.T, config *DisabledMapConfig) (*DisabledMap, error)

MakeDisabledMapFromOA3 Creates a new disabled map from an openapi3 definition

type DisabledMapConfig 

type DisabledMapConfig struct {
	// Key -> Path of REST endpoint (i.e. /v2/accounts/{account-id}/transactions)
	// Value -> Operation "get, post, etc" -> Sub-value: List of parameters disabled for that endpoint
	Data map[string]map[string][]string
}

DisabledMapConfig is a type that holds the configuration for setting up a DisabledMap

func GetDefaultDisabledMapConfigForPostgres 

func GetDefaultDisabledMapConfigForPostgres() *DisabledMapConfig

GetDefaultDisabledMapConfigForPostgres will generate a configuration that will block certain parameters. Should be used only for the postgres implementation

func MakeDisabledMapConfig 

func MakeDisabledMapConfig() *DisabledMapConfig

MakeDisabledMapConfig creates a new disabled map configuration with everything enabled

func MakeDisabledMapConfigFromFile 

func MakeDisabledMapConfigFromFile(swag *openapi3.T, filePath string) (*DisabledMapConfig, error)

MakeDisabledMapConfigFromFile loads a file containing a disabled map configuration.

type DisabledParameterErrorReporter 

type DisabledParameterErrorReporter interface {
	Errorf(format string, args ...interface{})
}

DisabledParameterErrorReporter defines an error reporting interface for the Verify functions

type DisplayDisabledMap 

type DisplayDisabledMap struct {
	// A complicated map but necessary to output the correct YAML.
	// This is supposed to represent a data structure with a similar YAML
	// representation:
	//  /v2/accounts/{account-id}:
	//    required:
	//      - account-id : enabled
	//  /v2/accounts:
	//    optional:
	//      - auth-addr : enabled
	//      - next: disabled
	Data map[string]map[string][]map[string]string
}

DisplayDisabledMap is a struct that contains the necessary information to output the current config to the screen

func MakeDisplayDisabledMapFromConfig 

func MakeDisplayDisabledMapFromConfig(swag *openapi3.T, mapConfig *DisabledMapConfig, limited bool) *DisplayDisabledMap

MakeDisplayDisabledMapFromConfig will make a DisplayDisabledMap that takes into account the DisabledMapConfig. If limited is set to true, then only disabled parameters will be added to the DisplayDisabledMap

func (*DisplayDisabledMap) String 

func (ddm *DisplayDisabledMap) String() (string, error)

type EndpointConfig 

type EndpointConfig struct {
	EndpointDisabled           bool
	DisabledOptionalParameters map[string]bool
}

EndpointConfig is a data structure that contains whether the endpoint is disabled (with a boolean) as well as a set that contains disabled optional parameters. The disabled optional parameter set is keyed by the name of the variable

type ErrDisabledMapConfig 

type ErrDisabledMapConfig struct {
	// Key -> REST Path that was mis-spelled
	// Value -> Operation "get, post, etc" -> Sub-value: Any parameters that were found to be mis-spelled in a valid REST PATH
	BadEntries map[string]map[string][]string
}

ErrDisabledMapConfig contains any mis-spellings that could be present in a configuration

func (*ErrDisabledMapConfig) Error 

func (edmc *ErrDisabledMapConfig) Error() string

type ErrVerifyFailedParameter 

type ErrVerifyFailedParameter struct {
	ParameterName string
}

ErrVerifyFailedParameter an error that signifies that a parameter was provided when it was disabled

func (ErrVerifyFailedParameter) Error 

func (evfp ErrVerifyFailedParameter) Error() string

type ExtraOptions 

type ExtraOptions struct {
	// Tokens are the access tokens which can access the API.
	Tokens []string

	// Respond to Private Network Access preflight requests sent to the indexer.
	EnablePrivateNetworkAccessHeader bool

	// MetricsEndpoint turns on the /metrics endpoint for prometheus metrics.
	MetricsEndpoint bool

	// MetricsEndpointVerbose generates separate histograms based on query parameters on the /metrics endpoint.
	MetricsEndpointVerbose bool

	// Maximum amount of time to wait before timing out writes to a response. Note that handler timeout is computed
	//off of this.
	WriteTimeout time.Duration

	// ReadTimeout is the maximum duration for reading the entire request, including the body.
	ReadTimeout time.Duration

	// DisabledMapConfig is the disabled map configuration that is being used by the server
	DisabledMapConfig *DisabledMapConfig

	// MaxAPIResourcesPerAccount is the maximum number of combined AppParams, AppLocalState, AssetParams,
	// and AssetHolding resources per address that can be returned by the /v2/accounts endpoints.
	// If an address exceeds this number, a 400 error is returned. Zero means unlimited.
	MaxAPIResourcesPerAccount uint64

	// Transactions
	MaxTransactionsLimit     uint64
	DefaultTransactionsLimit uint64

	// Accounts
	MaxAccountsLimit     uint64
	DefaultAccountsLimit uint64

	// Assets
	MaxAssetsLimit     uint64
	DefaultAssetsLimit uint64

	// Asset Balances
	MaxBalancesLimit     uint64
	DefaultBalancesLimit uint64

	// Applications
	MaxApplicationsLimit     uint64
	DefaultApplicationsLimit uint64

	// Boxes
	MaxBoxesLimit     uint64
	DefaultBoxesLimit uint64
}

ExtraOptions are options which change the behavior of the HTTP server.

type ServerImplementation 

type ServerImplementation struct {
	// contains filtered or unexported fields
}

ServerImplementation implements the handler interface used by the generated route definitions.

func (*ServerImplementation) LookupAccountAppLocalStates 

func (si *ServerImplementation) LookupAccountAppLocalStates(ctx echo.Context, accountID string, params generated.LookupAccountAppLocalStatesParams) error

LookupAccountAppLocalStates queries indexer for AppLocalState for a given account, and optionally a given app ID. (GET /v2/accounts/{account-id}/apps-local-state)

func (*ServerImplementation) LookupAccountAssets 

func (si *ServerImplementation) LookupAccountAssets(ctx echo.Context, accountID string, params generated.LookupAccountAssetsParams) error

LookupAccountAssets queries indexer for AssetHolding for a given account, and optionally a given asset ID. (GET /v2/accounts/{account-id}/assets)

func (*ServerImplementation) LookupAccountByID 

func (si *ServerImplementation) LookupAccountByID(ctx echo.Context, accountID string, params generated.LookupAccountByIDParams) error

LookupAccountByID queries indexer for a given account. (GET /v2/accounts/{account-id})

func (*ServerImplementation) LookupAccountCreatedApplications 

func (si *ServerImplementation) LookupAccountCreatedApplications(ctx echo.Context, accountID string, params generated.LookupAccountCreatedApplicationsParams) error

LookupAccountCreatedApplications queries indexer for AppParams for a given account, and optionally a given app ID. (GET /v2/accounts/{account-id}/created-applications)

func (*ServerImplementation) LookupAccountCreatedAssets 

func (si *ServerImplementation) LookupAccountCreatedAssets(ctx echo.Context, accountID string, params generated.LookupAccountCreatedAssetsParams) error

LookupAccountCreatedAssets queries indexer for AssetParams for a given account, and optionally a given asset ID. (GET /v2/accounts/{account-id}/created-assets)

func (*ServerImplementation) LookupAccountTransactions 

func (si *ServerImplementation) LookupAccountTransactions(ctx echo.Context, accountID string, params generated.LookupAccountTransactionsParams) error

LookupAccountTransactions looks up transactions associated with a particular account. (GET /v2/accounts/{account-id}/transactions)

func (*ServerImplementation) LookupApplicationBoxByIDAndName 

func (si *ServerImplementation) LookupApplicationBoxByIDAndName(ctx echo.Context, applicationID uint64, params generated.LookupApplicationBoxByIDAndNameParams) error

LookupApplicationBoxByIDAndName returns the value of an application's box (GET /v2/applications/{application-id}/box)

func (*ServerImplementation) LookupApplicationByID 

func (si *ServerImplementation) LookupApplicationByID(ctx echo.Context, applicationID uint64, params generated.LookupApplicationByIDParams) error

LookupApplicationByID returns one application for the requested ID. (GET /v2/applications/{application-id})

func (*ServerImplementation) LookupApplicationLogsByID 

func (si *ServerImplementation) LookupApplicationLogsByID(ctx echo.Context, applicationID uint64, params generated.LookupApplicationLogsByIDParams) error

LookupApplicationLogsByID returns one application logs (GET /v2/applications/{application-id}/logs)

func (*ServerImplementation) LookupAssetBalances 

func (si *ServerImplementation) LookupAssetBalances(ctx echo.Context, assetID uint64, params generated.LookupAssetBalancesParams) error

LookupAssetBalances looks up balances for a particular asset (GET /v2/assets/{asset-id}/balances)

func (*ServerImplementation) LookupAssetByID 

func (si *ServerImplementation) LookupAssetByID(ctx echo.Context, assetID uint64, params generated.LookupAssetByIDParams) error

LookupAssetByID looks up a particular asset (GET /v2/assets/{asset-id})

func (*ServerImplementation) LookupAssetTransactions 

func (si *ServerImplementation) LookupAssetTransactions(ctx echo.Context, assetID uint64, params generated.LookupAssetTransactionsParams) error

LookupAssetTransactions looks up transactions associated with a particular asset (GET /v2/assets/{asset-id}/transactions)

func (*ServerImplementation) LookupBlock 

func (si *ServerImplementation) LookupBlock(ctx echo.Context, roundNumber uint64, params generated.LookupBlockParams) error

LookupBlock returns the block for a given round number (GET /v2/blocks/{round-number})

func (*ServerImplementation) LookupTransaction 

func (si *ServerImplementation) LookupTransaction(ctx echo.Context, txid string) error

LookupTransaction searches for the requested transaction ID.

func (*ServerImplementation) MakeHealthCheck 

func (si *ServerImplementation) MakeHealthCheck(ctx echo.Context) error

MakeHealthCheck returns health check information about indexer and the IndexerDb being used. Returns 200 if healthy. (GET /health)

func (*ServerImplementation) SearchForAccounts 

func (si *ServerImplementation) SearchForAccounts(ctx echo.Context, params generated.SearchForAccountsParams) error

SearchForAccounts returns accounts matching the provided parameters (GET /v2/accounts)

func (*ServerImplementation) SearchForApplicationBoxes 

func (si *ServerImplementation) SearchForApplicationBoxes(ctx echo.Context, applicationID uint64, params generated.SearchForApplicationBoxesParams) error

SearchForApplicationBoxes returns box names for an app (GET /v2/applications/{application-id}/boxes)

func (*ServerImplementation) SearchForApplications 

func (si *ServerImplementation) SearchForApplications(ctx echo.Context, params generated.SearchForApplicationsParams) error

SearchForApplications returns applications for the provided parameters. (GET /v2/applications)

func (*ServerImplementation) SearchForAssets 

func (si *ServerImplementation) SearchForAssets(ctx echo.Context, params generated.SearchForAssetsParams) error

SearchForAssets returns assets matching the provided parameters (GET /v2/assets)

func (*ServerImplementation) SearchForTransactions 

func (si *ServerImplementation) SearchForTransactions(ctx echo.Context, params generated.SearchForTransactionsParams) error

SearchForTransactions returns transactions matching the provided parameters (GET /v2/transactions)

Source Files

View all Source files

Directories

Path Synopsis
generated
common
Package common provides primitives to interact with the openapi HTTP API.
 Package common provides primitives to interact with the openapi HTTP API.
v2
Package generated provides primitives to interact with the openapi HTTP API.
 Package generated provides primitives to interact with the openapi HTTP API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.