schematesting

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 16, 2024 License: Apache-2.0 Imports: 9 Imported by: 0

README

Schema & Indexer Testing

This module contains core test utilities and fixtures for testing cosmossdk.io/schema and cosmossdk.io/schema/indexer functionality. It is managed as a separate go module to manage versions better and allow for dependencies on useful testing libraries without imposing those elsewhere.

The two primary intended uses of this library are:

  • testing that indexers can handle all valid app data that they might be asked to index
  • testing that state management frameworks properly map their data to and from schema types

Testing Indexers

Indexers are expected to process all valid schema and appdata types, yet it may be hard to find a data set in the wild that comprehensively represents the full valid range of these types. This library provides utilities for simulating such data. The simplest way for an indexer to leverage this test framework is to implement the appdatasim.HasAppData type against their data store. Then the appdatasim.Simulator can be used to generate deterministically random valid data that can be sent to the indexer and also stored in the simulator. After each generated block is applied, appdatasim.DiffAppData can be used to compare the expected state in the simulator to the actual state in the indexer.

This example code shows how this might look in a test:

func TestMyIndexer(t *testing.T) {
	var myIndexerListener appdata.Listener
	var myIndexerAppData appdatasim.HasAppData
    // do the setup for your indexer and return an appdata.Listener to consume updates and the appdatasim.HasAppData instance to check the actual vs expected data
    myIndexerListener, myIndexerAppData := myIndexer.Setup() 
	
    simulator, err := appdatasim.NewSimulator(appdatatest.SimulatorOptions{
        AppSchema: indexertesting.ExampleAppSchema,
        StateSimOptions: statesim.Options{
            CanRetainDeletions: true,
        },
		Listener: myIndexerListener,
    })
    require.NoError(t, err)
    
    blockDataGen := simulator.BlockDataGen()
    for i := 0; i < 1000; i++ {
		// using Example generates a deterministic data set based
        // on a seed so that regression tests can be created OR rapid.Check can
        // be used for fully random property-based testing
        blockData := blockDataGen.Example(i)
		
        // process the generated block data with the simulator which will also
        // send it to the indexer
        require.NoError(t, simulator.ProcessBlockData(blockData))
		
        // compare the expected state in the simulator to the actual state in the indexer and expect the diff to be empty
		require.Empty(t, appdatasim.DiffAppData(simulator, myIndexerAppData))
    }
}

Testing State Management Frameworks

The compliance of frameworks like cosmossdk.io/collections and cosmossdk.io/orm with cosmossdk.io/schema can be tested with this framework. One example of how this might be done is if there is a KeyCodec that represents an array of schema.Fields then schematesting.ObjectKeyGen might be used to generate a random object key which encoded and then decoded and then schematesting.DiffObjectKeys is used to compare the expected key with the decoded key. If such state management frameworks require that users that schema compliance when implementing things like KeyCodecs then those state management frameworks should specify best practices for users.

Documentation

Overview

Package schematesting includes property-based testing generators for creating random valid data for testing schemas and state representing those schemas.

Index

Constants

This section is empty.

Variables

View Source
var AppSchemaGen = rapid.Custom(func(t *rapid.T) map[string]schema.ModuleSchema {
	schema := make(map[string]schema.ModuleSchema)
	numModules := rapid.IntRange(1, 10).Draw(t, "numModules")
	for i := 0; i < numModules; i++ {
		moduleName := NameGen.Draw(t, "moduleName")
		moduleSchema := ModuleSchemaGen().Draw(t, fmt.Sprintf("moduleSchema[%s]", moduleName))
		schema[moduleName] = moduleSchema
	}
	return schema
})

AppSchemaGen generates random valid app schemas, essentially a map of module names to module schemas.

View Source
var ExampleAppSchema = map[string]schema.ModuleSchema{
	"all_kinds": mkAllKindsModule(),
	"test_cases": schema.MustCompileModuleSchema(
		schema.StateObjectType{
			Name:      "Singleton",
			KeyFields: []schema.Field{},
			ValueFields: []schema.Field{
				{
					Name: "Value",
					Kind: schema.StringKind,
				},
				{
					Name: "Value2",
					Kind: schema.BytesKind,
				},
			},
		},
		schema.StateObjectType{
			Name: "Simple",
			KeyFields: []schema.Field{
				{
					Name: "Key",
					Kind: schema.StringKind,
				},
			},
			ValueFields: []schema.Field{
				{
					Name: "Value1",
					Kind: schema.Int32Kind,
				},
				{
					Name: "Value2",
					Kind: schema.BytesKind,
				},
			},
		},
		schema.StateObjectType{
			Name: "TwoKeys",
			KeyFields: []schema.Field{
				{
					Name: "Key1",
					Kind: schema.StringKind,
				},
				{
					Name: "Key2",
					Kind: schema.Int32Kind,
				},
			},
		},
		schema.StateObjectType{
			Name: "ThreeKeys",
			KeyFields: []schema.Field{
				{
					Name: "Key1",
					Kind: schema.StringKind,
				},
				{
					Name: "Key2",
					Kind: schema.Int32Kind,
				},
				{
					Name: "Key3",
					Kind: schema.Uint64Kind,
				},
			},
			ValueFields: []schema.Field{
				{
					Name: "Value1",
					Kind: schema.Int32Kind,
				},
			},
		},
		schema.StateObjectType{
			Name: "ManyValues",
			KeyFields: []schema.Field{
				{
					Name: "Key",
					Kind: schema.StringKind,
				},
			},
			ValueFields: []schema.Field{
				{
					Name: "Value1",
					Kind: schema.Int32Kind,
				},
				{
					Name: "Value2",
					Kind: schema.BytesKind,
				},
				{
					Name: "Value3",
					Kind: schema.Float64Kind,
				},
				{
					Name: "Value4",
					Kind: schema.Uint64Kind,
				},
			},
		},
		schema.StateObjectType{
			Name: "RetainDeletions",
			KeyFields: []schema.Field{
				{
					Name: "Key",
					Kind: schema.StringKind,
				},
			},
			ValueFields: []schema.Field{
				{
					Name: "Value1",
					Kind: schema.Int32Kind,
				},
				{
					Name: "Value2",
					Kind: schema.BytesKind,
				},
			},
			RetainDeletions: true,
		},
	),
}

ExampleAppSchema is an example app schema that intends to cover all schema cases that indexers should handle that can be used in reproducible unit testing and property based testing.

NameGen validates valid names that match the NameFormat regex.

Functions

func CompareKindValues

func CompareKindValues(kind schema.Kind, expected, actual any) (bool, error)

CompareKindValues compares the expected and actual values for the provided kind and returns true if they are equal, false if they are not, and an error if the types are not valid for the kind. For IntegerKind and DecimalKind values, comparisons are made based on equality of the underlying numeric values rather than their string encoding.

func DiffFieldValues

func DiffFieldValues(field schema.Field, expected, actual any) string

DiffFieldValues compares the values for the provided field and returns a diff if they differ or an empty string if they are equal.

func DiffObjectKeys

func DiffObjectKeys(fields []schema.Field, expected, actual any) string

DiffObjectKeys compares the values as object keys for the provided field and returns a diff if they differ or an empty string if they are equal.

func DiffObjectValues

func DiffObjectValues(fields []schema.Field, expected, actual any) string

DiffObjectValues compares the values as object values for the provided field and returns a diff if they differ or an empty string if they are equal. Object values cannot be ValueUpdates for this comparison.

func EnumType

func EnumType() *rapid.Generator[schema.EnumType]

EnumType generates random valid EnumTypes.

func FieldGen

func FieldGen(typeSet schema.TypeSet) *rapid.Generator[schema.Field]

FieldGen generates random Field's based on the validity criteria of fields.

func FieldValueGen

func FieldValueGen(field schema.Field, typeSet schema.TypeSet) *rapid.Generator[any]

FieldValueGen generates random valid values for the field, aiming to exercise the full range of possible values for the field.

func KeyFieldGen

func KeyFieldGen(typeSet schema.TypeSet) *rapid.Generator[schema.Field]

KeyFieldGen generates random key fields based on the validity criteria of key fields.

func ModuleSchemaGen

func ModuleSchemaGen() *rapid.Generator[schema.ModuleSchema]

ModuleSchemaGen generates random ModuleSchema's based on the validity criteria of module schemas.

func ObjectKeyGen

func ObjectKeyGen(keyFields []schema.Field, typeSet schema.TypeSet) *rapid.Generator[any]

ObjectKeyGen generates a value that is valid for the provided object key fields.

func ObjectKeyString

func ObjectKeyString(objectType schema.StateObjectType, key any) string

ObjectKeyString formats the object key as a string deterministically for storage in a map. The key must be valid for the object type and the object type must be valid. No validation is performed here.

func ObjectValueGen

func ObjectValueGen(valueFields []schema.Field, forUpdate bool, typeSet schema.TypeSet) *rapid.Generator[any]

ObjectValueGen generates a value that is valid for the provided object value fields. The forUpdate parameter indicates whether the generator should generate value that are valid for insertion (in the case forUpdate is false) or for update (in the case forUpdate is true). Values that are for update may skip some fields in a ValueUpdates instance whereas values for insertion will always contain all values.

func StateObjectInsertGen

func StateObjectInsertGen(objectType schema.StateObjectType, typeSet schema.TypeSet) *rapid.Generator[schema.StateObjectUpdate]

StateObjectInsertGen generates object updates that are valid for insertion.

func StateObjectTypeGen

func StateObjectTypeGen(typeSet schema.TypeSet) *rapid.Generator[schema.StateObjectType]

StateObjectTypeGen generates random StateObjectType's based on the validity criteria of object types.

func StateObjectUpdateGen

StateObjectUpdateGen generates object updates that are valid for updates using the provided state map as a source of valid existing keys.

Types

This section is empty.

Directories

Path Synopsis
Package appdatasim contains utilities for simulating valid streams of app data for testing indexer implementations.
Package appdatasim contains utilities for simulating valid streams of app data for testing indexer implementations.
Package statesim contains utilities for simulating state based on ObjectType's and ModuleSchema's for testing the conformance of state management libraries and indexers to schema rules.
Package statesim contains utilities for simulating state based on ObjectType's and ModuleSchema's for testing the conformance of state management libraries and indexers to schema rules.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL