simulation

package
v12.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 10, 2022 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package simulation implements a full fledged Cosmos SDK application used for executing simulation test suites.

Simulation App

The SimApp type defines an application used for running extensive simulation testing suites. It contains all core modules, including governance, staking, slashing, and distribution.

Simulation is executed with various inputs including the number of blocks to simulate, the block size, whether the app should commit or not, the invariant checking period, and a seed which is used as a source of pseudo-randomness.

In addition to the various inputs, simulation runs mainly in three modes:

1. Completely random where the initial state, module parameters and simulation parameters are pseudo-randomly generated.

2. From a genesis file where the initial state and the module parameters are defined. This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested.

3. From a params file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated.

The simulation test suite also supports testing determinism and import/export functionality.

Randomness

Currently, simulation uses a single seed (integer) as a source for a PRNG by which all random operations are executed from. Any call to the PRNG changes all future operations as the internal state of the PRNG is modified. For example, if a new message type is created and needs to be simulated, the new introduced PRNG call will change all subsequent operations.

This may can often be problematic when testing fixes to simulation faults. One current solution to this is to use a params file as mentioned above. In the future the simulation suite is expected to support a series of PRNGs that can be used uniquely per module and simulation component so that they will not effect each others state execution outcome.

Usage

To execute a completely pseudo-random simulation: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-v -timeout 24h

To execute simulation from a genesis file: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-Genesis=/path/to/genesis.json \
 	-v -timeout 24h

To execute simulation from a simulation params file: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-Params=/path/to/params.json \
	-v -timeout 24h

To export the simulation params to a file at a given block height: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportParamsPath=/path/to/params.json \
	-ExportParamsHeight=50 \
	 -v -timeout 24h

To export the simulation app state (i.e genesis) to a file: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportStatePath=/path/to/genesis.json \
	 v -timeout 24h

Params

Params that are provided to simulation from a JSON file are used to used to set both module parameters and simulation parameters. See sim_test.go for the full set of parameters that can be provided.

Index

Constants

View Source 
const (
	BeginBlockEntryKind = "begin_block"
	EndBlockEntryKind   = "end_block"
	MsgEntryKind        = "msg"
	QueuedMsgEntryKind  = "queued_msg"
)

entry kinds for use within OperationEntry

View Source 
const AverageBlockTime = 6 * time.Second

Variables

View Source 
var (
	FlagGenesisFileValue        string
	FlagParamsFileValue         string
	FlagExportParamsPathValue   string
	FlagExportParamsHeightValue int
	FlagExportStatePathValue    string
	FlagExportStatsPathValue    string
	FlagSeedValue               int64
	FlagInitialBlockHeightValue int
	FlagNumBlocksValue          int
	FlagBlockSizeValue          int
	FlagLeanValue               bool
	FlagCommitValue             bool
	FlagOnOperationValue        bool // TODO: Remove in favor of binary search for invariant violation
	FlagAllInvariantsValue      bool
	FlagWriteStatsToDB          bool

	FlagEnabledValue     bool
	FlagVerboseValue     bool
	FlagPeriodValue      uint
	FlagGenesisTimeValue int64
)

List of available flags for the simulator

Functions

func GetSimulatorFlags 

func GetSimulatorFlags()

GetSimulatorFlags gets the values of all the available simulation flags

func PrintStats 

func PrintStats(db dbm.DB)

PrintStats prints the corresponding statistics from the app DB.

func RandomRequestBeginBlock 

func RandomRequestBeginBlock(r *rand.Rand, params Params,
	validators mockValidators, pastTimes []time.Time,
	pastVoteInfos [][]abci.VoteInfo,
	event func(route, op, evResult string), header tmproto.Header,
) abci.RequestBeginBlock

RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction

func RandomSimAccounts 

func RandomSimAccounts(r *rand.Rand, n int) []simulation.Account

RandomAccounts generates n random accounts

func SimulateFromSeed 

func SimulateFromSeed(
	tb testing.TB,
	w io.Writer,
	appCreator simtypes.AppCreator,
	initFunctions InitFunctions,
	config Config,
) (lastCommitId storetypes.CommitID, stopEarly bool, err error)

SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided config.Seed. TODO: Inputs should be: * SimManager for module configs * Config file for params * whatever is needed for logging (tb + w rn) OR: * Could be a struct or something with options, to give caller ability to step through / instrument benchmarking if they wanted to, and add a cleanup function.

Types

type AppStateFn 

type AppStateFn func(simManager *simtypes.Manager, r *rand.Rand, accs []legacysim.Account, config InitializationConfig) (
	appState json.RawMessage, accounts []legacysim.Account, chainId string, genesisTimestamp time.Time,
)

AppStateFn returns the app state json bytes and the genesis accounts

type Config 

type Config struct {
	InitializationConfig InitializationConfig
	ExportConfig         ExportConfig
	ExecutionDbConfig    ExecutionDbConfig

	Seed int64 // simulation random seed

	NumBlocks int // number of new blocks to simulate from the initial block height
	BlockSize int // operations per block

	Lean bool // lean simulation log output

	OnOperation   bool // run slow invariants every operation
	AllInvariants bool // print all failed invariants if a broken invariant is found
}

func NewConfigFromFlags 

func NewConfigFromFlags() Config

NewConfigFromFlags creates a simulation from the retrieved values of the flags.

func SetupSimulation 

func SetupSimulation(dirPrefix, dbName string) (cfg Config, db dbm.DB, logger log.Logger, cleanup func(), err error)

SetupSimulation creates the config, db (levelDB), temporary directory and logger for the simulation tests. If `FlagEnabledValue` is false it skips the current test. Returns error on an invalid db intantiation or temp dir creation.

type DummyLogWriter 

type DummyLogWriter struct{}

dummy log writter

func (*DummyLogWriter) AddEntry 

func (lw *DummyLogWriter) AddEntry(_ OperationEntry)

do nothing

func (*DummyLogWriter) PrintLogs 

func (lw *DummyLogWriter) PrintLogs()

do nothing

type EmptyAppOptions 

type EmptyAppOptions struct{}

EmptyAppOptions is a stub implementing AppOptions

func (EmptyAppOptions) Get 

func (ao EmptyAppOptions) Get(o string) interface{}

Get implements AppOptions

type EventStats 

type EventStats map[string]map[string]map[string]int

EventStats defines an object that keeps a tally of each event that has occurred during a simulation.

func NewEventStats 

func NewEventStats() EventStats

NewEventStats creates a new empty EventStats object

func (EventStats) ExportJSON 

func (es EventStats) ExportJSON(path string)

ExportJSON saves the event stats as a JSON file on a given path

func (EventStats) Print 

func (es EventStats) Print(w io.Writer)

Print the event stats in JSON format.

func (EventStats) Tally 

func (es EventStats) Tally(route, op, evResult string)

Tally increases the count of a simulation event.

type ExecutionDbConfig 

type ExecutionDbConfig struct {
	UseMerkleTree bool // Use merkle tree underneath, vs using a "fake" merkle tree
}

func NewExecutionDbConfigFromFlags 

func NewExecutionDbConfigFromFlags() ExecutionDbConfig

type ExportConfig 

type ExportConfig struct {
	ExportParamsPath   string // custom file path to save the exported params JSON
	ExportParamsHeight int    // height to which export the randomly generated params
	ExportStatePath    string // custom file path to save the exported app state JSON
	ExportStatsPath    string // custom file path to save the exported simulation statistics JSON
	WriteStatsToDB     bool
}

func NewExportConfigFromFlags 

func NewExportConfigFromFlags() ExportConfig

type InitFunctions 

type InitFunctions struct {
	// Why does this take in Numkeys / why isn't this part of the initial state function / config to decide?
	RandomAccountFn   RandomAccountFn
	AppInitialStateFn AppStateFn
}

TODO: Consider adding consensus parameters / simulator params / tendermint params to this.

func DefaultSimInitFunctions 

func DefaultSimInitFunctions(moduleAccountAddresses map[string]bool) InitFunctions

TODO: cleanup args in the future, should ideally just be a slice.

type InitializationConfig 

type InitializationConfig struct {
	GenesisFile        string // custom simulation genesis file; cannot be used with params file
	ParamsFile         string // custom simulation params file which overrides any random params; cannot be used with genesis
	InitialBlockHeight int    // initial block to start the simulation
	ChainID            string // chain-id used on the simulation
}

Config for how to initialize the simulator state

func NewInitializationConfigFromFlags 

func NewInitializationConfigFromFlags() InitializationConfig

type LogWriter 

type LogWriter interface {
	AddEntry(OperationEntry)
	PrintLogs()
}

log writter

func NewLogWriter 

func NewLogWriter(tb testing.TB) LogWriter

LogWriter - return a dummy or standard log writer given the testingmode

type OperationEntry 

type OperationEntry struct {
	EntryKind string          `json:"entry_kind" yaml:"entry_kind"`
	Height    int64           `json:"height" yaml:"height"`
	Order     int64           `json:"order" yaml:"order"`
	Operation json.RawMessage `json:"operation" yaml:"operation"`
}

OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)

func BeginBlockEntry 

func BeginBlockEntry(height int64) OperationEntry

BeginBlockEntry - operation entry for begin block

func EndBlockEntry 

func EndBlockEntry(height int64) OperationEntry

EndBlockEntry - operation entry for end block

func MsgEntry 

func MsgEntry(height, order int64, opMsg simulation.OperationMsg) OperationEntry

MsgEntry - operation entry for standard msg

func NewOperationEntry 

func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry

NewOperationEntry creates a new OperationEntry instance

func QueuedMsgEntry 

func QueuedMsgEntry(height int64, opMsg simulation.OperationMsg) OperationEntry

QueuedMsgEntry creates an operation entry for a given queued message.

func (OperationEntry) MustMarshal 

func (oe OperationEntry) MustMarshal() json.RawMessage

MustMarshal marshals the operation entry, panic on error.

type OperationQueue 

type OperationQueue map[int][]simulation.Operation

OperationQueue defines an object for a queue of operations

func NewOperationQueue 

func NewOperationQueue() OperationQueue

NewOperationQueue creates a new OperationQueue instance.

type Params 

type Params struct {
	// contains filtered or unexported fields
}

Params define the parameters necessary for running the simulations

func RandomParams 

func RandomParams(r *rand.Rand) Params

RandomParams returns random simulation parameters

func (Params) BlockSizeTransitionMatrix 

func (p Params) BlockSizeTransitionMatrix() simulation.TransitionMatrix

func (Params) EvidenceFraction 

func (p Params) EvidenceFraction() float64

func (Params) InitialLivenessWeightings 

func (p Params) InitialLivenessWeightings() []int

func (Params) LivenessTransitionMatrix 

func (p Params) LivenessTransitionMatrix() simulation.TransitionMatrix

func (Params) NumKeys 

func (p Params) NumKeys() int

func (Params) PastEvidenceFraction 

func (p Params) PastEvidenceFraction() float64

type RandomAccountFn 

type RandomAccountFn func(r *rand.Rand, n int) []legacysim.Account

RandomAccountFn returns a slice of n random simulation accounts

func WrapRandAccFnForResampling 

func WrapRandAccFnForResampling(randFn RandomAccountFn, blockList map[string]bool) RandomAccountFn

type StandardLogWriter 

type StandardLogWriter struct {
	OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
}

log writter

func (*StandardLogWriter) AddEntry 

func (lw *StandardLogWriter) AddEntry(opEntry OperationEntry)

add an entry to the log writter

func (*StandardLogWriter) PrintLogs 

func (lw *StandardLogWriter) PrintLogs()

PrintLogs - print the logs to a simulation file

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.