simulation

package
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 11, 2022 License: Apache-2.0, Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package simulation implements a full fledged Cosmos SDK application used for executing simulation test suites.

Simulation App

The SimApp type defines an application used for running extensive simulation testing suites. It contains all core modules, including governance, staking, slashing, and distribution.

Simulation is executed with various inputs including the number of blocks to simulate, the block size, whether the app should commit or not, the invariant checking period, and a seed which is used as a source of pseudo-randomness.

In addition to the various inputs, simulation runs mainly in three modes:

1. Completely random where the initial state, module parameters and simulation parameters are pseudo-randomly generated.

2. From a genesis file where the initial state and the module parameters are defined. This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested.

3. From a params file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated.

The simulation test suite also supports testing determinism and import/export functionality.

Randomness

Currently, simulation uses a single seed (integer) as a source for a PRNG by which all random operations are executed from. Any call to the PRNG changes all future operations as the internal state of the PRNG is modified. For example, if a new message type is created and needs to be simulated, the new introduced PRNG call will change all subsequent operations.

This may can often be problematic when testing fixes to simulation faults. One current solution to this is to use a params file as mentioned above. In the future the simulation suite is expected to support a series of PRNGs that can be used uniquely per module and simulation component so that they will not effect each others state execution outcome.

Usage

To execute a completely pseudo-random simulation: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-v -timeout 24h

To execute simulation from a genesis file: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-Genesis=/path/to/genesis.json \
 	-v -timeout 24h

To execute simulation from a simulation params file: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-Params=/path/to/params.json \
	-v -timeout 24h

To export the simulation params to a file at a given block height: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportParamsPath=/path/to/params.json \
	-ExportParamsHeight=50 \
	 -v -timeout 24h

To export the simulation app state (i.e genesis) to a file: 

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportStatePath=/path/to/genesis.json \
	 v -timeout 24h

Params

Params that are provided to simulation from a JSON file are used to used to set both module parameters and simulation parameters. See sim_test.go for the full set of parameters that can be provided.

Index

Constants

View Source 
const (
	BeginBlockEntryKind = "begin_block"
	EndBlockEntryKind   = "end_block"
	MsgEntryKind        = "msg"
	QueuedMsgEntryKind  = "queued_msg"
)

entry kinds for use within OperationEntry

Variables

This section is empty.

Functions

func DeriveRand 

func DeriveRand(r *rand.Rand) *rand.Rand

DeriveRand derives a new Rand deterministically from another random source. Unlike rand.New(rand.NewSource(seed)), the result is "more random" depending on the source and state of r.

NOTE: not crypto safe.

func GetMemberOfInitialState 

func GetMemberOfInitialState(r *rand.Rand, weights []int) int

GetMemberOfInitialState takes an initial array of weights, of size n. It returns a weighted random number in [0,n).

func RandIntBetween 

func RandIntBetween(r *rand.Rand, min, max int) int

RandIntBetween returns a random int between two numbers inclusively.

func RandPositiveInt 

func RandPositiveInt(r *rand.Rand, max sdk.Dec) (sdk.Int, error)

RandPositiveInt get a rand positive sdk.Int

func RandStringOfLength 

func RandStringOfLength(r *rand.Rand, n int) string

RandStringOfLength generates a random string of a particular length

func RandSubsetCoins 

func RandSubsetCoins(r *rand.Rand, coins sdk.Coins) sdk.Coins

returns random subset of the provided coins will return at least one coin unless coins argument is empty or malformed i.e. 0 amt in coins

func RandTimestamp 

func RandTimestamp(r *rand.Rand) time.Time

RandTimestamp generates a random timestamp

func RandomAmount 

func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int

RandomAmount generates a random amount Note: The range of RandomAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomDecAmount 

func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec

RandomDecAmount generates a random decimal amount Note: The range of RandomDecAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomFees 

func RandomFees(r *rand.Rand, ctx sdk.Context, spendableCoins sdk.Coins) (sdk.Coins, error)

RandomFees returns a random fee by selecting a random coin denomination and amount from the account's available balance. If the user doesn't have enough funds for paying fees, it returns empty coins.

func RandomRequestBeginBlock 

func RandomRequestBeginBlock(r *rand.Rand, params Params,
	validators mockValidators, pastTimes []time.Time,
	pastVoteInfos [][]abci.VoteInfo,
	event func(route, op, evResult string), header abci.Header) abci.RequestBeginBlock

RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction

Types

type Account 

type Account struct {
	PrivKey crypto.PrivKey
	PubKey  crypto.PubKey
	Address sdk.AccAddress
}

Account contains a privkey, pubkey, address tuple eventually more useful data can be placed in here. (e.g. number of coins)

func FindAccount 

func FindAccount(accs []Account, address sdk.Address) (Account, bool)

FindAccount iterates over all the simulation accounts to find the one that matches the given address

func RandomAcc 

func RandomAcc(r *rand.Rand, accs []Account) (Account, int)

RandomAcc picks and returns a random account from an array and returs its position in the array.

func RandomAccounts 

func RandomAccounts(r *rand.Rand, n int) []Account

RandomAccounts generates n random accounts

func (Account) Equals 

func (acc Account) Equals(acc2 Account) bool

Equals returns true if two accounts are equal

type AppParams 

type AppParams map[string]json.RawMessage

AppParams defines a flat JSON of key/values for all possible configurable simulation parameters. It might contain: operation weights, simulation parameters and flattened module state parameters (i.e not stored under it's respective module name).

func (AppParams) GetOrGenerate 

func (sp AppParams) GetOrGenerate(cdc *codec.Codec, key string, ptr interface{}, r *rand.Rand, ps ParamSimulator)

GetOrGenerate attempts to get a given parameter by key from the AppParams object. If it exists, it'll be decoded and returned. Otherwise, the provided ParamSimulator is used to generate a random value or default value (eg: in the case of operation weights where Rand is not used).

type AppStateFn 

type AppStateFn func(r *rand.Rand, accs []Account, config Config) (
	appState json.RawMessage, accounts []Account, chainId string, genesisTimestamp time.Time,
)

AppStateFn returns the app state json bytes and the genesis accounts

type Config 

type Config struct {
	GenesisFile string // custom simulation genesis file; cannot be used with params file
	ParamsFile  string // custom simulation params file which overrides any random params; cannot be used with genesis

	ExportParamsPath   string // custom file path to save the exported params JSON
	ExportParamsHeight int    //height to which export the randomly generated params
	ExportStatePath    string //custom file path to save the exported app state JSON
	ExportStatsPath    string // custom file path to save the exported simulation statistics JSON

	Seed               int64  // simulation random seed
	InitialBlockHeight int    // initial block to start the simulation
	NumBlocks          int    // number of new blocks to simulate from the initial block height
	BlockSize          int    // operations per block
	ChainID            string // chain-id used on the simulation

	Lean   bool // lean simulation log output
	Commit bool // have the simulation commit

	OnOperation   bool // run slow invariants every operation
	AllInvariants bool // print all failed invariants if a broken invariant is found
}

Config contains the necessary configuration flags for the simulator

type ContentSimulatorFn 

type ContentSimulatorFn func(r *rand.Rand, ctx sdk.Context, accs []Account) govtypes.Content

ContentSimulatorFn defines a function type alias for generating random proposal content.

type DummyLogWriter 

type DummyLogWriter struct{}

_____________________ dummy log writter

func (*DummyLogWriter) AddEntry 

func (lw *DummyLogWriter) AddEntry(_ OperationEntry)

do nothing

func (*DummyLogWriter) PrintLogs 

func (lw *DummyLogWriter) PrintLogs()

do nothing

type EventStats 

type EventStats map[string]map[string]map[string]int

EventStats defines an object that keeps a tally of each event that has occurred during a simulation.

func NewEventStats 

func NewEventStats() EventStats

NewEventStats creates a new empty EventStats object

func (EventStats) ExportJSON 

func (es EventStats) ExportJSON(path string)

ExportJSON saves the event stats as a JSON file on a given path

func (EventStats) Print 

func (es EventStats) Print(w io.Writer)

Print the event stats in JSON format.

func (EventStats) Tally 

func (es EventStats) Tally(route, op, evResult string)

Tally increases the count of a simulation event.

type FutureOperation 

type FutureOperation struct {
	BlockHeight int
	BlockTime   time.Time
	Op          Operation
}

FutureOperation is an operation which will be ran at the beginning of the provided BlockHeight. If both a BlockHeight and BlockTime are specified, it will use the BlockHeight. In the (likely) event that multiple operations are queued at the same block height, they will execute in a FIFO pattern.

type LogWriter 

type LogWriter interface {
	AddEntry(OperationEntry)
	PrintLogs()
}

log writter

func NewLogWriter 

func NewLogWriter(testingmode bool) LogWriter

LogWriter - return a dummy or standard log writer given the testingmode

type Operation 

type Operation func(r *rand.Rand, app *baseapp.BaseApp,
	ctx sdk.Context, accounts []Account, chainID string) (
	OperationMsg OperationMsg, futureOps []FutureOperation, err error)

Operation runs a state machine transition, and ensures the transition happened as expected. The operation could be running and testing a fuzzed transaction, or doing the same for a message.

For ease of debugging, an operation returns a descriptive message "action", which details what this fuzzed state machine transition actually did.

Operations can optionally provide a list of "FutureOperations" to run later These will be ran at the beginning of the corresponding block.

type OperationEntry 

type OperationEntry struct {
	EntryKind string          `json:"entry_kind" yaml:"entry_kind"`
	Height    int64           `json:"height" yaml:"height"`
	Order     int64           `json:"order" yaml:"order"`
	Operation json.RawMessage `json:"operation" yaml:"operation"`
}

OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)

func BeginBlockEntry 

func BeginBlockEntry(height int64) OperationEntry

BeginBlockEntry - operation entry for begin block

func EndBlockEntry 

func EndBlockEntry(height int64) OperationEntry

EndBlockEntry - operation entry for end block

func MsgEntry 

func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry

MsgEntry - operation entry for standard msg

func NewOperationEntry 

func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry

NewOperationEntry creates a new OperationEntry instance

func QueuedMsgEntry 

func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry

QueuedMsgEntry creates an operation entry for a given queued message.

func (OperationEntry) MustMarshal 

func (oe OperationEntry) MustMarshal() json.RawMessage

MustMarshal marshals the operation entry, panic on error.

type OperationMsg 

type OperationMsg struct {
	Route   string          `json:"route" yaml:"route"`     // msg route (i.e module name)
	Name    string          `json:"name" yaml:"name"`       // operation name (msg Type or "no-operation")
	Comment string          `json:"comment" yaml:"comment"` // additional comment
	OK      bool            `json:"ok" yaml:"ok"`           // success
	Msg     json.RawMessage `json:"msg" yaml:"msg"`         // JSON encoded msg
}

OperationMsg - structure for operation output

func NewOperationMsg 

func NewOperationMsg(msg sdk.Msg, ok bool, comment string) OperationMsg

NewOperationMsg - create a new operation message from sdk.Msg

func NewOperationMsgBasic 

func NewOperationMsgBasic(route, name, comment string, ok bool, msg []byte) OperationMsg

NewOperationMsgBasic creates a new operation message from raw input.

func NoOpMsg 

func NoOpMsg(route string) OperationMsg

NoOpMsg - create a no-operation message

func (OperationMsg) LogEvent 

func (om OperationMsg) LogEvent(eventLogger func(route, op, evResult string))

LogEvent adds an event for the events stats

func (OperationMsg) MustMarshal 

func (om OperationMsg) MustMarshal() json.RawMessage

MustMarshal Marshals the operation msg, panic on error

func (OperationMsg) String 

func (om OperationMsg) String() string

log entry text for this operation msg

type OperationQueue 

type OperationQueue map[int][]Operation

OperationQueue defines an object for a queue of operations

func NewOperationQueue 

func NewOperationQueue() OperationQueue

NewOperationQueue creates a new OperationQueue instance.

type ParamChange 

type ParamChange struct {
	Subspace string
	Key      string
	SimValue SimValFn
}

ParamChange defines the object used for simulating parameter change proposals

func NewSimParamChange 

func NewSimParamChange(subspace, key string, simVal SimValFn) ParamChange

NewSimParamChange creates a new ParamChange instance

func (ParamChange) ComposedKey 

func (spc ParamChange) ComposedKey() string

ComposedKey creates a new composed key for the param change proposal

type ParamSimulator 

type ParamSimulator func(r *rand.Rand)

ParamSimulator creates a parameter value from a source of random number

type Params 

type Params struct {
	PastEvidenceFraction      float64
	NumKeys                   int
	EvidenceFraction          float64
	InitialLivenessWeightings []int
	LivenessTransitionMatrix  TransitionMatrix
	BlockSizeTransitionMatrix TransitionMatrix
}

Params define the parameters necessary for running the simulations

func RandomParams 

func RandomParams(r *rand.Rand) Params

RandomParams returns random simulation parameters

func SimulateFromSeed 

func SimulateFromSeed(
	tb testing.TB, w io.Writer, app *baseapp.BaseApp,
	appStateFn AppStateFn, ops WeightedOperations,
	blackListedAccs map[string]bool, config Config,
) (stopEarly bool, exportedParams Params, err error)

SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided config.Seed. TODO: split this monster function up

type SimValFn 

type SimValFn func(r *rand.Rand) string

SimValFn function to generate the randomized parameter change value

type StandardLogWriter 

type StandardLogWriter struct {
	OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
}

log writter

func (*StandardLogWriter) AddEntry 

func (lw *StandardLogWriter) AddEntry(opEntry OperationEntry)

add an entry to the log writter

func (*StandardLogWriter) PrintLogs 

func (lw *StandardLogWriter) PrintLogs()

PrintLogs - print the logs to a simulation file

type TransitionMatrix 

type TransitionMatrix struct {
	// contains filtered or unexported fields
}

TransitionMatrix is _almost_ a left stochastic matrix. It is technically not one due to not normalizing the column values. In the future, if we want to find the steady state distribution, it will be quite easy to normalize these values to get a stochastic matrix. Floats aren't currently used as the default due to non-determinism across architectures

func CreateTransitionMatrix 

func CreateTransitionMatrix(weights [][]int) (TransitionMatrix, error)

CreateTransitionMatrix creates a transition matrix from the provided weights. TODO: Provide example usage

func (TransitionMatrix) NextState 

func (t TransitionMatrix) NextState(r *rand.Rand, i int) int

NextState returns the next state randomly chosen using r, and the weightings provided in the transition matrix.

type WeightedOperation 

type WeightedOperation struct {
	Weight int
	Op     Operation
}

WeightedOperation is an operation with associated weight. This is used to bias the selection operation within the simulator.

func NewWeightedOperation 

func NewWeightedOperation(weight int, op Operation) WeightedOperation

NewWeightedOperation creates a new WeightedOperation instance

type WeightedOperations 

type WeightedOperations []WeightedOperation

WeightedOperations is the group of all weighted operations to simulate.

type WeightedProposalContent 

type WeightedProposalContent struct {
	AppParamsKey       string             // key used to retrieve the value of the weight from the simulation application params
	DefaultWeight      int                // default weight
	ContentSimulatorFn ContentSimulatorFn // content simulator function
}

WeightedProposalContent defines a common struct for proposal contents defined by external modules (i.e outside gov)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.