simulation

package
v0.0.0-...-77283c5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 10, 2019 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package simulation implements a simulation framework for any state machine built on the SDK which utilizes auth.

It is primarily intended for fuzz testing the integration of modules. It will test that the provided operations are interoperable, and that the desired invariants hold. It can additionally be used to detect what the performance benchmarks in the system are, by using benchmarking mode and cpu / mem profiling. If it detects a failure, it provides the entire log of what was ran.

The simulator takes as input: a random seed, the set of operations to run, the invariants to test, and additional parameters to configure how long to run, and misc. parameters that affect simulation speed.

It is intended that every module provides a list of Operations which will randomly create and run a message / tx in a manner that is interesting to fuzz, and verify that the state transition was executed as exported. Each module should additionally provide methods to assert that the desired invariants hold.

Then to perform a randomized simulation, select the set of desired operations, the weightings for each, the invariants you want to test, and how long to run it for. Then run simulation.Simulate! The simulator will handle things like ensuring that validators periodically double signing, or go offline.

Index

Constants

View Source 
const (
	BeginBlockEntryKind = "begin_block"
	EndBlockEntryKind   = "end_block"
	MsgEntryKind        = "msg"
	QueuedMsgEntryKind  = "queued_msg"
)

entry kinds for use within OperationEntry

View Source 
const (

	// Simulation parameter constants
	SendEnabled              = "send_enabled"
	MaxMemoChars             = "max_memo_characters"
	TxSigLimit               = "tx_sig_limit"
	TxSizeCostPerByte        = "tx_size_cost_per_byte"
	SigVerifyCostED25519     = "sig_verify_cost_ed25519"
	SigVerifyCostSECP256K1   = "sig_verify_cost_secp256k1"
	DepositParamsMinDeposit  = "deposit_params_min_deposit"
	VotingParamsVotingPeriod = "voting_params_voting_period"
	TallyParamsQuorum        = "tally_params_quorum"
	TallyParamsThreshold     = "tally_params_threshold"
	TallyParamsVeto          = "tally_params_veto"
	UnbondingTime            = "unbonding_time"
	MaxValidators            = "max_validators"
	SignedBlocksWindow       = "signed_blocks_window"
	MinSignedPerWindow       = "min_signed_per_window"
	DowntimeJailDuration     = "downtime_jail_duration"
	SlashFractionDoubleSign  = "slash_fraction_double_sign"
	SlashFractionDowntime    = "slash_fraction_downtime"
	InflationRateChange      = "inflation_rate_change"
	Inflation                = "inflation"
	InflationMax             = "inflation_max"
	InflationMin             = "inflation_min"
	GoalBonded               = "goal_bonded"
	CommunityTax             = "community_tax"
	BaseProposerReward       = "base_proposer_reward"
	BonusProposerReward      = "bonus_proposer_reward"
)

nolint

Variables

View Source 
var (

	// ModuleParamSimulator defines module parameter value simulators. All
	// values simulated should be within valid acceptable range for the given
	// parameter.
	ModuleParamSimulator = map[string]func(r *rand.Rand) interface{}{
		SendEnabled: func(r *rand.Rand) interface{} {
			return r.Int63n(2) == 0
		},
		MaxMemoChars: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 100, 200))
		},
		TxSigLimit: func(r *rand.Rand) interface{} {
			return uint64(r.Intn(7) + 1)
		},
		TxSizeCostPerByte: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 5, 15))
		},
		SigVerifyCostED25519: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 500, 1000))
		},
		SigVerifyCostSECP256K1: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 500, 1000))
		},
		DepositParamsMinDeposit: func(r *rand.Rand) interface{} {
			return sdk.Coins{sdk.NewInt64Coin(sdk.DefaultBondDenom, int64(RandIntBetween(r, 1, 1e3)))}
		},
		VotingParamsVotingPeriod: func(r *rand.Rand) interface{} {
			return time.Duration(RandIntBetween(r, 1, 2*60*60*24*2)) * time.Second
		},
		TallyParamsQuorum: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(RandIntBetween(r, 334, 500)), 3)
		},
		TallyParamsThreshold: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(RandIntBetween(r, 450, 550)), 3)
		},
		TallyParamsVeto: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(RandIntBetween(r, 250, 334)), 3)
		},
		UnbondingTime: func(r *rand.Rand) interface{} {
			return time.Duration(RandIntBetween(r, 60, 60*60*24*3*2)) * time.Second
		},
		MaxValidators: func(r *rand.Rand) interface{} {
			return uint16(r.Intn(250) + 1)
		},
		SignedBlocksWindow: func(r *rand.Rand) interface{} {
			return int64(RandIntBetween(r, 10, 1000))
		},
		MinSignedPerWindow: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(r.Intn(10)), 1)
		},
		DowntimeJailDuration: func(r *rand.Rand) interface{} {
			return time.Duration(RandIntBetween(r, 60, 60*60*24)) * time.Second
		},
		SlashFractionDoubleSign: func(r *rand.Rand) interface{} {
			return sdk.NewDec(1).Quo(sdk.NewDec(int64(r.Intn(50) + 1)))
		},
		SlashFractionDowntime: func(r *rand.Rand) interface{} {
			return sdk.NewDec(1).Quo(sdk.NewDec(int64(r.Intn(200) + 1)))
		},
		InflationRateChange: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(r.Intn(99)), 2)
		},
		Inflation: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(r.Intn(99)), 2)
		},
		InflationMax: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(20, 2)
		},
		InflationMin: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(7, 2)
		},
		GoalBonded: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(67, 2)
		},
		CommunityTax: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(1, 2).Add(sdk.NewDecWithPrec(int64(r.Intn(30)), 2))
		},
		BaseProposerReward: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(1, 2).Add(sdk.NewDecWithPrec(int64(r.Intn(30)), 2))
		},
		BonusProposerReward: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(1, 2).Add(sdk.NewDecWithPrec(int64(r.Intn(30)), 2))
		},
	}
)

TODO explain transitional matrix usage

Functions

func DeriveRand 

func DeriveRand(r *rand.Rand) *rand.Rand

Derive a new rand deterministically from a rand. Unlike rand.New(rand.NewSource(seed)), the result is "more random" depending on the source and state of r. NOTE: not crypto safe.

func GetMemberOfInitialState 

func GetMemberOfInitialState(r *rand.Rand, weights []int) int

GetMemberOfInitialState takes an initial array of weights, of size n. It returns a weighted random number in [0,n).

func PeriodicInvariants 

func PeriodicInvariants(invariants []sdk.Invariant, period, offset int) []sdk.Invariant

PeriodicInvariants returns an array of wrapped Invariants. Where each invariant function is only executed periodically defined by period and offset.

func RandIntBetween 

func RandIntBetween(r *rand.Rand, min, max int) int

RandIntBetween returns a random int between two numbers inclusively.

func RandPositiveInt 

func RandPositiveInt(r *rand.Rand, max sdk.Int) (sdk.Int, error)

get a rand positive sdk.Int

func RandStringOfLength 

func RandStringOfLength(r *rand.Rand, n int) string

shamelessly copied from https://stackoverflow.com/questions/22892120/how-to-generate-a-random-string-of-a-fixed-length-in-golang#31832326 Generate a random string of a particular length

func RandTimestamp 

func RandTimestamp(r *rand.Rand) time.Time

RandTimestamp generates a random timestamp

func RandomAmount 

func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int

Generate a random amount Note: The range of RandomAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomDecAmount 

func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec

RandomDecAmount generates a random decimal amount Note: The range of RandomDecAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomRequestBeginBlock 

func RandomRequestBeginBlock(r *rand.Rand, params Params,
	validators mockValidators, pastTimes []time.Time,
	pastVoteInfos [][]abci.VoteInfo,
	event func(route, op, evResult string), header abci.Header) abci.RequestBeginBlock

RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction

Types

type Account 

type Account struct {
	PrivKey crypto.PrivKey
	PubKey  crypto.PubKey
	Address sdk.AccAddress
}

Account contains a privkey, pubkey, address tuple eventually more useful data can be placed in here. (e.g. number of coins)

func RandomAcc 

func RandomAcc(r *rand.Rand, accs []Account) Account

RandomAcc pick a random account from an array

func RandomAccounts 

func RandomAccounts(r *rand.Rand, n int) []Account

RandomAccounts generates n random accounts

func (Account) Equals 

func (acc Account) Equals(acc2 Account) bool

are two accounts equal

type AppParams 

type AppParams map[string]json.RawMessage

TODO add description

func (AppParams) GetOrGenerate 

func (sp AppParams) GetOrGenerate(cdc *codec.Codec, key string, ptr interface{}, r *rand.Rand, ps ParamSimulator)

GetOrGenerate attempts to get a given parameter by key from the AppParams object. If it exists, it'll be decoded and returned. Otherwise, the provided ParamSimulator is used to generate a random value.

type AppStateFn 

type AppStateFn func(
	r *rand.Rand, accs []Account,
) (appState json.RawMessage, accounts []Account, chainId string, genesisTimestamp time.Time)

AppStateFn returns the app state json bytes, the genesis accounts, and the chain identifier

type DummyLogWriter 

type DummyLogWriter struct{}

_____________________ dummy log writter

func (*DummyLogWriter) AddEntry 

func (lw *DummyLogWriter) AddEntry(_ OperationEntry)

do nothing

func (*DummyLogWriter) PrintLogs 

func (lw *DummyLogWriter) PrintLogs()

do nothing

type EventStats 

type EventStats map[string]map[string]map[string]int

EventStats defines an object that keeps a tally of each event that has occurred during a simulation.

func NewEventStats 

func NewEventStats() EventStats

NewEventStats creates a new empty EventStats object

func (EventStats) ExportJSON 

func (es EventStats) ExportJSON(path string)

ExportJSON saves the event stats as a JSON file on a given path

func (EventStats) Print 

func (es EventStats) Print(w io.Writer)

Print the event stats in JSON format.

func (EventStats) Tally 

func (es EventStats) Tally(route, op, evResult string)

Tally increases the count of a simulation event.

type FutureOperation 

type FutureOperation struct {
	BlockHeight int
	BlockTime   time.Time
	Op          Operation
}

FutureOperation is an operation which will be ran at the beginning of the provided BlockHeight. If both a BlockHeight and BlockTime are specified, it will use the BlockHeight. In the (likely) event that multiple operations are queued at the same block height, they will execute in a FIFO pattern.

type LogWriter 

type LogWriter interface {
	AddEntry(OperationEntry)
	PrintLogs()
}

log writter

func NewLogWriter 

func NewLogWriter(testingmode bool) LogWriter

LogWriter - return a dummy or standard log writer given the testingmode

type Operation 

type Operation func(r *rand.Rand, app *baseapp.BaseApp,
	ctx sdk.Context, accounts []Account) (
	OperationMsg OperationMsg, futureOps []FutureOperation, err error)

Operation runs a state machine transition, and ensures the transition happened as exported. The operation could be running and testing a fuzzed transaction, or doing the same for a message.

For ease of debugging, an operation returns a descriptive message "action", which details what this fuzzed state machine transition actually did.

Operations can optionally provide a list of "FutureOperations" to run later These will be ran at the beginning of the corresponding block.

type OperationEntry 

type OperationEntry struct {
	EntryKind string          `json:"entry_kind" yaml:"entry_kind"`
	Height    int64           `json:"height" yaml:"height"`
	Order     int64           `json:"order" yaml:"order"`
	Operation json.RawMessage `json:"operation" yaml:"operation"`
}

OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)

func BeginBlockEntry 

func BeginBlockEntry(height int64) OperationEntry

BeginBlockEntry - operation entry for begin block

func EndBlockEntry 

func EndBlockEntry(height int64) OperationEntry

EndBlockEntry - operation entry for end block

func MsgEntry 

func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry

MsgEntry - operation entry for standard msg

func NewOperationEntry 

func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry

NewOperationEntry creates a new OperationEntry instance

func QueuedMsgEntry 

func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry

QueuedMsgEntry creates an operation entry for a given queued message.

func (OperationEntry) MustMarshal 

func (oe OperationEntry) MustMarshal() json.RawMessage

MustMarshal marshals the operation entry, panic on error.

type OperationMsg 

type OperationMsg struct {
	Route   string          `json:"route" yaml:"route"`
	Name    string          `json:"name" yaml:"name"`
	Comment string          `json:"comment" yaml:"comment"`
	OK      bool            `json:"ok" yaml:"ok"`
	Msg     json.RawMessage `json:"msg" yaml:"msg"`
}

OperationMsg - structure for operation output

func NewOperationMsg 

func NewOperationMsg(msg sdk.Msg, ok bool, comment string) OperationMsg

NewOperationMsg - create a new operation message from sdk.Msg

func NewOperationMsgBasic 

func NewOperationMsgBasic(route, name, comment string, ok bool, msg []byte) OperationMsg

NewOperationMsgBasic creates a new operation message from raw input.

func NoOpMsg 

func NoOpMsg(route string) OperationMsg

NoOpMsg - create a no-operation message

func (OperationMsg) LogEvent 

func (om OperationMsg) LogEvent(eventLogger func(route, op, evResult string))

LogEvent adds an event for the events stats

func (OperationMsg) MustMarshal 

func (om OperationMsg) MustMarshal() json.RawMessage

MustMarshal Marshals the operation msg, panic on error

func (OperationMsg) String 

func (om OperationMsg) String() string

log entry text for this operation msg

type OperationQueue 

type OperationQueue map[int][]Operation

OperationQueue defines an object for a queue of operations

func NewOperationQueue 

func NewOperationQueue() OperationQueue

NewOperationQueue creates a new OperationQueue instance.

type ParamSimulator 

type ParamSimulator func(r *rand.Rand)

TODO add description

type Params 

type Params struct {
	PastEvidenceFraction      float64
	NumKeys                   int
	EvidenceFraction          float64
	InitialLivenessWeightings []int
	LivenessTransitionMatrix  TransitionMatrix
	BlockSizeTransitionMatrix TransitionMatrix
}

Simulation parameters

func RandomParams 

func RandomParams(r *rand.Rand) Params

Return random simulation parameters

func SimulateFromSeed 

func SimulateFromSeed(
	tb testing.TB, w io.Writer, app *baseapp.BaseApp,
	appStateFn AppStateFn, seed int64,
	ops WeightedOperations, invariants sdk.Invariants,
	initialHeight, numBlocks, exportParamsHeight, blockSize int,
	exportStatsPath string,
	exportParams, commit, lean, onOperation, allInvariants bool,
	blackListedAccs map[string]bool,
) (stopEarly bool, exportedParams Params, err error)

SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided seed. TODO: split this monster function up

type StandardLogWriter 

type StandardLogWriter struct {
	OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
}

log writter

func (*StandardLogWriter) AddEntry 

func (lw *StandardLogWriter) AddEntry(opEntry OperationEntry)

add an entry to the log writter

func (*StandardLogWriter) PrintLogs 

func (lw *StandardLogWriter) PrintLogs()

PrintLogs - print the logs to a simulation file

type TransitionMatrix 

type TransitionMatrix struct {
	// contains filtered or unexported fields
}

TransitionMatrix is _almost_ a left stochastic matrix. It is technically not one due to not normalizing the column values. In the future, if we want to find the steady state distribution, it will be quite easy to normalize these values to get a stochastic matrix. Floats aren't currently used as the default due to non-determinism across architectures

func CreateTransitionMatrix 

func CreateTransitionMatrix(weights [][]int) (TransitionMatrix, error)

CreateTransitionMatrix creates a transition matrix from the provided weights. TODO: Provide example usage

func (TransitionMatrix) NextState 

func (t TransitionMatrix) NextState(r *rand.Rand, i int) int

NextState returns the next state randomly chosen using r, and the weightings provided in the transition matrix.

type WeightedOperation 

type WeightedOperation struct {
	Weight int
	Op     Operation
}

WeightedOperation is an operation with associated weight. This is used to bias the selection operation within the simulator.

type WeightedOperations 

type WeightedOperations []WeightedOperation

WeightedOperations is the group of all weighted operations to simulate.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.