README ¶
devp2p Simulations
The p2p/simulations
package implements a simulation framework which supports
creating a collection of devp2p nodes, connecting them together to form a
simulation network, performing simulation actions in that network and then
extracting useful information.
Nodes
Each node in a simulation network runs multiple services by wrapping a collection
of objects which implement the node.Service
interface meaning they:
- can be started and stopped
- run p2p protocols
- expose RPC APIs
This means that any object which implements the node.Service
interface can be
used to run a node in the simulation.
Services
Before running a simulation, a set of service initializers must be registered which can then be used to run nodes in the network.
A service initializer is a function with the following signature:
func(ctx *adapters.ServiceContext) (node.Service, error)
These initializers should be registered by calling the adapters.RegisterServices
function in an init()
hook:
func init() {
adapters.RegisterServices(adapters.Services{
"service1": initService1,
"service2": initService2,
})
}
Node Adapters
The simulation framework includes multiple "node adapters" which are responsible for creating an environment in which a node runs.
SimAdapter
The SimAdapter
runs nodes in-memory, connecting them using an in-memory,
synchronous net.Pipe
and connecting to their RPC server using an in-memory
rpc.Client
.
ExecAdapter
The ExecAdapter
runs nodes as child processes of the running simulation.
It does this by executing the binary which is running the simulation but
setting argv[0]
(i.e. the program name) to p2p-node
which is then
detected by an init hook in the child process which runs the node.Service
using the devp2p node stack rather than executing main()
.
The nodes listen for devp2p connections and WebSocket RPC clients on random localhost ports.
DockerAdapter
The DockerAdapter
is similar to the ExecAdapter
but executes docker run
to run the node in a Docker container using a Docker image containing the
simulation binary at /bin/p2p-node
.
The Docker image is built using docker build
when the adapter is initialised,
meaning no prior setup is necessary other than having a working Docker client.
Each node listens on the external IP of the container and the default p2p and
RPC ports (30303
and 8546
respectively).
Network
A simulation network is created with an ID and default service (which is used if a node is created without an explicit service), exposes methods for creating, starting, stopping, connecting and disconnecting nodes, and emits events when certain actions occur.
Events
A simulation network emits the following events:
- node event - when nodes are created / started / stopped
- connection event - when nodes are connected / disconnected
- message event - when a protocol message is sent between two nodes
The events have a "control" flag which when set indicates that the event is the outcome of a controlled simulation action (e.g. creating a node or explicitly connecting two nodes together).
This is in contrast to a non-control event, otherwise called a "live" event, which is the outcome of something happening in the network as a result of a control event (e.g. a node actually started up or a connection was actually established between two nodes).
Live events are detected by the simulation network by subscribing to node peer events via RPC when the nodes start up.
Testing Framework
The Simulation
type can be used in tests to perform actions in a simulation
network and then wait for expectations to be met.
With a running simulation network, the Simulation.Run
method can be called
with a Step
which has the following fields:
-
Action
- a function which performs some action in the network -
Expect
- an expectation function which returns whether or not a given node meets the expectation -
Trigger
- a channel which receives node IDs which then trigger a check of the expectation function to be performed against that node
As a concrete example, consider a simulated network of Ethereum nodes. An
Action
could be the sending of a transaction, Expect
it being included in
a block, and Trigger
a check for every block that is mined.
On return, the Simulation.Run
method returns a StepResult
which can be used
to determine if all nodes met the expectation, how long it took them to meet
the expectation and what network events were emitted during the step run.
HTTP API
The simulation framework includes a HTTP API which can be used to control the simulation.
The API is initialised with a particular node adapter and has the following endpoints:
GET / Get network information
POST /start Start all nodes in the network
POST /stop Stop all nodes in the network
GET /events Stream network events
GET /snapshot Take a network snapshot
POST /snapshot Load a network snapshot
POST /nodes Create a node
GET /nodes Get all nodes in the network
GET /nodes/:nodeid Get node information
POST /nodes/:nodeid/start Start a node
POST /nodes/:nodeid/stop Stop a node
POST /nodes/:nodeid/conn/:peerid Connect two nodes
DELETE /nodes/:nodeid/conn/:peerid Disconnect two nodes
GET /nodes/:nodeid/rpc Make RPC requests to a node via WebSocket
For convenience, nodeid
in the URL can be the name of a node rather than its
ID.
Command line client
p2psim
is a command line client for the HTTP API, located in
cmd/p2psim
.
It provides the following commands:
p2psim show
p2psim events [--current] [--filter=FILTER]
p2psim snapshot
p2psim load
p2psim node create [--name=NAME] [--services=SERVICES] [--key=KEY]
p2psim node list
p2psim node show <node>
p2psim node start <node>
p2psim node stop <node>
p2psim node connect <node> <peer>
p2psim node disconnect <node> <peer>
p2psim node rpc <node> <method> [<args>] [--subscribe]
Example
Documentation ¶
Overview ¶
Package simulations simulates p2p networks. A mocker simulates starting and stopping real nodes in a network.
Index ¶
- Variables
- func ConnLabel(source, target discover.NodeID) string
- func GetMockerList() []string
- func LookupMocker(mockerType string) func(net *Network, quit chan struct{}, nodeCount int)
- type Client
- func (c *Client) ConnectNode(nodeID, peerID string) error
- func (c *Client) CreateNode(config *adapters.NodeConfig) (*p2p.NodeInfo, error)
- func (c *Client) CreateSnapshot() (*Snapshot, error)
- func (c *Client) Delete(path string) error
- func (c *Client) DisconnectNode(nodeID, peerID string) error
- func (c *Client) Get(path string, out interface{}) error
- func (c *Client) GetNetwork() (*Network, error)
- func (c *Client) GetNode(nodeID string) (*p2p.NodeInfo, error)
- func (c *Client) GetNodes() ([]*p2p.NodeInfo, error)
- func (c *Client) LoadSnapshot(snap *Snapshot) error
- func (c *Client) Post(path string, in, out interface{}) error
- func (c *Client) RPCClient(ctx context.Context, nodeID string) (*rpc.Client, error)
- func (c *Client) Send(method, path string, in, out interface{}) error
- func (c *Client) StartNetwork() error
- func (c *Client) StartNode(nodeID string) error
- func (c *Client) StopNetwork() error
- func (c *Client) StopNode(nodeID string) error
- func (c *Client) SubscribeNetwork(events chan *Event, opts SubscribeOpts) (event.Subscription, error)
- type Conn
- type Event
- type EventType
- type Expectation
- type Msg
- type MsgFilter
- type MsgFilters
- type Network
- func (net *Network) Config() *NetworkConfig
- func (net *Network) Connect(oneID, otherID discover.NodeID) error
- func (net *Network) DidConnect(one, other discover.NodeID) error
- func (net *Network) DidDisconnect(one, other discover.NodeID) error
- func (net *Network) DidReceive(sender, receiver discover.NodeID, proto string, code uint64) error
- func (net *Network) DidSend(sender, receiver discover.NodeID, proto string, code uint64) error
- func (net *Network) Disconnect(oneID, otherID discover.NodeID) error
- func (net *Network) Events() *event.Feed
- func (net *Network) GetConn(oneID, otherID discover.NodeID) *Conn
- func (net *Network) GetNode(id discover.NodeID) *Node
- func (net *Network) GetNodeByName(name string) *Node
- func (net *Network) GetNodes() (nodes []*Node)
- func (net *Network) GetOrCreateConn(oneID, otherID discover.NodeID) (*Conn, error)
- func (net *Network) InitConn(oneID, otherID discover.NodeID) (*Conn, error)
- func (net *Network) Load(snap *Snapshot) error
- func (net *Network) NewNodeWithConfig(conf *adapters.NodeConfig) (*Node, error)
- func (net *Network) Reset()
- func (net *Network) Shutdown()
- func (net *Network) Snapshot() (*Snapshot, error)
- func (net *Network) Start(id discover.NodeID) error
- func (net *Network) StartAll() error
- func (net *Network) Stop(id discover.NodeID) error
- func (net *Network) StopAll() error
- func (net *Network) Subscribe(events chan *Event)
- type NetworkConfig
- type Node
- type NodeSnapshot
- type Server
- func (s *Server) ConnectNode(w http.ResponseWriter, req *http.Request)
- func (s *Server) CreateNode(w http.ResponseWriter, req *http.Request)
- func (s *Server) CreateSnapshot(w http.ResponseWriter, req *http.Request)
- func (s *Server) DELETE(path string, handle http.HandlerFunc)
- func (s *Server) DisconnectNode(w http.ResponseWriter, req *http.Request)
- func (s *Server) GET(path string, handle http.HandlerFunc)
- func (s *Server) GetMockers(w http.ResponseWriter, req *http.Request)
- func (s *Server) GetNetwork(w http.ResponseWriter, req *http.Request)
- func (s *Server) GetNode(w http.ResponseWriter, req *http.Request)
- func (s *Server) GetNodes(w http.ResponseWriter, req *http.Request)
- func (s *Server) JSON(w http.ResponseWriter, status int, data interface{})
- func (s *Server) LoadSnapshot(w http.ResponseWriter, req *http.Request)
- func (s *Server) NodeRPC(w http.ResponseWriter, req *http.Request)
- func (s *Server) OPTIONS(path string, handle http.HandlerFunc)
- func (s *Server) Options(w http.ResponseWriter, req *http.Request)
- func (s *Server) POST(path string, handle http.HandlerFunc)
- func (s *Server) ResetNetwork(w http.ResponseWriter, req *http.Request)
- func (s *Server) ServeHTTP(w http.ResponseWriter, req *http.Request)
- func (s *Server) StartMocker(w http.ResponseWriter, req *http.Request)
- func (s *Server) StartNetwork(w http.ResponseWriter, req *http.Request)
- func (s *Server) StartNode(w http.ResponseWriter, req *http.Request)
- func (s *Server) StopMocker(w http.ResponseWriter, req *http.Request)
- func (s *Server) StopNetwork(w http.ResponseWriter, req *http.Request)
- func (s *Server) StopNode(w http.ResponseWriter, req *http.Request)
- func (s *Server) StreamNetworkEvents(w http.ResponseWriter, req *http.Request)
- type Simulation
- type Snapshot
- type Step
- type StepResult
- type SubscribeOpts
Constants ¶
This section is empty.
Variables ¶
var DefaultClient = NewClient("http://localhost:8888")
DefaultClient is the default simulation API client which expects the API to be running at http://localhost:8888
var DialBanTimeout = 200 * time.Millisecond
Functions ¶
func ConnLabel ¶
ConnLabel generates a deterministic string which represents a connection between two nodes, used to compare if two connections are between the same nodes
func GetMockerList ¶
func GetMockerList() []string
Get a list of mockers (keys of the map) Useful for frontend to build available mocker selection
func LookupMocker ¶
Lookup a mocker by its name, returns the mockerFn
Types ¶
type Client ¶
type Client struct { URL string // contains filtered or unexported fields }
Client is a client for the simulation HTTP API which supports creating and managing simulation networks
func (*Client) ConnectNode ¶
ConnectNode connects a node to a peer node
func (*Client) CreateNode ¶
CreateNode creates a node in the network using the given configuration
func (*Client) CreateSnapshot ¶
CreateSnapshot creates a network snapshot
func (*Client) DisconnectNode ¶
DisconnectNode disconnects a node from a peer node
func (*Client) Get ¶
Get performs a HTTP GET request decoding the resulting JSON response into "out"
func (*Client) GetNetwork ¶
GetNetwork returns details of the network
func (*Client) LoadSnapshot ¶
LoadSnapshot loads a snapshot into the network
func (*Client) Post ¶
Post performs a HTTP POST request sending "in" as the JSON body and decoding the resulting JSON response into "out"
func (*Client) Send ¶
Send performs a HTTP request, sending "in" as the JSON request body and decoding the JSON response into "out"
func (*Client) StartNetwork ¶
StartNetwork starts all existing nodes in the simulation network
func (*Client) StopNetwork ¶
StopNetwork stops all existing nodes in a simulation network
func (*Client) SubscribeNetwork ¶
func (c *Client) SubscribeNetwork(events chan *Event, opts SubscribeOpts) (event.Subscription, error)
SubscribeNetwork subscribes to network events which are sent from the server as a server-sent-events stream, optionally receiving events for existing nodes and connections and filtering message events
type Conn ¶
type Conn struct { // One is the node which initiated the connection One discover.NodeID `json:"one"` // Other is the node which the connection was made to Other discover.NodeID `json:"other"` // Up tracks whether or not the connection is active Up bool `json:"up"` // contains filtered or unexported fields }
Conn represents a connection between two nodes in the network
type Event ¶
type Event struct { // Type is the type of the event Type EventType `json:"type"` // Time is the time the event happened Time time.Time `json:"time"` // Control indicates whether the event is the result of a controlled // action in the network Control bool `json:"control"` // Node is set if the type is EventTypeNode Node *Node `json:"node,omitempty"` // Conn is set if the type is EventTypeConn Conn *Conn `json:"conn,omitempty"` // Msg is set if the type is EventTypeMsg Msg *Msg `json:"msg,omitempty"` }
Event is an event emitted by a simulation network
func ControlEvent ¶
func ControlEvent(v interface{}) *Event
ControlEvent creates a new control event
type EventType ¶
type EventType string
EventType is the type of event emitted by a simulation network
const ( // EventTypeNode is the type of event emitted when a node is either // created, started or stopped EventTypeNode EventType = "node" // EventTypeConn is the type of event emitted when a connection is // is either established or dropped between two nodes EventTypeConn EventType = "conn" // EventTypeMsg is the type of event emitted when a p2p message it // sent between two nodes EventTypeMsg EventType = "msg" )
type Expectation ¶
type Msg ¶
type Msg struct { One discover.NodeID `json:"one"` Other discover.NodeID `json:"other"` Protocol string `json:"protocol"` Code uint64 `json:"code"` Received bool `json:"received"` }
Msg represents a p2p message sent between two nodes in the network
type MsgFilter ¶
type MsgFilter struct { // Proto is matched against a message's protocol Proto string // Code is matched against a message's code, with -1 matching all codes Code int64 }
MsgFilter is used to filter message events based on protocol and message code
type MsgFilters ¶
type MsgFilters map[MsgFilter]struct{}
MsgFilters is a collection of filters which are used to filter message events
func NewMsgFilters ¶
func NewMsgFilters(filterParam string) (MsgFilters, error)
NewMsgFilters constructs a collection of message filters from a URL query parameter.
The parameter is expected to be a dash-separated list of individual filters, each having the format '<proto>:<codes>', where <proto> is the name of a protocol and <codes> is a comma-separated list of message codes.
A message code of '*' or '-1' is considered a wildcard and matches any code.
func (MsgFilters) Match ¶
func (m MsgFilters) Match(msg *Msg) bool
Match checks if the given message matches any of the filters
type Network ¶
type Network struct { NetworkConfig Nodes []*Node `json:"nodes"` Conns []*Conn `json:"conns"` // contains filtered or unexported fields }
Network models a p2p simulation network which consists of a collection of simulated nodes and the connections which exist between them.
The Network has a single NodeAdapter which is responsible for actually starting nodes and connecting them together.
The Network emits events when nodes are started and stopped, when they are connected and disconnected, and also when messages are sent between nodes.
func NewNetwork ¶
func NewNetwork(nodeAdapter adapters.NodeAdapter, conf *NetworkConfig) *Network
NewNetwork returns a Network which uses the given NodeAdapter and NetworkConfig
func (*Network) Config ¶
func (net *Network) Config() *NetworkConfig
Config returns the network configuration
func (*Network) Connect ¶
Connect connects two nodes together by calling the "admin_addPeer" RPC method on the "one" node so that it connects to the "other" node
func (*Network) DidConnect ¶
DidConnect tracks the fact that the "one" node connected to the "other" node
func (*Network) DidDisconnect ¶
DidDisconnect tracks the fact that the "one" node disconnected from the "other" node
func (*Network) DidReceive ¶
DidReceive tracks the fact that "receiver" received a message from "sender"
func (*Network) Disconnect ¶
Disconnect disconnects two nodes by calling the "admin_removePeer" RPC method on the "one" node so that it disconnects from the "other" node
func (*Network) GetConn ¶
GetConn returns the connection which exists between "one" and "other" regardless of which node initiated the connection
func (*Network) GetNode ¶
GetNode gets the node with the given ID, returning nil if the node does not exist
func (*Network) GetNodeByName ¶
GetNode gets the node with the given name, returning nil if the node does not exist
func (*Network) GetOrCreateConn ¶
GetOrCreateConn is like GetConn but creates the connection if it doesn't already exist
func (*Network) InitConn ¶
InitConn(one, other) retrieves the connectiton model for the connection between peers one and other, or creates a new one if it does not exist the order of nodes does not matter, i.e., Conn(i,j) == Conn(j, i) it checks if the connection is already up, and if the nodes are running NOTE: it also checks whether there has been recent attempt to connect the peers this is cheating as the simulation is used as an oracle and know about remote peers attempt to connect to a node which will then not initiate the connection
func (*Network) NewNodeWithConfig ¶
func (net *Network) NewNodeWithConfig(conf *adapters.NodeConfig) (*Node, error)
NewNodeWithConfig adds a new node to the network with the given config, returning an error if a node with the same ID or name already exists
func (*Network) Reset ¶
func (net *Network) Reset()
Reset resets all network properties: emtpies the nodes and the connection list
func (*Network) Shutdown ¶
func (net *Network) Shutdown()
Shutdown stops all nodes in the network and closes the quit channel
type NetworkConfig ¶
type NetworkConfig struct { ID string `json:"id"` DefaultService string `json:"default_service,omitempty"` }
NetworkConfig defines configuration options for starting a Network
type Node ¶
type Node struct { adapters.Node `json:"-"` // Config if the config used to created the node Config *adapters.NodeConfig `json:"config"` // Up tracks whether or not the node is running Up bool `json:"up"` }
Node is a wrapper around adapters.Node which is used to track the status of a node in the network
func (*Node) MarshalJSON ¶
MarshalJSON implements the json.Marshaler interface so that the encoded JSON includes the NodeInfo
type NodeSnapshot ¶
type NodeSnapshot struct { Node Node `json:"node,omitempty"` // Snapshots is arbitrary data gathered from calling node.Snapshots() Snapshots map[string][]byte `json:"snapshots,omitempty"` }
NodeSnapshot represents the state of a node in the network
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is an HTTP server providing an API to manage a simulation network
func (*Server) ConnectNode ¶
func (s *Server) ConnectNode(w http.ResponseWriter, req *http.Request)
ConnectNode connects a node to a peer node
func (*Server) CreateNode ¶
func (s *Server) CreateNode(w http.ResponseWriter, req *http.Request)
CreateNode creates a node in the network using the given configuration
func (*Server) CreateSnapshot ¶
func (s *Server) CreateSnapshot(w http.ResponseWriter, req *http.Request)
CreateSnapshot creates a network snapshot
func (*Server) DELETE ¶
func (s *Server) DELETE(path string, handle http.HandlerFunc)
DELETE registers a handler for DELETE requests to a particular path
func (*Server) DisconnectNode ¶
func (s *Server) DisconnectNode(w http.ResponseWriter, req *http.Request)
DisconnectNode disconnects a node from a peer node
func (*Server) GET ¶
func (s *Server) GET(path string, handle http.HandlerFunc)
GET registers a handler for GET requests to a particular path
func (*Server) GetMockers ¶
func (s *Server) GetMockers(w http.ResponseWriter, req *http.Request)
GetMockerList returns a list of available mockers
func (*Server) GetNetwork ¶
func (s *Server) GetNetwork(w http.ResponseWriter, req *http.Request)
GetNetwork returns details of the network
func (*Server) GetNode ¶
func (s *Server) GetNode(w http.ResponseWriter, req *http.Request)
GetNode returns details of a node
func (*Server) GetNodes ¶
func (s *Server) GetNodes(w http.ResponseWriter, req *http.Request)
GetNodes returns all nodes which exist in the network
func (*Server) JSON ¶
func (s *Server) JSON(w http.ResponseWriter, status int, data interface{})
JSON sends "data" as a JSON HTTP response
func (*Server) LoadSnapshot ¶
func (s *Server) LoadSnapshot(w http.ResponseWriter, req *http.Request)
LoadSnapshot loads a snapshot into the network
func (*Server) NodeRPC ¶
func (s *Server) NodeRPC(w http.ResponseWriter, req *http.Request)
NodeRPC forwards RPC requests to a node in the network via a WebSocket connection
func (*Server) OPTIONS ¶
func (s *Server) OPTIONS(path string, handle http.HandlerFunc)
OPTIONS registers a handler for OPTIONS requests to a particular path
func (*Server) Options ¶
func (s *Server) Options(w http.ResponseWriter, req *http.Request)
Options responds to the OPTIONS HTTP method by returning a 200 OK response with the "Access-Control-Allow-Headers" header set to "Content-Type"
func (*Server) POST ¶
func (s *Server) POST(path string, handle http.HandlerFunc)
POST registers a handler for POST requests to a particular path
func (*Server) ResetNetwork ¶
func (s *Server) ResetNetwork(w http.ResponseWriter, req *http.Request)
ResetNetwork resets all properties of a network to its initial (empty) state
func (*Server) ServeHTTP ¶
func (s *Server) ServeHTTP(w http.ResponseWriter, req *http.Request)
ServeHTTP implements the http.Handler interface by delegating to the underlying httprouter.Router
func (*Server) StartMocker ¶
func (s *Server) StartMocker(w http.ResponseWriter, req *http.Request)
StartMocker starts the mocker node simulation
func (*Server) StartNetwork ¶
func (s *Server) StartNetwork(w http.ResponseWriter, req *http.Request)
StartNetwork starts all nodes in the network
func (*Server) StartNode ¶
func (s *Server) StartNode(w http.ResponseWriter, req *http.Request)
StartNode starts a node
func (*Server) StopMocker ¶
func (s *Server) StopMocker(w http.ResponseWriter, req *http.Request)
StopMocker stops the mocker node simulation
func (*Server) StopNetwork ¶
func (s *Server) StopNetwork(w http.ResponseWriter, req *http.Request)
StopNetwork stops all nodes in the network
func (*Server) StopNode ¶
func (s *Server) StopNode(w http.ResponseWriter, req *http.Request)
StopNode stops a node
func (*Server) StreamNetworkEvents ¶
func (s *Server) StreamNetworkEvents(w http.ResponseWriter, req *http.Request)
StreamNetworkEvents streams network events as a server-sent-events stream
type Simulation ¶
type Simulation struct {
// contains filtered or unexported fields
}
Simulation provides a framework for running actions in a simulated network and then waiting for expectations to be met
func NewSimulation ¶
func NewSimulation(network *Network) *Simulation
NewSimulation returns a new simulation which runs in the given network
func (*Simulation) Run ¶
func (s *Simulation) Run(ctx context.Context, step *Step) (result *StepResult)
Run performs a step of the simulation by performing the step's action and then waiting for the step's expectation to be met
type Snapshot ¶
type Snapshot struct { Nodes []NodeSnapshot `json:"nodes,omitempty"` Conns []Conn `json:"conns,omitempty"` }
Snapshot represents the state of a network at a single point in time and can be used to restore the state of a network
type Step ¶
type Step struct { // Action is the action to perform for this step Action func(context.Context) error // Trigger is a channel which receives node ids and triggers an // expectation check for that node Trigger chan discover.NodeID // Expect is the expectation to wait for when performing this step Expect *Expectation }
type StepResult ¶
type StepResult struct { // Error is the error encountered whilst running the step Error error // StartedAt is the time the step started StartedAt time.Time // FinishedAt is the time the step finished FinishedAt time.Time // Passes are the timestamps of the successful node expectations Passes map[discover.NodeID]time.Time // NetworkEvents are the network events which occurred during the step NetworkEvents []*Event }
type SubscribeOpts ¶
type SubscribeOpts struct { // Current instructs the server to send events for existing nodes and // connections first Current bool // Filter instructs the server to only send a subset of message events Filter string }
SubscribeOpts is a collection of options to use when subscribing to network events