simulations

package
v0.0.0-...-b4cfd71 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 26, 2021 License: GPL-3.0 Imports: 21 Imported by: 0

README

devp2p Simulations

The p2p/simulations package implements a simulation framework which supports creating a collection of devp2p nodes, connecting them together to form a simulation network, performing simulation actions in that network and then extracting useful information.

Nodes

Each node in a simulation network runs multiple services by wrapping a collection of objects which implement the node.Service interface meaning they:

  • can be started and stopped
  • run p2p protocols
  • expose RPC APIs

This means that any object which implements the node.Service interface can be used to run a node in the simulation.

Services

Before running a simulation, a set of service initializers must be registered which can then be used to run nodes in the network.

A service initializer is a function with the following signature:

func(ctx *adapters.ServiceContext) (node.Service, error)

These initializers should be registered by calling the adapters.RegisterServices function in an init() hook:

func init() {
	adapters.RegisterServices(adapters.Services{
		"service1": initService1,
		"service2": initService2,
	})
}

Node Adapters

The simulation framework includes multiple "node adapters" which are responsible for creating an environment in which a node runs.

SimAdapter

The SimAdapter runs nodes in-memory, connecting them using an in-memory, synchronous net.Pipe and connecting to their RPC server using an in-memory rpc.Client.

ExecAdapter

The ExecAdapter runs nodes as child processes of the running simulation.

It does this by executing the binary which is running the simulation but setting argv[0] (i.e. the program name) to p2p-node which is then detected by an init hook in the child process which runs the node.Service using the devp2p node stack rather than executing main().

The nodes listen for devp2p connections and WebSocket RPC clients on random localhost ports.

DockerAdapter

The DockerAdapter is similar to the ExecAdapter but executes docker run to run the node in a Docker container using a Docker image containing the simulation binary at /bin/p2p-node.

The Docker image is built using docker build when the adapter is initialised, meaning no prior setup is necessary other than having a working Docker client.

Each node listens on the external IP of the container and the default p2p and RPC ports (20202 and 8546 respectively).

Network

A simulation network is created with an ID and default service (which is used if a node is created without an explicit service), exposes methods for creating, starting, stopping, connecting and disconnecting nodes, and emits events when certain actions occur.

Events

A simulation network emits the following events:

  • node event - when nodes are created / started / stopped
  • connection event - when nodes are connected / disconnected
  • message event - when a protocol message is sent between two nodes

The events have a "control" flag which when set indicates that the event is the outcome of a controlled simulation action (e.g. creating a node or explicitly connecting two nodes together).

This is in contrast to a non-control event, otherwise called a "live" event, which is the outcome of something happening in the network as a result of a control event (e.g. a node actually started up or a connection was actually established between two nodes).

Live events are detected by the simulation network by subscribing to node peer events via RPC when the nodes start up.

Testing Framework

The Simulation type can be used in tests to perform actions in a simulation network and then wait for expectations to be met.

With a running simulation network, the Simulation.Run method can be called with a Step which has the following fields:

  • Action - a function which performs some action in the network

  • Expect - an expectation function which returns whether or not a given node meets the expectation

  • Trigger - a channel which receives node IDs which then trigger a check of the expectation function to be performed against that node

As a concrete example, consider a simulated network of Ethereum nodes. An Action could be the sending of a transaction, Expect it being included in a block, and Trigger a check for every block that is mined.

On return, the Simulation.Run method returns a StepResult which can be used to determine if all nodes met the expectation, how long it took them to meet the expectation and what network events were emitted during the step run.

HTTP API

The simulation framework includes a HTTP API which can be used to control the simulation.

The API is initialised with a particular node adapter and has the following endpoints:

GET    /                            Get network information
POST   /start                       Start all nodes in the network
POST   /stop                        Stop all nodes in the network
GET    /events                      Stream network events
GET    /snapshot                    Take a network snapshot
POST   /snapshot                    Load a network snapshot
POST   /nodes                       Create a node
GET    /nodes                       Get all nodes in the network
GET    /nodes/:nodeid               Get node information
POST   /nodes/:nodeid/start         Start a node
POST   /nodes/:nodeid/stop          Stop a node
POST   /nodes/:nodeid/conn/:peerid  Connect two nodes
DELETE /nodes/:nodeid/conn/:peerid  Disconnect two nodes
GET    /nodes/:nodeid/rpc           Make RPC requests to a node via WebSocket

For convenience, nodeid in the URL can be the name of a node rather than its ID.

Command line client

p2psim is a command line client for the HTTP API, located in cmd/p2psim.

It provides the following commands:

p2psim show
p2psim events [--current] [--filter=FILTER]
p2psim snapshot
p2psim load
p2psim node create [--name=NAME] [--services=SERVICES] [--key=KEY]
p2psim node list
p2psim node show <node>
p2psim node start <node>
p2psim node stop <node>
p2psim node connect <node> <peer>
p2psim node disconnect <node> <peer>
p2psim node rpc <node> <method> [<args>] [--subscribe]

Example

See p2p/simulations/examples/README.md.

Documentation

Overview

Package simulations simulates p2p networks. A mocker simulates starting and stopping real nodes in a network.

Index

Constants

This section is empty.

Variables

View Source 
var DefaultClient = NewClient("http://localhost:8888")

DefaultClient is the default simulation API client which expects the API to be running at http://localhost:8888

Functions

func ConnLabel 

func ConnLabel(source, target discover.NodeID) string

ConnLabel generates a deterministic string which represents a connection between two nodes, used to compare if two connections are between the same nodes

func GetMockerList 

func GetMockerList() []string

Get a list of mockers (keys of the map) Useful for frontend to build available mocker selection

func LookupMocker 

func LookupMocker(mockerType string) func(net *Network, quit chan struct{}, nodeCount int)

Lookup a mocker by its name, returns the mockerFn

Types

type Client 

type Client struct {
	URL string
	// contains filtered or unexported fields
}

Client is a client for the simulation HTTP API which supports creating and managing simulation networks

func NewClient 

func NewClient(url string) *Client

NewClient returns a new simulation API client

func (*Client) ConnectNode 

func (c *Client) ConnectNode(nodeID, peerID string) error

ConnectNode connects a node to a peer node

func (*Client) CreateNode 

func (c *Client) CreateNode(config *adapters.NodeConfig) (*p2p.NodeInfo, error)

CreateNode creates a node in the network using the given configuration

func (*Client) CreateSnapshot 

func (c *Client) CreateSnapshot() (*Snapshot, error)

CreateSnapshot creates a network snapshot

func (*Client) Delete 

func (c *Client) Delete(path string) error

Delete performs a HTTP DELETE request

func (*Client) DisconnectNode 

func (c *Client) DisconnectNode(nodeID, peerID string) error

DisconnectNode disconnects a node from a peer node

func (*Client) Get 

func (c *Client) Get(path string, out interface{}) error

Get performs a HTTP GET request decoding the resulting JSON response into "out"

func (*Client) GetNetwork 

func (c *Client) GetNetwork() (*Network, error)

GetNetwork returns details of the network

func (*Client) GetNode 

func (c *Client) GetNode(nodeID string) (*p2p.NodeInfo, error)

GetNode returns details of a node

func (*Client) GetNodes 

func (c *Client) GetNodes() ([]*p2p.NodeInfo, error)

GetNodes returns all nodes which exist in the network

func (*Client) LoadSnapshot 

func (c *Client) LoadSnapshot(snap *Snapshot) error

LoadSnapshot loads a snapshot into the network

func (*Client) Post 

func (c *Client) Post(path string, in, out interface{}) error

Post performs a HTTP POST request sending "in" as the JSON body and decoding the resulting JSON response into "out"

func (*Client) RPCClient 

func (c *Client) RPCClient(ctx context.Context, nodeID string) (*rpc.Client, error)

RPCClient returns an RPC client connected to a node

func (*Client) Send 

func (c *Client) Send(method, path string, in, out interface{}) error

Send performs a HTTP request, sending "in" as the JSON request body and decoding the JSON response into "out"

func (*Client) StartNetwork 

func (c *Client) StartNetwork() error

StartNetwork starts all existing nodes in the simulation network

func (*Client) StartNode 

func (c *Client) StartNode(nodeID string) error

StartNode starts a node

func (*Client) StopNetwork 

func (c *Client) StopNetwork() error

StopNetwork stops all existing nodes in a simulation network

func (*Client) StopNode 

func (c *Client) StopNode(nodeID string) error

StopNode stops a node

func (*Client) SubscribeNetwork 

func (c *Client) SubscribeNetwork(events chan *Event, opts SubscribeOpts) (event.Subscription, error)

SubscribeNetwork subscribes to network events which are sent from the server as a server-sent-events stream, optionally receiving events for existing nodes and connections and filtering message events

type Conn 

type Conn struct {
	// One is the node which initiated the connection
	One discover.NodeID `json:"one"`

	// Other is the node which the connection was made to
	Other discover.NodeID `json:"other"`

	// Up tracks whether or not the connection is active
	Up bool `json:"up"`
	// contains filtered or unexported fields
}

Conn represents a connection between two nodes in the network

func (*Conn) String 

func (self *Conn) String() string

String returns a log-friendly string

type Event 

type Event struct {
	// Type is the type of the event
	Type EventType `json:"type"`

	// Time is the time the event happened
	Time time.Time `json:"time"`

	// Control indicates whether the event is the result of a controlled
	// action in the network
	Control bool `json:"control"`

	// Node is set if the type is EventTypeNode
	Node *Node `json:"node,omitempty"`

	// Conn is set if the type is EventTypeConn
	Conn *Conn `json:"conn,omitempty"`

	// Msg is set if the type is EventTypeMsg
	Msg *Msg `json:"msg,omitempty"`
}

Event is an event emitted by a simulation network

func ControlEvent 

func ControlEvent(v interface{}) *Event

ControlEvent creates a new control event

func NewEvent 

func NewEvent(v interface{}) *Event

NewEvent creates a new event for the given object which should be either a Node, Conn or Msg.

The object is copied so that the event represents the state of the object when NewEvent is called.

func (*Event) String 

func (e *Event) String() string

String returns the string representation of the event

type EventType 

type EventType string

EventType is the type of event emitted by a simulation network 

const (
	// EventTypeNode is the type of event emitted when a node is either
	// created, started or stopped
	EventTypeNode EventType = "node"

	// EventTypeConn is the type of event emitted when a connection is
	// is either established or dropped between two nodes
	EventTypeConn EventType = "conn"

	// EventTypeMsg is the type of event emitted when a p2p message it
	// sent between two nodes
	EventTypeMsg EventType = "msg"
)

type Expectation 

type Expectation struct {
	// Nodes is a list of nodes to check
	Nodes []discover.NodeID

	// Check checks whether a given node meets the expectation
	Check func(context.Context, discover.NodeID) (bool, error)
}

type Msg 

type Msg struct {
	One      discover.NodeID `json:"one"`
	Other    discover.NodeID `json:"other"`
	Protocol string          `json:"protocol"`
	Code     uint64          `json:"code"`
	Received bool            `json:"received"`
}

Msg represents a p2p message sent between two nodes in the network

func (*Msg) String 

func (self *Msg) String() string

String returns a log-friendly string

type MsgFilter 

type MsgFilter struct {
	// Proto is matched against a message's protocol
	Proto string

	// Code is matched against a message's code, with -1 matching all codes
	Code int64
}

MsgFilter is used to filter message events based on protocol and message code

type MsgFilters 

type MsgFilters map[MsgFilter]struct{}

MsgFilters is a collection of filters which are used to filter message events

func NewMsgFilters 

func NewMsgFilters(filterParam string) (MsgFilters, error)

NewMsgFilters constructs a collection of message filters from a URL query parameter.

The parameter is expected to be a dash-separated list of individual filters, each having the format '<proto>:<codes>', where <proto> is the name of a protocol and <codes> is a comma-separated list of message codes.

A message code of '*' or '-1' is considered a wildcard and matches any code.

func (MsgFilters) Match 

func (m MsgFilters) Match(msg *Msg) bool

Match checks if the given message matches any of the filters

type Network 

type Network struct {
	NetworkConfig

	Nodes []*Node `json:"nodes"`

	Conns []*Conn `json:"conns"`
	// contains filtered or unexported fields
}

Network models a p2p simulation network which consists of a collection of simulated nodes and the connections which exist between them.

The Network has a single NodeAdapter which is responsible for actually starting nodes and connecting them together.

The Network emits events when nodes are started and stopped, when they are connected and disconnected, and also when messages are sent between nodes.

func NewNetwork 

func NewNetwork(nodeAdapter adapters.NodeAdapter, conf *NetworkConfig) *Network

NewNetwork returns a Network which uses the given NodeAdapter and NetworkConfig

func (*Network) Config 

func (self *Network) Config() *NetworkConfig

Config returns the network configuration

func (*Network) Connect 

func (self *Network) Connect(oneID, otherID discover.NodeID) error

Connect connects two nodes together by calling the "admin_addPeer" RPC method on the "one" node so that it connects to the "other" node

func (*Network) DidConnect 

func (self *Network) DidConnect(one, other discover.NodeID) error

DidConnect tracks the fact that the "one" node connected to the "other" node

func (*Network) DidDisconnect 

func (self *Network) DidDisconnect(one, other discover.NodeID) error

DidDisconnect tracks the fact that the "one" node disconnected from the "other" node

func (*Network) DidReceive 

func (self *Network) DidReceive(sender, receiver discover.NodeID, proto string, code uint64) error

DidReceive tracks the fact that "receiver" received a message from "sender"

func (*Network) DidSend 

func (self *Network) DidSend(sender, receiver discover.NodeID, proto string, code uint64) error

DidSend tracks the fact that "sender" sent a message to "receiver"

func (*Network) Disconnect 

func (self *Network) Disconnect(oneID, otherID discover.NodeID) error

Disconnect disconnects two nodes by calling the "admin_removePeer" RPC method on the "one" node so that it disconnects from the "other" node

func (*Network) Events 

func (self *Network) Events() *event.Feed

Events returns the output event feed of the Network.

func (*Network) GetConn 

func (self *Network) GetConn(oneID, otherID discover.NodeID) *Conn

GetConn returns the connection which exists between "one" and "other" regardless of which node initiated the connection

func (*Network) GetNode 

func (self *Network) GetNode(id discover.NodeID) *Node

GetNode gets the node with the given ID, returning nil if the node does not exist

func (*Network) GetNodeByName 

func (self *Network) GetNodeByName(name string) *Node

GetNode gets the node with the given name, returning nil if the node does not exist

func (*Network) GetNodes 

func (self *Network) GetNodes() (nodes []*Node)

GetNodes returns the existing nodes

func (*Network) GetOrCreateConn 

func (self *Network) GetOrCreateConn(oneID, otherID discover.NodeID) (*Conn, error)

GetOrCreateConn is like GetConn but creates the connection if it doesn't already exist

func (*Network) InitConn 

func (self *Network) InitConn(oneID, otherID discover.NodeID) (*Conn, error)

InitConn(one, other) retrieves the connectiton model for the connection between peers one and other, or creates a new one if it does not exist the order of nodes does not matter, i.e., Conn(i,j) == Conn(j, i) it checks if the connection is already up, and if the nodes are running NOTE: it also checks whether there has been recent attempt to connect the peers this is cheating as the simulation is used as an oracle and know about remote peers attempt to connect to a node which will then not initiate the connection

func (*Network) Load 

func (self *Network) Load(snap *Snapshot) error

Load loads a network snapshot

func (*Network) NewNode 

func (self *Network) NewNode() (*Node, error)

NewNode adds a new node to the network with a random ID

func (*Network) NewNodeWithConfig 

func (self *Network) NewNodeWithConfig(conf *adapters.NodeConfig) (*Node, error)

NewNodeWithConfig adds a new node to the network with the given config, returning an error if a node with the same ID or name already exists

func (*Network) Reset 

func (self *Network) Reset()

Reset resets all network properties: emtpies the nodes and the connection list

func (*Network) Shutdown 

func (self *Network) Shutdown()

Shutdown stops all nodes in the network and closes the quit channel

func (*Network) Snapshot 

func (self *Network) Snapshot() (*Snapshot, error)

Snapshot creates a network snapshot

func (*Network) Start 

func (self *Network) Start(id discover.NodeID) error

Start starts the node with the given ID

func (*Network) StartAll 

func (self *Network) StartAll() error

StartAll starts all nodes in the network

func (*Network) Stop 

func (self *Network) Stop(id discover.NodeID) error

Stop stops the node with the given ID

func (*Network) StopAll 

func (self *Network) StopAll() error

StopAll stops all nodes in the network

func (*Network) Subscribe 

func (self *Network) Subscribe(events chan *Event)

Subscribe reads control events from a channel and executes them

type NetworkConfig 

type NetworkConfig struct {
	ID             string `json:"id"`
	DefaultService string `json:"default_service,omitempty"`
}

NetworkConfig defines configuration options for starting a Network

type Node 

type Node struct {
	adapters.Node `json:"-"`

	// Config if the config used to created the node
	Config *adapters.NodeConfig `json:"config"`

	// Up tracks whether or not the node is running
	Up bool `json:"up"`
}

Node is a wrapper around adapters.Node which is used to track the status of a node in the network

func (*Node) ID 

func (self *Node) ID() discover.NodeID

ID returns the ID of the node

func (*Node) MarshalJSON 

func (self *Node) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface so that the encoded JSON includes the NodeInfo

func (*Node) NodeInfo 

func (self *Node) NodeInfo() *p2p.NodeInfo

NodeInfo returns information about the node

func (*Node) String 

func (self *Node) String() string

String returns a log-friendly string

type NodeSnapshot 

type NodeSnapshot struct {
	Node Node `json:"node,omitempty"`

	// Snapshots is arbitrary data gathered from calling node.Snapshots()
	Snapshots map[string][]byte `json:"snapshots,omitempty"`
}

NodeSnapshot represents the state of a node in the network

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server is an HTTP server providing an API to manage a simulation network

func NewServer 

func NewServer(network *Network) *Server

NewServer returns a new simulation API server

func (*Server) ConnectNode 

func (s *Server) ConnectNode(w http.ResponseWriter, req *http.Request)

ConnectNode connects a node to a peer node

func (*Server) CreateNode 

func (s *Server) CreateNode(w http.ResponseWriter, req *http.Request)

CreateNode creates a node in the network using the given configuration

func (*Server) CreateSnapshot 

func (s *Server) CreateSnapshot(w http.ResponseWriter, req *http.Request)

CreateSnapshot creates a network snapshot

func (*Server) DELETE 

func (s *Server) DELETE(path string, handle http.HandlerFunc)

DELETE registers a handler for DELETE requests to a particular path

func (*Server) DisconnectNode 

func (s *Server) DisconnectNode(w http.ResponseWriter, req *http.Request)

DisconnectNode disconnects a node from a peer node

func (*Server) GET 

func (s *Server) GET(path string, handle http.HandlerFunc)

GET registers a handler for GET requests to a particular path

func (*Server) GetMockers 

func (s *Server) GetMockers(w http.ResponseWriter, req *http.Request)

GetMockerList returns a list of available mockers

func (*Server) GetNetwork 

func (s *Server) GetNetwork(w http.ResponseWriter, req *http.Request)

GetNetwork returns details of the network

func (*Server) GetNode 

func (s *Server) GetNode(w http.ResponseWriter, req *http.Request)

GetNode returns details of a node

func (*Server) GetNodes 

func (s *Server) GetNodes(w http.ResponseWriter, req *http.Request)

GetNodes returns all nodes which exist in the network

func (*Server) JSON 

func (s *Server) JSON(w http.ResponseWriter, status int, data interface{})

JSON sends "data" as a JSON HTTP response

func (*Server) LoadSnapshot 

func (s *Server) LoadSnapshot(w http.ResponseWriter, req *http.Request)

LoadSnapshot loads a snapshot into the network

func (*Server) NodeRPC 

func (s *Server) NodeRPC(w http.ResponseWriter, req *http.Request)

NodeRPC forwards RPC requests to a node in the network via a WebSocket connection

func (*Server) OPTIONS 

func (s *Server) OPTIONS(path string, handle http.HandlerFunc)

OPTIONS registers a handler for OPTIONS requests to a particular path

func (*Server) Options 

func (s *Server) Options(w http.ResponseWriter, req *http.Request)

Options responds to the OPTIONS HTTP method by returning a 200 OK response with the "Access-Control-Allow-Headers" header set to "Content-Type"

func (*Server) POST 

func (s *Server) POST(path string, handle http.HandlerFunc)

POST registers a handler for POST requests to a particular path

func (*Server) ResetNetwork 

func (s *Server) ResetNetwork(w http.ResponseWriter, req *http.Request)

ResetNetwork resets all properties of a network to its initial (empty) state

func (*Server) ServeHTTP 

func (s *Server) ServeHTTP(w http.ResponseWriter, req *http.Request)

ServeHTTP implements the http.Handler interface by delegating to the underlying httprouter.Router

func (*Server) StartMocker 

func (s *Server) StartMocker(w http.ResponseWriter, req *http.Request)

StartMocker starts the mocker node simulation

func (*Server) StartNetwork 

func (s *Server) StartNetwork(w http.ResponseWriter, req *http.Request)

StartNetwork starts all nodes in the network

func (*Server) StartNode 

func (s *Server) StartNode(w http.ResponseWriter, req *http.Request)

StartNode starts a node

func (*Server) StopMocker 

func (s *Server) StopMocker(w http.ResponseWriter, req *http.Request)

StopMocker stops the mocker node simulation

func (*Server) StopNetwork 

func (s *Server) StopNetwork(w http.ResponseWriter, req *http.Request)

StopNetwork stops all nodes in the network

func (*Server) StopNode 

func (s *Server) StopNode(w http.ResponseWriter, req *http.Request)

StopNode stops a node

func (*Server) StreamNetworkEvents 

func (s *Server) StreamNetworkEvents(w http.ResponseWriter, req *http.Request)

StreamNetworkEvents streams network events as a server-sent-events stream

type Simulation 

type Simulation struct {
	// contains filtered or unexported fields
}

Simulation provides a framework for running actions in a simulated network and then waiting for expectations to be met

func NewSimulation 

func NewSimulation(network *Network) *Simulation

NewSimulation returns a new simulation which runs in the given network

func (*Simulation) Run 

func (s *Simulation) Run(ctx context.Context, step *Step) (result *StepResult)

Run performs a step of the simulation by performing the step's action and then waiting for the step's expectation to be met

type Snapshot 

type Snapshot struct {
	Nodes []NodeSnapshot `json:"nodes,omitempty"`
	Conns []Conn         `json:"conns,omitempty"`
}

Snapshot represents the state of a network at a single point in time and can be used to restore the state of a network

type Step 

type Step struct {
	// Action is the action to perform for this step
	Action func(context.Context) error

	// Trigger is a channel which receives node ids and triggers an
	// expectation check for that node
	Trigger chan discover.NodeID

	// Expect is the expectation to wait for when performing this step
	Expect *Expectation
}

type StepResult 

type StepResult struct {
	// Error is the error encountered whilst running the step
	Error error

	// StartedAt is the time the step started
	StartedAt time.Time

	// FinishedAt is the time the step finished
	FinishedAt time.Time

	// Passes are the timestamps of the successful node expectations
	Passes map[discover.NodeID]time.Time

	// NetworkEvents are the network events which occurred during the step
	NetworkEvents []*Event
}

type SubscribeOpts 

type SubscribeOpts struct {
	// Current instructs the server to send events for existing nodes and
	// connections first
	Current bool

	// Filter instructs the server to only send a subset of message events
	Filter string
}

SubscribeOpts is a collection of options to use when subscribing to network events

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.