p2p

README

P2P

Direct connections

If you are running multiple nodes in the local network and exposing all of them is not convenient there is an option to setup network manually, by making couple of nodes publicly available and the rest connected to them directly.

Get network id

• get it with grpcurl

grpcurl -plaintext 127.0.0.1:9093 spacemesh.v1.DebugService.NetworkInfo

{
  "id": "12D3KooWRfy4Sj4rDHDuBaYw3Mg5d2puwiCyqBCWMziFquaGQ5g8"
}

• get it from stored data

cat ~/spacemesh/p2p/p2p.key

{"Key":"CAESQAQN38GXvr+L+G+/JWoimpqPBK7I6INe+PKYA+hRJg0I65Q3IPK49Ii9dcnC3+UqB+jMEL16sqDfUubxTs62rZU=","ID":"12D3KooWRfy4Sj4rDHDuBaYw3Mg5d2puwiCyqBCWMziFquaGQ5g8"}

Configuration for public node

Public node should have higher peer limits to help with network connectivity and a list of botnodes. Direct connections should be reciprocal, otherwise public node may prune your private node if overloaded.

Setup more than one public node, and perform rolling upgrades or restart for them if needed.

{
    "p2p": {
        "min-peers": 30,
        "low-peers": 60,
        "high-peers": 100,
        "direct": [
            "/ip4/0.0.0.0/tcp/6000/p2p/12D3KooWRkBh6QayKLb1pDRJGMHE94Lix4ZBVh2BJJeX6mghk8VH"
        ]
    }
}

[!NOTE] Please note that 0.0.0.0 in the above config will work ONLY if all nodes are on the same host. If you're using multiple hosts make sure that you're using proper IPs on both sides. If you're on Windows system, it's recommended to use 127.0.0.1 instead of 0.0.0.0 in the direct part of the config on the same host. You can obviously use any other IP address that is available on the host too, that would allow to use multiple machines for the setup.

Configuration for private node

Set min-peers to the number of peers in the config and disable-dht to true. low-peers and high-peers should not be lower than min-peers.

[!IMPORTANT] Please note that having disable-dht option is critical to properly working public/private node setup.

{
    "p2p": {
        "listen": "/ip4/0.0.0.0/tcp/6000",
        "min-peers": 1,
        "low-peers": 10,
        "high-peers": 20,
        "disable-dht": true,
        "bootnodes": [],
        "direct": [
            "/ip4/0.0.0.0/tcp/7513/p2p/12D3KooWRfy4Sj4rDHDuBaYw3Mg5d2puwiCyqBCWMziFquaGQ5g8"
        ]
    }
}

[!NOTE] Please note that 0.0.0.0 in the above config will work ONLY if all nodes are on the same host. If you're using multiple hosts make sure that you're using proper IPs on both sides. If you're on Windows system, it's recommended to use 127.0.0.1 instead of 0.0.0.0 in the direct part of the config on the same host. You can obviously use any other IP address that is available on the host too, that would allow to use multiple machines for the setup.

Expected result

Public node will maintain many open connections

ss -npO4 | rg spacemesh | rg 7513 | rg ESTAB | wc -l 52

Private will connect only to the specified public node:

ss -npO4 | rg spacemesh | rg 6000

tcp   ESTAB      0      0              127.0.0.1:7513        127.0.0.1:6000  users:(("go-spacemesh",pid=39165,fd=11))
tcp   ESTAB      0      0              127.0.0.1:6000        127.0.0.1:7513  users:(("go-spacemesh",pid=39202,fd=47))

You can also check that by querying the GRPC endpoint:

grpcurl -plaintext 127.0.0.1:9093 spacemesh.v1.AdminService.PeerInfoStream

On your public node you should see many nodes (possibly including bootnodes) and on your private node you should see only the configured public node(s).

Documentation

Index

• Constants
• func EnsureIdentity(dir string) (crypto.PrivKey, error)
• func IdentityInfoFromDir(dir string) (string, error)
• func IsNoPeer(p Peer) bool
• type AddressList
  • func AddressListFromString(s string) (AddressList, error)
  • func AddressListFromStringSlice(s []string) (AddressList, error)
  • func MustParseAddresses(s ...string) AddressList
  • func (al AddressList) String() string
• type AutoNATServer
• type Config
  • func DefaultConfig() Config
  • func (cfg *Config) Validate() error
• type ConnectionInfo
• type DataStats
• type DiscoveryTimings
• type Host
  • func AutoStart(ctx context.Context, logger log.Log, cfg Config, prologue []byte, ...) (*Host, error)
  • func New(_ context.Context, logger log.Log, cfg Config, prologue []byte, ...) (*Host, error)
  • func Upgrade(h host.Host, opts ...Opt) (*Host, error)
  • func (fh *Host) Connected(p Peer) bool
  • func (fh *Host) ConnectedPeerInfo(id peer.ID) *PeerInfo
  • func (fh *Host) DHTServerEnabled() bool
  • func (fh *Host) GetPeers() []Peer
  • func (fh *Host) HaveRelay() bool
  • func (fh *Host) KnownAddresses() []ma.Multiaddr
  • func (fh *Host) ListenAddresses() []ma.Multiaddr
  • func (fh *Host) NATDeviceType() (udpNATType, tcpNATType network.NATDeviceType)
  • func (fh *Host) NeedPeerDiscovery() bool
  • func (fh *Host) PeerCount() uint64
  • func (fh *Host) PeerInfo() peerinfo.PeerInfo
  • func (fh *Host) PeerProtocols(p Peer) ([]protocol.ID, error)
  • func (fh *Host) Ping() *Ping
  • func (fh *Host) ProtocolDataStats() map[protocol.ID]*peerinfo.DataStats
  • func (fh *Host) Reachability() network.Reachability
  • func (fh *Host) Start() error
  • func (fh *Host) Stop() error
• type Opt
  • func WithBootnodes(bootnodes map[peer.ID]struct{}) Opt
  • func WithConfig(cfg Config) Opt
  • func WithContext(ctx context.Context) Opt
  • func WithDirectNodes(direct map[peer.ID]struct{}) Opt
  • func WithLog(logger log.Log) Opt
  • func WithNodeReporter(reporter func()) Opt
  • func WithPeerInfo(pi peerinfo.PeerInfo) Opt
  • func WithRelayCandidateChannel(relayCh chan<- peer.AddrInfo) Opt
• type Peer
• type PeerInfo
• type PeerRequestStats
• type Ping
  • func NewPing(logger *zap.Logger, h host.Host, peers []peer.ID, pr routing.PeerRouting, ...) *Ping
  • func (p *Ping) Start()
  • func (p *Ping) Stats(peerID peer.ID) (numSuccess, numFail int)
  • func (p *Ping) Stop()
• type PingOpt
  • func WithPingInterval(d time.Duration) PingOpt
• type RelayServer

Constants

const (
	PublicReachability  = "public"
	PrivateReachability = "private"
)

Functions

func EnsureIdentity

func EnsureIdentity(dir string) (crypto.PrivKey, error)

EnsureIdentity generates an identity key file in given directory.

func IdentityInfoFromDir

func IdentityInfoFromDir(dir string) (string, error)

IdentityInfoFromDir returns a printable ID from a given identity directory.

func IsNoPeer

func IsNoPeer(p Peer) bool

IsNoPeer checks if it's any peer.

Types

type AddressList

type AddressList []multiaddr.Multiaddr

AddressList represents a list of addresses.

func AddressListFromString

func AddressListFromString(s string) (AddressList, error)

AddressListFromString parses a string containing a Multiaddr and returns it as an AddressList.

func AddressListFromStringSlice

func AddressListFromStringSlice(s []string) (AddressList, error)

AddressListFromStringSlice parses strings in the slice into Multiaddr values and returns them as an AddressList.

func MustParseAddresses

func MustParseAddresses(s ...string) AddressList

MustParseAddresses parses multiaddr strings into AddressList, and panics upon any parse errors.

func (AddressList) String

func (al AddressList) String() string

String implements Stringer.

type AutoNATServer

type AutoNATServer struct {
	GlobalMax   int           `mapstructure:"global-max"`
	PeerMax     int           `mapstructure:"peer-max"`
	ResetPeriod time.Duration `mapstructure:"reset-period"`
}

type Config

type Config struct {
	DataDir            string
	LogLevel           log.Level
	GracePeersShutdown time.Duration `mapstructure:"gracepeersshutdown"`
	MaxMessageSize     int           `mapstructure:"maxmessagesize"`

	// see https://lwn.net/Articles/542629/ for reuseport explanation
	DisableReusePort            bool               `mapstructure:"disable-reuseport"`
	DisableNatPort              bool               `mapstructure:"disable-natport"`
	DisableConnectionManager    bool               `mapstructure:"disable-connection-manager"`
	DisableResourceManager      bool               `mapstructure:"disable-resource-manager"`
	DisableDHT                  bool               `mapstructure:"disable-dht"`
	DisablePubSub               bool               `mapstructure:"disable-pubsub"`
	Flood                       bool               `mapstructure:"flood"`
	Listen                      AddressList        `mapstructure:"listen"`
	Bootnodes                   []string           `mapstructure:"bootnodes"`
	Direct                      []string           `mapstructure:"direct"`
	MinPeers                    int                `mapstructure:"min-peers"`
	LowPeers                    int                `mapstructure:"low-peers"`
	HighPeers                   int                `mapstructure:"high-peers"`
	InboundFraction             float64            `mapstructure:"inbound-fraction"`
	OutboundFraction            float64            `mapstructure:"outbound-fraction"`
	AutoscalePeers              bool               `mapstructure:"autoscale-peers"`
	AdvertiseAddress            AddressList        `mapstructure:"advertise-address"`
	AcceptQueue                 int                `mapstructure:"p2p-accept-queue"`
	Metrics                     bool               `mapstructure:"p2p-metrics"`
	Bootnode                    bool               `mapstructure:"p2p-bootnode"`
	ForceReachability           string             `mapstructure:"p2p-reachability"`
	ForceDHTServer              bool               `mapstructure:"force-dht-server"`
	EnableHolepunching          bool               `mapstructure:"p2p-holepunching"`
	PrivateNetwork              bool               `mapstructure:"p2p-private-network"`
	RelayServer                 RelayServer        `mapstructure:"relay-server"`
	IP4Blocklist                []string           `mapstructure:"ip4-blocklist"`
	IP6Blocklist                []string           `mapstructure:"ip6-blocklist"`
	GossipQueueSize             int                `mapstructure:"gossip-queue-size"`
	GossipPeerOutboundQueueSize int                `mapstructure:"gossip-peer-outbound-queue-size"`
	GossipValidationThrottle    int                `mapstructure:"gossip-validation-throttle"`
	GossipAtxValidationThrottle int                `mapstructure:"gossip-atx-validation-throttle"`
	GossipEvictionStrategy      timecache.Strategy `mapstructure:"gossip-eviction-strategy"`
	PingPeers                   []string           `mapstructure:"ping-peers"`
	PingInterval                time.Duration      `mapstructure:"ping-interval"`
	Relay                       bool               `mapstructure:"relay"`
	StaticRelays                []string           `mapstructure:"static-relays"`
	EnableTCPTransport          bool               `mapstructure:"enable-tcp-transport"`
	EnableQUICTransport         bool               `mapstructure:"enable-quic-transport"`
	EnableRoutingDiscovery      bool               `mapstructure:"enable-routing-discovery"`
	RoutingDiscoveryAdvertise   bool               `mapstructure:"routing-discovery-advertise"`
	DiscoveryTimings            DiscoveryTimings   `mapstructure:"discovery-timings"`
	AutoNATServer               AutoNATServer      `mapstructure:"auto-nat-server"`
}

Config for all things related to p2p layer.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig config.

func (*Config) Validate

func (cfg *Config) Validate() error

type ConnectionInfo

type ConnectionInfo struct {
	Address  ma.Multiaddr
	Uptime   time.Duration
	Outbound bool
	Kind     peerinfo.Kind
}

type DataStats

type DataStats struct {
	BytesSent     int64
	BytesReceived int64
	SendRate      [2]int64
	RecvRate      [2]int64
}

type DiscoveryTimings

type DiscoveryTimings struct {
	AdvertiseDelay          time.Duration `mapstructure:"advertise-delay"`
	AdvertiseInterval       time.Duration `mapstructure:"advertise-interval"`
	AdvertiseIntervalSpread time.Duration `mapstructure:"advertise-interval-spread"`
	AdvertiseRetryDelay     time.Duration `mapstructure:"advertise-retry-delay"`
	FindPeersRetryDelay     time.Duration `mapstructure:"find-peers-retry-delay"`
	MinBackoff              time.Duration `mapstructure:"min-backoff"`
	MaxBackoff              time.Duration `mapstructure:"max-backoff"`
	MinConnBackoff          time.Duration `mapstructure:"min-conn-backoff"`
	MaxConnBackoff          time.Duration `mapstructure:"max-conn-backoff"`
	DialTimeout             time.Duration `mapstructure:"dial-timeout"`
}

type Host

type Host struct {
	host.Host

	pubsub.PubSub
	// contains filtered or unexported fields
}

Host is a conveniency wrapper for all p2p related functionality required to run a full spacemesh node.

func AutoStart

func AutoStart(ctx context.Context,
	logger log.Log,
	cfg Config,
	prologue []byte,
	quicNetCookie handshake.NetworkCookie,
	opts ...Opt,
) (*Host, error)

AutoStart initializes a new host and starts it.

func New

func New(
	_ context.Context,
	logger log.Log,
	cfg Config,
	prologue []byte,
	quicNetCookie handshake.NetworkCookie,
	opts ...Opt,
) (*Host, error)

New initializes libp2p host configured for spacemesh.

func Upgrade

func Upgrade(h host.Host, opts ...Opt) (*Host, error)

Upgrade creates Host instance from host.Host.

func (*Host) Connected

func (fh *Host) Connected(p Peer) bool

Connected returns true if the specified peer is connected. Peers that only have transient connections to them aren't considered connected.

func (*Host) ConnectedPeerInfo

func (fh *Host) ConnectedPeerInfo(id peer.ID) *PeerInfo

ConnectedPeerInfo retrieves a peer info object for the given peer.ID, if the given peer is not connected then nil is returned.

func (*Host) DHTServerEnabled

func (fh *Host) DHTServerEnabled() bool

DHTServerEnabled returns true if the server has DHT running in server mode.

func (*Host) GetPeers

func (fh *Host) GetPeers() []Peer

GetPeers returns connected peers.

func (*Host) HaveRelay

func (fh *Host) HaveRelay() bool

HaveRelay returns true if this host can be used as a relay, that is, it supports relay service and has public reachability.

func (*Host) KnownAddresses

func (fh *Host) KnownAddresses() []ma.Multiaddr

KnownAddresses returns the addresses by which the peers know this one.

func (*Host) ListenAddresses

func (fh *Host) ListenAddresses() []ma.Multiaddr

ListenAddresses returns the addresses on which this host listens.

func (*Host) NATDeviceType

func (fh *Host) NATDeviceType() (udpNATType, tcpNATType network.NATDeviceType)

NATDeviceType returns NATDeviceType returns the NAT device types for UDP and TCP so far for this host.

func (*Host) NeedPeerDiscovery

func (fh *Host) NeedPeerDiscovery() bool

NeedPeerDiscovery returns true if it makes sense to do additional discovery of non-DHT (NATed) peers.

func (*Host) PeerCount

func (fh *Host) PeerCount() uint64

PeerCount returns number of connected peers.

func (*Host) PeerInfo

func (fh *Host) PeerInfo() peerinfo.PeerInfo

func (*Host) PeerProtocols

func (fh *Host) PeerProtocols(p Peer) ([]protocol.ID, error)

PeerProtocols returns the protocols supported by peer.

func (*Host) Ping

func (fh *Host) Ping() *Ping

Ping returns Ping structure for this Host, if any PingPeers are specified in the config. Otherwise, it returns nil.

func (*Host) ProtocolDataStats

func (fh *Host) ProtocolDataStats() map[protocol.ID]*peerinfo.DataStats

ProtocolDataStats returns per-protocol data stats.

func (*Host) Reachability

func (fh *Host) Reachability() network.Reachability

Reachability returns reachability of the host (public, private, unknown).

func (*Host) Start

func (fh *Host) Start() error

func (*Host) Stop

func (fh *Host) Stop() error

Stop background workers and release external resources.

type Opt

type Opt func(fh *Host)

Opt is for configuring Host.

func WithBootnodes

func WithBootnodes(bootnodes map[peer.ID]struct{}) Opt

func WithConfig

func WithConfig(cfg Config) Opt

WithConfig sets Config for Host.

func WithContext

func WithContext(ctx context.Context) Opt

WithContext set context for Host.

func WithDirectNodes

func WithDirectNodes(direct map[peer.ID]struct{}) Opt

func WithLog

func WithLog(logger log.Log) Opt

WithLog configures logger for Host.

func WithNodeReporter

func WithNodeReporter(reporter func()) Opt

WithNodeReporter updates reporter that is notified every time when node added or removed a peer.

func WithPeerInfo

func WithPeerInfo(pi peerinfo.PeerInfo) Opt

func WithRelayCandidateChannel

func WithRelayCandidateChannel(relayCh chan<- peer.AddrInfo) Opt

type Peer

type Peer = peer.ID

Peer is an alias to libp2p's peer.ID.

const NoPeer Peer = ""

NoPeer is used when peer doesn't matter.

type PeerInfo

type PeerInfo struct {
	ID          Peer
	Connections []ConnectionInfo
	ClientStats PeerRequestStats
	ServerStats PeerRequestStats
	DataStats   DataStats
	Tags        []string
}

PeerInfo groups relevant information about a peer.

type PeerRequestStats

type PeerRequestStats struct {
	SuccessCount int
	FailureCount int
	Latency      time.Duration
}

type Ping

type Ping struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func NewPing

func NewPing(logger *zap.Logger, h host.Host, peers []peer.ID, pr routing.PeerRouting, opts ...PingOpt) *Ping

func (*Ping) Start

func (p *Ping) Start()

func (*Ping) Stats

func (p *Ping) Stats(peerID peer.ID) (numSuccess, numFail int)

func (*Ping) Stop

func (p *Ping) Stop()

type PingOpt

type PingOpt func(p *Ping)

func WithPingInterval

func WithPingInterval(d time.Duration) PingOpt

type RelayServer

type RelayServer struct {
	Enable                 bool          `mapstructure:"enable"`
	Reservations           int           `mapstructure:"reservations"`
	TTL                    time.Duration `mapstructure:"ttl"`
	ConnDurationLimit      time.Duration `mapstructure:"conn-duration-limit"`
	ConnDataLimit          int64         `mapstructure:"conn-data-limit"`
	MaxCircuits            int           `mapstructure:"max-circuits"`
	BufferSize             int           `mapstructure:"buffer-size"`
	MaxReservationsPerPeer int           `mapstructure:"max-reservations-per-peer"`
	MaxReservationsPerIP   int           `mapstructure:"max-reservations-per-ip"`
	MaxReservationsPerASN  int           `mapstructure:"max-reservations-per-asn"`
}