dhtnetwork

package
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 1, 2018 License: Apache-2.0 Imports: 35 Imported by: 0

README

Insolar – Host Network

Physical networking layer

Go Report Card GoDoc

Overview

We took Kademlia DHT original specifications and made significant improvements to make it ready for real world application by enterprises.

Key features of our blockchain network layer:
  • Support of heterogeneous network topology with different types of hosts being able to communicate with each other. In classic peer-to-peer networks, any host can communicate directly with any other host on the network. In a real enterprise environment, this condition is often unacceptable for a variety of reasons including security.
  • Network routing with a host or host group becoming relays for others hosts. The network can continue to function despite various network restrictions such as firewalls, NATs, etc.
  • Ability to limit number of gateways to corporate host group via relays to keep the host group secure while being able to interact with the rest of the network through relays. This feature mitigates the risk of DDoS attacks.

Key components

Transport

Network transport interface. It allows to abstract our network from physical transport. It can either be IP based network or any other kind of packet courier (e.g. an industrial packet bus).

Host

Host is a fundamental part of networking system. Each host has:

  • one real network address (IP or any other transport protocol address)
  • multiple abstract network IDs (either host's own or ones belonging to relayed hosts)
Routing

It is actually a Kademlia hash table used to store network hosts and calculate distances between them. See Kademlia whitepaper and XLattice design specification for details.

Packet

A set of data transferred by this module between hosts.

  • Request packet
  • Response packet

Now packets are serialized simply with encoding/gob. In future there will be a powerful robust serialization system based on Google's Protocol Buffers.

RPC

RPC module allows higher level components to register methods that can be called by other network hosts.

Configuration

Default Host Network config:

host:
  transport:
    protocol: UTP
    address: 127.0.0.1:0
    behindnat: true
  bootstraphosts: []
  isrelay: false

The listen port is zero(which means random) for unit test purpose. For host bootstrapping you need to fill bootstraphosts slice. See godoc for more details.

Usage

package main

import (
	"github.com/insolar/insolar/network/hostnetwork"
	"github.com/insolar/insolar/configuration"
)

func main() {
	cfg := configuration.NewConfiguration().Host
	cfg.Address = "0.0.0.0:31337"

	network, err := hostnetwork.NewHostNetwork(cfg)
	if err != nil {
		panic(err)
	}
	defer network.Disconnect()

	network.Listen()
}

For more detailed usage example see cmd/example/network/hostnetwork/main.go

Documentation

Overview

Package hostnetwork is an implementation of Kademlia DHT. It is mostly based on original specification but has multiple backward-incompatible changes.

Usage: 

package main

import (
	"github.com/insolar/insolar/network/hostnetwork"
	"github.com/insolar/insolar/configuration"
)

func main() {
	cfg := configuration.NewConfiguration().Host
	cfg.Address = "0.0.0.0:31337"

	network, err := hostnetwork.NewHostNetwork(cfg)
	if err != nil {
		panic(err)
	}
	defer network.Disconnect()

	network.Listen()
}

Index

Constants

View Source 
const (
	CtxTableIndex = ctxKey("table_index")
	DefaultHostID = 0
)

Variables

This section is empty.

Functions

func AuthenticationRequest 

func AuthenticationRequest(hostHandler hosthandler.HostHandler, command, targetID string) error

AuthenticationRequest sends an authentication request.

func BuildContext 

func BuildContext(cb ContextBuilder, msg *packet.Packet) hosthandler.Context

BuildContext builds a context for packet.

func CascadeSendMessage 

func CascadeSendMessage(hostHandler hosthandler.HostHandler, data core.Cascade, targetID string, method string, args [][]byte) error

CascadeSendMessage sends a message to the next cascade layer.

func CheckOriginRequest 

func CheckOriginRequest(hostHandler hosthandler.HostHandler, targetID string) error

CheckOriginRequest send a request to check target host originality

func CreateDHTContext 

func CreateDHTContext(handler hosthandler.HostHandler) hosthandler.Context

func DispatchPacketType 

func DispatchPacketType(
	hostHandler hosthandler.HostHandler,
	ctx hosthandler.Context,
	msg *packet.Packet,
	packetBuilder packet.Builder,
) (*packet.Packet, error)

DispatchPacketType checks message type.

func GetDefaultCtx 

func GetDefaultCtx(hostHandler hosthandler.HostHandler) hosthandler.Context

GetDefaultCtx creates default context for the host handler.

func GetNonceRequest 

func GetNonceRequest(hostHandler hosthandler.HostHandler, targetID string) ([]*core.Node, error)

func NewDhtHostNetwork 

func NewDhtHostNetwork(conf configuration.Configuration, certificate core.Certificate, pulseCallback network.OnPulse) (network.HostNetwork, error)

func NewDhtNetworkController 

func NewDhtNetworkController(network network.HostNetwork) (network.Controller, error)

func NewNetworkConsensus 

func NewNetworkConsensus(network network.HostNetwork) consensus.Processor

func ObtainIPRequest 

func ObtainIPRequest(hostHandler hosthandler.HostHandler, targetID string) error

ObtainIPRequest is request to self IP obtaining.

func ParseIncomingPacket 

func ParseIncomingPacket(hostHandler hosthandler.HostHandler, ctx hosthandler.Context, msg *packet.Packet, packetBuilder packet.Builder) (*packet.Packet, error)

ParseIncomingPacket detects a packet type.

func RelayOwnershipRequest 

func RelayOwnershipRequest(hostHandler hosthandler.HostHandler, targetID string) error

RelayOwnershipRequest sends a relay ownership request.

func RelayRequest 

func RelayRequest(hostHandler hosthandler.HostHandler, command, targetID string) error

RelayRequest sends relay request to target.

func ResendPulseToKnownHosts 

func ResendPulseToKnownHosts(hostHandler hosthandler.HostHandler, hosts []host.Host, pulse *packet.RequestPulse)

ResendPulseToKnownHosts resends received pulse to all known hosts

func SendRelayOwnership 

func SendRelayOwnership(hostHandler hosthandler.HostHandler, subnetIDs []string)

SendRelayOwnership send a relay ownership request.

Types

type AuthInfo 

type AuthInfo struct {
	// Sent/received unique auth keys.
	SentKeys     map[string][]byte
	ReceivedKeys map[string][]byte

	AuthenticatedHosts map[string]bool
}

AuthInfo collects some information about authentication.

type ContextBuilder 

type ContextBuilder struct {
	// contains filtered or unexported fields
}

ContextBuilder allows to lazy configure and build new Context.

func NewContextBuilder 

func NewContextBuilder(hostHandler hosthandler.HostHandler) ContextBuilder

NewContextBuilder creates new ContextBuilder.

func (ContextBuilder) Build 

func (cb ContextBuilder) Build() (ctx hosthandler.Context, err error)

Build builds and returns new Context.

func (ContextBuilder) SetDefaultHost 

func (cb ContextBuilder) SetDefaultHost() ContextBuilder

SetDefaultHost sets first host id in Context.

func (ContextBuilder) SetHostByID 

func (cb ContextBuilder) SetHostByID(hostID id.ID) ContextBuilder

SetHostByID sets host id in Context.

type DHT 

type DHT struct {
	// contains filtered or unexported fields
}

DHT represents the state of the local host in the distributed hash table.

func NewDHT 

func NewDHT(
	store store.Store,
	origin *host.Origin,
	transport transport.Transport,
	ncf hosthandler.NetworkCommonFacade,
	options *Options,
	proxy relay.Proxy,
	timeout int,
	infbootstrap bool,
	nodeID core.RecordRef,
	majorityRule int,
	certificate core.Certificate,
) (dht *DHT, err error)

NewDHT initializes a new DHT host.

func NewHostNetwork 

func NewHostNetwork(
	cfg configuration.Configuration,
	cascade *cascade.Cascade,
	certificate core.Certificate,
	pulseCallback network.OnPulse,
) (*DHT, error)

NewHostNetwork creates and returns DHT network.

func (*DHT) AddActiveNodes 

func (dht *DHT) AddActiveNodes(activeNodes []*core.Node) error

AddActiveNodes adds an active nodes slice.

func (*DHT) AddAuthSentKey 

func (dht *DHT) AddAuthSentKey(id string, key []byte)

AddAuthSentKey adds a sent key to auth.

func (*DHT) AddHost 

func (dht *DHT) AddHost(ctx hosthandler.Context, host *routing.RouteHost)

AddHost adds a host into the appropriate k bucket we store these buckets in big-endian order so we look at the bits from right to left in order to find the appropriate bucket

func (*DHT) AddPossibleProxyID 

func (dht *DHT) AddPossibleProxyID(id string)

AddPossibleProxyID adds an id which could be a proxy.

func (*DHT) AddPossibleRelayID 

func (dht *DHT) AddPossibleRelayID(id string)

AddPossibleRelayID add a host id which can be a relay.

func (*DHT) AddProxyHost 

func (dht *DHT) AddProxyHost(targetID string)

AddProxyHost adds a proxy host.

func (*DHT) AddReceivedKey 

func (dht *DHT) AddReceivedKey(target string, key []byte)

AddReceivedKey adds a new received key from target.

func (*DHT) AddRelayClient 

func (dht *DHT) AddRelayClient(host *host.Host) error

AddRelayClient adds a new relay client.

func (*DHT) AddSubnetID 

func (dht *DHT) AddSubnetID(ip, targetID string)

AddSubnetID adds a subnet ID.

func (*DHT) AddUnsync 

func (dht *DHT) AddUnsync(nodeID core.RecordRef, roles []core.NodeRole, address string,
	version string) (chan *core.Node, error)

func (*DHT) AnalyzeNetwork 

func (dht *DHT) AnalyzeNetwork(ctx hosthandler.Context) error

AnalyzeNetwork is func to analyze the network after IP obtaining.

func (*DHT) Bootstrap 

func (dht *DHT) Bootstrap() error

Bootstrap attempts to bootstrap the network using the BootstrapHosts provided to the Options struct. This will trigger an iterateBootstrap to the provided BootstrapHosts.

func (*DHT) CascadeSendMessage 

func (dht *DHT) CascadeSendMessage(data core.Cascade, targetID string, method string, args [][]byte) error

CascadeSendMessage sends a message to the next cascade layer.

func (*DHT) CheckNodeRole 

func (dht *DHT) CheckNodeRole(domainID string) error

CheckNodeRole starting a check all known nodes.

func (*DHT) ConfirmNodeRole 

func (dht *DHT) ConfirmNodeRole(roleKey string) bool

ConfirmNodeRole is a node role confirmation.

func (*DHT) Disconnect 

func (dht *DHT) Disconnect()

Disconnect will trigger a Stop from the network.

func (*DHT) EqualAuthSentKey 

func (dht *DHT) EqualAuthSentKey(targetID string, key []byte) bool

EqualAuthSentKey returns true if a given key equals to targetID key.

func (*DHT) FindHost 

func (dht *DHT) FindHost(ctx hosthandler.Context, key string) (*host.Host, bool, error)

FindHost returns target host's real network address.

func (*DHT) Get 

func (dht *DHT) Get(ctx hosthandler.Context, key string) ([]byte, bool, error)

Get retrieves data from the transport using key. Key is the base58 encoded identifier of the data.

func (*DHT) GetActiveNodesList 

func (dht *DHT) GetActiveNodesList() []*core.Node

GetActiveNodesList returns an active nodes list.

func (*DHT) GetExpirationTime 

func (dht *DHT) GetExpirationTime(ctx hosthandler.Context, key []byte) time.Time

GetExpirationTime returns a expiration time after which a key/value pair expires.

func (*DHT) GetHighKnownHostID 

func (dht *DHT) GetHighKnownHostID() string

GetHighKnownHostID returns a high known host ID.

func (*DHT) GetHostsFromBootstrap 

func (dht *DHT) GetHostsFromBootstrap()

func (*DHT) GetNetworkCommonFacade 

func (dht *DHT) GetNetworkCommonFacade() hosthandler.NetworkCommonFacade

GetNetworkCommonFacade returns a networkcommonfacade ptr.

func (*DHT) GetNodeID 

func (dht *DHT) GetNodeID() core.RecordRef

GetNodeID returns a node ID.

func (*DHT) GetOriginHost 

func (dht *DHT) GetOriginHost() *host.Origin

GetOriginHost returns the local host.

func (*DHT) GetOuterHostsCount 

func (dht *DHT) GetOuterHostsCount() int

GetOuterHostsCount returns a outer hosts count.

func (*DHT) GetPacketTimeout 

func (dht *DHT) GetPacketTimeout() time.Duration

GetPacketTimeout returns the maximum time to wait for a response to any packet.

func (*DHT) GetPrivateKey 

func (dht *DHT) GetPrivateKey() *ecdsa.PrivateKey

GetPrivateKey returns a private key.

func (*DHT) GetProxyHostsCount 

func (dht *DHT) GetProxyHostsCount() int

GetProxyHostsCount returns a proxy hosts count.

func (*DHT) GetReplicationTime 

func (dht *DHT) GetReplicationTime() time.Duration

GetReplicationTime returns a interval between Kademlia replication events.

func (*DHT) GetSelfKnownOuterHosts 

func (dht *DHT) GetSelfKnownOuterHosts() int

GetSelfKnownOuterHosts return a self known hosts count.

func (*DHT) HostIsAuthenticated 

func (dht *DHT) HostIsAuthenticated(targetID string) bool

HostIsAuthenticated returns true if target ID is authenticated host.

func (*DHT) HtFromCtx 

func (dht *DHT) HtFromCtx(ctx hosthandler.Context) *routing.HashTable

HtFromCtx returns a routing hashtable known by ctx.

func (*DHT) InvokeRPC 

func (dht *DHT) InvokeRPC(sender *host.Host, method string, args [][]byte) ([]byte, error)

InvokeRPC - invoke a method to rpc.

func (*DHT) KeyIsReceived 

func (dht *DHT) KeyIsReceived(key string) ([]byte, bool)

KeyIsReceived returns true and a key from targetID if exist.

func (*DHT) Listen 

func (dht *DHT) Listen() error

Listen begins listening on the socket for incoming Packets.

func (*DHT) NumHosts 

func (dht *DHT) NumHosts(ctx hosthandler.Context) int

NumHosts returns the total number of hosts stored in the local routing table.

func (*DHT) ObtainIP 

func (dht *DHT) ObtainIP() error

ObtainIP starts to self IP obtaining.

func (*DHT) RemoteProcedureCall 

func (dht *DHT) RemoteProcedureCall(ctx hosthandler.Context, targetID string, method string, args [][]byte) (result []byte, err error)

RemoteProcedureCall calls remote procedure on target host.

func (*DHT) RemoteProcedureRegister 

func (dht *DHT) RemoteProcedureRegister(name string, method core.RemoteProcedure)

RemoteProcedureRegister registers procedure for remote call on this host

func (*DHT) RemoveAuthHost 

func (dht *DHT) RemoveAuthHost(key string)

RemoveAuthHost removes a host from auth.

func (*DHT) RemoveAuthSentKeys 

func (dht *DHT) RemoveAuthSentKeys(targetID string)

RemoveAuthSentKeys removes a targetID from sent keys.

func (*DHT) RemovePossibleProxyID 

func (dht *DHT) RemovePossibleProxyID(id string)

RemovePossibleProxyID removes if from possible proxy ids list.

func (*DHT) RemoveProxyHost 

func (dht *DHT) RemoveProxyHost(targetID string)

RemoveProxyHost removes host from proxy list.

func (*DHT) RemoveRelayClient 

func (dht *DHT) RemoveRelayClient(host *host.Host) error

RemoveRelayClient removes a client from relay list.

func (*DHT) SendRequest 

func (dht *DHT) SendRequest(packet *packet.Packet) (transport.Future, error)

SendRequest sends a packet.

func (*DHT) SetAuthStatus 

func (dht *DHT) SetAuthStatus(targetID string, status bool)

SetAuthStatus sets a new auth status to targetID.

func (*DHT) SetHighKnownHostID 

func (dht *DHT) SetHighKnownHostID(id string)

SetHighKnownHostID sets a new high known host ID.

func (*DHT) SetNodeKeeper 

func (dht *DHT) SetNodeKeeper(keeper network.NodeKeeper)

func (*DHT) SetOuterHostsCount 

func (dht *DHT) SetOuterHostsCount(hosts int)

SetOuterHostsCount sets a new value to outer hosts count.

func (*DHT) StartAuthorize 

func (dht *DHT) StartAuthorize() error

StartAuthorize start authorize to discovery nodes.

func (*DHT) Store 

func (dht *DHT) Store(key store.Key, data []byte, replication time.Time, expiration time.Time, publisher bool) error

Store should store a key/value pair for the local host with the given replication and expiration times.

func (*DHT) StoreData 

func (dht *DHT) StoreData(ctx hosthandler.Context, data []byte) (id string, err error)

StoreData stores data on the network. This will trigger an iterateStore loop. The base58 encoded identifier will be returned if the store is successful.

func (*DHT) StoreRetrieve 

func (dht *DHT) StoreRetrieve(key store.Key) ([]byte, bool)

StoreRetrieve should return the local key/value if it exists.

type HighKnownOuterHostsHost 

type HighKnownOuterHostsHost struct {
	ID                  string
	OuterHosts          int // high known outer hosts by ID host
	SelfKnownOuterHosts int
}

HighKnownOuterHostsHost collects an information about host in home subnet which have a more known outer hosts.

type MockLedger 

type MockLedger struct {
	// contains filtered or unexported fields
}

func (*MockLedger) GetArtifactManager 

func (l *MockLedger) GetArtifactManager() core.ArtifactManager

GetArtifactManager returns artifact manager to work with.

func (*MockLedger) GetJetCoordinator 

func (l *MockLedger) GetJetCoordinator() core.JetCoordinator

GetJetCoordinator returns jet coordinator to work with.

func (*MockLedger) GetPulseManager 

func (l *MockLedger) GetPulseManager() core.PulseManager

GetPulseManager returns pulse manager to work with.

type MockPulseManager 

type MockPulseManager struct {
	// contains filtered or unexported fields
}

MockPulseManager mock struct to read and write pulse that implements core.PulseManager interface.

func (*MockPulseManager) Current 

func (pm *MockPulseManager) Current(context.Context) (*core.Pulse, error)

func (*MockPulseManager) Set 

func (pm *MockPulseManager) Set(ctx context.Context, pulse core.Pulse) error

func (*MockPulseManager) SetCallback 

func (pm *MockPulseManager) SetCallback(callback func(core.Pulse))

type Options 

type Options struct {
	// The hosts being used to bootstrap the network. Without a bootstrap
	// host there is no way to connect to the network. NetworkHosts can be
	// initialized via host.NewHost().
	BootstrapHosts []*host.Host

	// The time after which a key/value pair expires;
	// this is a time-to-live (TTL) from the original publication date.
	ExpirationTime time.Duration

	// Seconds after which an otherwise unaccessed bucket must be refreshed.
	RefreshTime time.Duration

	// The interval between Kademlia replication events, when a host is
	// required to publish its entire database.
	ReplicateTime time.Duration

	// The time after which the original publisher must
	// republish a key/value pair. Currently not implemented.
	RepublishTime time.Duration

	// The maximum time to wait for a response from a host before discarding
	// it from the bucket.
	PingTimeout time.Duration

	// The maximum time to wait for a response to any packet.
	PacketTimeout time.Duration
}

Options contains configuration options for the local host.

type Subnet 

type Subnet struct {
	SubnetIDs        map[string][]string // key - ip, value - id
	HomeSubnetKey    string              // key of home subnet fo SubnetIDs
	PossibleRelayIDs []string
	PossibleProxyIDs []string
	HighKnownHosts   HighKnownOuterHostsHost
}

Subnet collects some information about self network part

type Wrapper 

type Wrapper struct {
	HostNetwork hosthandler.HostHandler
}

func (*Wrapper) AnalyzeNetwork 

func (w *Wrapper) AnalyzeNetwork() error

AnalyzeNetwork legacy method for old DHT network (should be removed in

func (*Wrapper) Authorize 

func (w *Wrapper) Authorize() error

Authorize start authorization process on discovery node.

func (*Wrapper) Bootstrap 

func (w *Wrapper) Bootstrap() error

Bootstrap init bootstrap process: 1. Connect to discovery node; 2. Reconnect to new discovery node if redirected.

func (*Wrapper) BuildResponse 

func (w *Wrapper) BuildResponse(request network.Request, responseData interface{}) network.Response

func (*Wrapper) GetConsensus 

func (w *Wrapper) GetConsensus() consensus.Processor

func (*Wrapper) GetNodeID 

func (w *Wrapper) GetNodeID() core.RecordRef

GetNodeID get self node id (should be removed in far future)

func (*Wrapper) Inject 

func (w *Wrapper) Inject(components core.Components)

Inject inject components

func (*Wrapper) NewRequestBuilder 

func (w *Wrapper) NewRequestBuilder() network.RequestBuilder

NewRequestBuilder create packet builder for an outgoing request with sender set to current node.

func (*Wrapper) PublicAddress 

func (w *Wrapper) PublicAddress() string

PublicAddress returns public address that can be published for all nodes.

func (*Wrapper) RegisterRequestHandler 

func (w *Wrapper) RegisterRequestHandler(t types.PacketType, handler network.RequestHandler)

RegisterRequestHandler register a handler function to process incoming requests of a specific type.

func (*Wrapper) RemoteProcedureRegister 

func (w *Wrapper) RemoteProcedureRegister(name string, method core.RemoteProcedure)

RemoteProcedureRegister register remote procedure that will be executed when message is received

func (*Wrapper) ResendPulseToKnownHosts 

func (w *Wrapper) ResendPulseToKnownHosts(pulse core.Pulse)

ResendPulseToKnownHosts resend pulse when we receive pulse from pulsar daemon

func (*Wrapper) SendCascadeMessage 

func (w *Wrapper) SendCascadeMessage(data core.Cascade, method string, msg core.SignedMessage) error

func (*Wrapper) SendMessage 

func (w *Wrapper) SendMessage(nodeID core.RecordRef, method string, msg core.SignedMessage) ([]byte, error)

SendMessage send message to nodeID

func (*Wrapper) SendRequest 

func (w *Wrapper) SendRequest(network.Request, core.RecordRef) (network.Future, error)

SendRequest send request to a remote node.

func (*Wrapper) Start 

func (w *Wrapper) Start()

Start listening to network requests.

func (*Wrapper) Stop 

func (w *Wrapper) Stop()

Disconnect stop listening to network requests.

Source Files

View all Source files

Directories

Path Synopsis
Package routing implements Kademlia hash tables with XOR distance metrics.
 Package routing implements Kademlia hash tables with XOR distance metrics.
Package rpc allows higher level components to register methods that can be called by other network hosts.
 Package rpc allows higher level components to register methods that can be called by other network hosts.
Package store provides interfaces and default in-memory implementation of storage for DHT metadata.
 Package store provides interfaces and default in-memory implementation of storage for DHT metadata.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.