dht

package module
v0.7.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 13, 2020 License: MIT Imports: 52 Imported by: 0

README

go-libp2p-kad-dht

GoDoc Build Status Discourse posts

A Kademlia DHT implementation on go-libp2p

Table of Contents

Install

go get github.com/libp2p/go-libp2p-kad-dht

Usage

Go to https://godoc.org/github.com/libp2p/go-libp2p-kad-dht.

Contribute

Contributions welcome. Please check out the issues.

Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to libp2p are subject to the IPFS Code of Conduct.

Small note: If editing the README, please conform to the standard-readme specification.

License

MIT © Protocol Labs Inc.

The last gx published version of this module was: 4.4.34: QmXuNFLZc6Nb5akB4sZsxK3doShsFKT1sZFvxLXJvZQwAW

Documentation

Overview

Package dht implements a distributed hash table that satisfies the ipfs routing interface. This DHT is modeled after kademlia with S/Kademlia modifications.

Index

Constants

View Source 
const BaseConnMgrScore = 5

BaseConnMgrScore is the base of the score set on the connection manager "kbucket" tag. It is added with the common prefix length between two peer IDs.

View Source 
const DefaultPrefix protocol.ID = "/ipfs"

DefaultPrefix is the application specific prefix attached to all DHT protocols by default.

Variables

View Source 
var (
	// ProtocolDHT is the default DHT protocol.
	ProtocolDHT protocol.ID = "/ipfs/kad/1.0.0"
	// DefaultProtocols spoken by the DHT.
	DefaultProtocols = []protocol.ID{ProtocolDHT}
)
View Source 
var DefaultBootstrapPeers []multiaddr.Multiaddr

DefaultBootstrapPeers is a set of public DHT bootstrap peers provided by libp2p.

View Source 
var ErrNoPeersQueried = errors.New("failed to query any peers")

ErrNoPeersQueried is returned when we failed to connect to any peers.

View Source 
var ErrReadTimeout = fmt.Errorf("timed out reading response")

ErrReadTimeout is an error that occurs when no message is read within the timeout period.

View Source 
var LookupEventBufferSize = 16

LookupEventBufferSize is the number of events to buffer.

Functions

func PrivateQueryFilter added in v0.7.11 

func PrivateQueryFilter(dht *IpfsDHT, ai peer.AddrInfo) bool

PrivateQueryFilter doens't currently restrict which peers we are willing to query from the local DHT.

func PrivateRoutingTableFilter added in v0.7.11 

func PrivateRoutingTableFilter(dht *IpfsDHT, conns []network.Conn) bool

PrivateRoutingTableFilter allows a peer to be added to the routing table if the connections to that peer indicate that it is on a private network

func PublicQueryFilter added in v0.7.11 

func PublicQueryFilter(_ *IpfsDHT, ai peer.AddrInfo) bool

PublicQueryFilter returns true if the peer is suspected of being publicly accessible

func PublicRoutingTableFilter added in v0.7.11 

func PublicRoutingTableFilter(dht *IpfsDHT, conns []network.Conn) bool

PublicRoutingTableFilter allows a peer to be added to the routing table if the connections to that peer indicate that it is on a public network

func PublishLookupEvent added in v0.7.11 

func PublishLookupEvent(ctx context.Context, ev *LookupEvent)

PublishLookupEvent publishes a query event to the query event channel associated with the given context, if any.

func Quorum 

func Quorum(n int) routing.Option

Quorum is a DHT option that tells the DHT how many peers it needs to get values from before returning the best one. Zero means the DHT query should complete instead of returning early.

Default: 0

func RegisterForLookupEvents added in v0.7.11 

func RegisterForLookupEvents(ctx context.Context) (context.Context, <-chan *LookupEvent)

RegisterForLookupEvents registers a lookup event channel with the given context. The returned context can be passed to DHT queries to receive lookup events on the returned channels.

The passed context MUST be canceled when the caller is no longer interested in query events.

func WriteMsg 

func WriteMsg(w io.Writer, mes *pb.Message) error

Types

type IpfsDHT 

type IpfsDHT struct {

	// ProviderManager stores & manages the provider records for this Dht peer.
	ProviderManager *providers.ProviderManager

	Validator record.Validator
	// contains filtered or unexported fields
}

IpfsDHT is an implementation of Kademlia with S/Kademlia modifications. It is used to implement the base Routing module.

func New 

func New(ctx context.Context, h host.Host, options ...Option) (*IpfsDHT, error)

New creates a new DHT with the specified host and options. Please note that being connected to a DHT peer does not necessarily imply that it's also in the DHT Routing Table. If the Routing Table has more than "minRTRefreshThreshold" peers, we consider a peer as a Routing Table candidate ONLY when we successfully get a query response from it OR if it send us a query.

func NewDHT 

func NewDHT(ctx context.Context, h host.Host, dstore ds.Batching) *IpfsDHT

NewDHT creates a new DHT object with the given peer as the 'local' host. IpfsDHT's initialized with this function will respond to DHT requests, whereas IpfsDHT's initialized with NewDHTClient will not.

func NewDHTClient 

func NewDHTClient(ctx context.Context, h host.Host, dstore ds.Batching) *IpfsDHT

NewDHTClient creates a new DHT object with the given peer as the 'local' host. IpfsDHT clients initialized with this function will not respond to DHT requests. If you need a peer to respond to DHT requests, use NewDHT instead. NewDHTClient creates a new DHT object with the given peer as the 'local' host

func (*IpfsDHT) Bootstrap 

func (dht *IpfsDHT) Bootstrap(_ context.Context) error

Bootstrap tells the DHT to get into a bootstrapped state satisfying the IpfsRouter interface.

func (*IpfsDHT) Close 

func (dht *IpfsDHT) Close() error

Close calls Process Close.

func (*IpfsDHT) Context 

func (dht *IpfsDHT) Context() context.Context

Context returns the DHT's context.

func (*IpfsDHT) FindLocal 

func (dht *IpfsDHT) FindLocal(id peer.ID) peer.AddrInfo

FindLocal looks for a peer with a given ID connected to this dht and returns the peer and the table it was found in.

func (*IpfsDHT) FindPeer 

func (dht *IpfsDHT) FindPeer(ctx context.Context, id peer.ID) (_ peer.AddrInfo, err error)

FindPeer searches for a peer with given ID.

func (*IpfsDHT) FindProviders 

func (dht *IpfsDHT) FindProviders(ctx context.Context, c cid.Cid) ([]peer.AddrInfo, error)

FindProviders searches until the context expires.

func (*IpfsDHT) FindProvidersAsync 

func (dht *IpfsDHT) FindProvidersAsync(ctx context.Context, key cid.Cid, count int) <-chan peer.AddrInfo

FindProvidersAsync is the same thing as FindProviders, but returns a channel. Peers will be returned on the channel as soon as they are found, even before the search query completes. If count is zero then the query will run until it completes. Note: not reading from the returned channel may block the query from progressing.

func (*IpfsDHT) GetClosestPeers 

func (dht *IpfsDHT) GetClosestPeers(ctx context.Context, key string) (<-chan peer.ID, error)

GetClosestPeers is a Kademlia 'node lookup' operation. Returns a channel of the K closest peers to the given key.

If the context is canceled, this function will return the context error along with the closest K peers it has found so far.

func (*IpfsDHT) GetPublicKey 

func (dht *IpfsDHT) GetPublicKey(ctx context.Context, p peer.ID) (ci.PubKey, error)

GetPublicKey gets the public key when given a Peer ID. It will extract from the Peer ID if inlined or ask the node it belongs to or ask the DHT.

func (*IpfsDHT) GetValue 

func (dht *IpfsDHT) GetValue(ctx context.Context, key string, opts ...routing.Option) (_ []byte, err error)

GetValue searches for the value corresponding to given Key.

func (*IpfsDHT) GetValues 

func (dht *IpfsDHT) GetValues(ctx context.Context, key string, nvals int) (_ []RecvdVal, err error)

GetValues gets nvals values corresponding to the given key.

func (*IpfsDHT) Host 

func (dht *IpfsDHT) Host() host.Host

Host returns the libp2p host this DHT is operating with.

func (*IpfsDHT) Mode added in v0.7.11 

func (dht *IpfsDHT) Mode() ModeOpt

Mode allows introspection of the operation mode of the DHT

func (*IpfsDHT) PeerID 

func (dht *IpfsDHT) PeerID() peer.ID

PeerID returns the DHT node's Peer ID.

func (*IpfsDHT) PeerKey 

func (dht *IpfsDHT) PeerKey() []byte

PeerKey returns a DHT key, converted from the DHT node's Peer ID.

func (*IpfsDHT) Ping 

func (dht *IpfsDHT) Ping(ctx context.Context, p peer.ID) error

Ping sends a ping message to the passed peer and waits for a response.

func (*IpfsDHT) Process 

func (dht *IpfsDHT) Process() goprocess.Process

Process returns the DHT's process.

func (*IpfsDHT) Provide 

func (dht *IpfsDHT) Provide(ctx context.Context, key cid.Cid, brdcst bool) (err error)

Provide makes this node announce that it can provide a value for the given key

func (*IpfsDHT) PutValue 

func (dht *IpfsDHT) PutValue(ctx context.Context, key string, value []byte, opts ...routing.Option) (err error)

PutValue adds value corresponding to given Key. This is the top level "Store" operation of the DHT

func (*IpfsDHT) RefreshRoutingTable added in v0.7.11 

func (dht *IpfsDHT) RefreshRoutingTable() <-chan error

RefreshRoutingTable tells the DHT to refresh it's routing tables.

The returned channel will block until the refresh finishes, then yield the error and close. The channel is buffered and safe to ignore.

func (*IpfsDHT) RoutingTable 

func (dht *IpfsDHT) RoutingTable() *kb.RoutingTable

RoutingTable returns the DHT's routingTable.

func (*IpfsDHT) SearchValue 

func (dht *IpfsDHT) SearchValue(ctx context.Context, key string, opts ...routing.Option) (<-chan []byte, error)

SearchValue searches for the value corresponding to given Key and streams the results.

type KeyKadID added in v0.7.11 

type KeyKadID struct {
	Key string
	Kad kbucket.ID
}

KeyKadID contains the Kademlia key in string and binary form.

func NewKeyKadID added in v0.7.11 

func NewKeyKadID(k string) *KeyKadID

NewKeyKadID creates a KeyKadID from a string Kademlia ID.

type LookupEvent added in v0.7.11 

type LookupEvent struct {
	// Node is the ID of the node performing the lookup.
	Node *PeerKadID
	// ID is a unique identifier for the lookup instance.
	ID uuid.UUID
	// Key is the Kademlia key used as a lookup target.
	Key *KeyKadID
	// Request, if not nil, describes a state update event, associated with an outgoing query request.
	Request *LookupUpdateEvent
	// Response, if not nil, describes a state update event, associated with an outgoing query response.
	Response *LookupUpdateEvent
	// Terminate, if not nil, describe a termination event.
	Terminate *LookupTerminateEvent
}

LookupEvent is emitted for every notable event that happens during a DHT lookup. LookupEvent supports JSON marshalling because all of its fields do, recursively.

func NewLookupEvent added in v0.7.11 

func NewLookupEvent(
	node peer.ID,
	id uuid.UUID,
	key string,
	request *LookupUpdateEvent,
	response *LookupUpdateEvent,
	terminate *LookupTerminateEvent,
) *LookupEvent

NewLookupEvent creates a LookupEvent automatically converting the node libp2p Peer ID to a PeerKadID and the string Kademlia key to a KeyKadID.

type LookupTerminateEvent added in v0.7.11 

type LookupTerminateEvent struct {
	// Reason is the reason for lookup termination.
	Reason LookupTerminationReason
}

LookupTerminateEvent describes a lookup termination event.

func NewLookupTerminateEvent added in v0.7.11 

func NewLookupTerminateEvent(reason LookupTerminationReason) *LookupTerminateEvent

NewLookupTerminateEvent creates a new lookup termination event with a given reason.

type LookupTerminationReason added in v0.7.11 

type LookupTerminationReason int

LookupTerminationReason captures reasons for terminating a lookup. 

const (
	// LookupStopped indicates that the lookup was aborted by the user's stopFn.
	LookupStopped LookupTerminationReason = iota
	// LookupCancelled indicates that the lookup was aborted by the context.
	LookupCancelled
	// LookupStarvation indicates that the lookup terminated due to lack of unqueried peers.
	LookupStarvation
	// LookupCompleted indicates that the lookup terminated successfully, reaching the Kademlia end condition.
	LookupCompleted
)

func (LookupTerminationReason) MarshalJSON added in v0.7.11 

func (r LookupTerminationReason) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON encoding of the passed lookup termination reason.

func (LookupTerminationReason) String added in v0.7.11 

func (r LookupTerminationReason) String() string

type LookupUpdateEvent added in v0.7.11 

type LookupUpdateEvent struct {
	// Cause is the peer whose response (or lack of response) caused the update event.
	// If Cause is nil, this is the first update event in the lookup, caused by the seeding.
	Cause *PeerKadID
	// Source is the peer who informed us about the peer IDs in this update (below).
	Source *PeerKadID
	// Heard is a set of peers whose state in the lookup's peerset is being set to "heard".
	Heard []*PeerKadID
	// Waiting is a set of peers whose state in the lookup's peerset is being set to "waiting".
	Waiting []*PeerKadID
	// Queried is a set of peers whose state in the lookup's peerset is being set to "queried".
	Queried []*PeerKadID
	// Unreachable is a set of peers whose state in the lookup's peerset is being set to "unreachable".
	Unreachable []*PeerKadID
}

LookupUpdateEvent describes a lookup state update event.

func NewLookupUpdateEvent added in v0.7.11 

func NewLookupUpdateEvent(
	cause peer.ID,
	source peer.ID,
	heard []peer.ID,
	waiting []peer.ID,
	queried []peer.ID,
	unreachable []peer.ID,
) *LookupUpdateEvent

NewLookupUpdateEvent creates a new lookup update event, automatically converting the passed peer IDs to peer Kad IDs.

type ModeOpt added in v0.7.11 

type ModeOpt int

ModeOpt describes what mode the dht should operate in 

const (
	// ModeAuto utilizes EvtLocalReachabilityChanged events sent over the event bus to dynamically switch the DHT
	// between Client and Server modes based on network conditions
	ModeAuto ModeOpt = iota
	// ModeClient operates the DHT as a client only, it cannot respond to incoming queries
	ModeClient
	// ModeServer operates the DHT as a server, it can both send and respond to queries
	ModeServer
	// ModeAutoServer operates in the same way as ModeAuto, but acts as a server when reachability is unknown
	ModeAutoServer
)

type Option added in v0.7.11 

type Option func(*config) error

Option DHT option type.

func BucketSize added in v0.7.11 

func BucketSize(bucketSize int) Option

BucketSize configures the bucket size (k in the Kademlia paper) of the routing table.

The default value is 20.

func Concurrency added in v0.7.11 

func Concurrency(alpha int) Option

Concurrency configures the number of concurrent requests (alpha in the Kademlia paper) for a given query path.

The default value is 10.

func Datastore added in v0.7.11 

func Datastore(ds ds.Batching) Option

Datastore configures the DHT to use the specified datastore.

Defaults to an in-memory (temporary) map.

func DisableAutoRefresh added in v0.7.11 

func DisableAutoRefresh() Option

DisableAutoRefresh completely disables 'auto-refresh' on the DHT routing table. This means that we will neither refresh the routing table periodically nor when the routing table size goes below the minimum threshold.

func DisableProviders added in v0.7.11 

func DisableProviders() Option

DisableProviders disables storing and retrieving provider records.

Defaults to enabled.

WARNING: do not change this unless you're using a forked DHT (i.e., a private network and/or distinct DHT protocols with the `Protocols` option).

func DisableValues added in v0.7.11 

func DisableValues() Option

DisableValues disables storing and retrieving value records (including public keys).

Defaults to enabled.

WARNING: do not change this unless you're using a forked DHT (i.e., a private network and/or distinct DHT protocols with the `Protocols` option).

func MaxRecordAge 

func MaxRecordAge(maxAge time.Duration) Option

MaxRecordAge specifies the maximum time that any node will hold onto a record ("PutValue record") from the time its received. This does not apply to any other forms of validity that the record may contain. For example, a record may contain an ipns entry with an EOL saying its valid until the year 2020 (a great time in the future). For that record to stick around it must be rebroadcasted more frequently than once every 'MaxRecordAge'

func Mode added in v0.7.11 

func Mode(m ModeOpt) Option

Mode configures which mode the DHT operates in (Client, Server, Auto).

Defaults to ModeAuto.

func NamespacedValidator added in v0.7.11 

func NamespacedValidator(ns string, v record.Validator) Option

NamespacedValidator adds a validator namespaced under `ns`. This option fails if the DHT is not using a `record.NamespacedValidator` as its validator (it uses one by default but this can be overridden with the `Validator` option). Adding a namespaced validator without changing the `Validator` will result in adding a new validator in addition to the default public key and IPNS validators. The "pk" and "ipns" namespaces cannot be overridden here unless a new `Validator` has been set first.

Example: Given a validator registered as `NamespacedValidator("ipns", myValidator)`, all records with keys starting with `/ipns/` will be validated with `myValidator`.

func ProtocolExtension added in v0.7.11 

func ProtocolExtension(ext protocol.ID) Option

ProtocolExtension adds an application specific protocol to the DHT protocol. For example, /ipfs/lan/kad/1.0.0 instead of /ipfs/kad/1.0.0. extension should be of the form /lan.

func ProtocolPrefix added in v0.7.11 

func ProtocolPrefix(prefix protocol.ID) Option

ProtocolPrefix sets an application specific prefix to be attached to all DHT protocols. For example, /myapp/kad/1.0.0 instead of /ipfs/kad/1.0.0. Prefix should be of the form /myapp.

Defaults to dht.DefaultPrefix

func ProvidersOptions added in v0.7.11 

func ProvidersOptions(opts []providers.Option) Option

ProvidersOptions are options passed directly to the provider manager.

The provider manager adds and gets provider records from the datastore, cahing them in between. These options are passed to the provider manager allowing customisation of things like the GC interval and cache implementation.

func QueryFilter added in v0.7.11 

func QueryFilter(filter QueryFilterFunc) Option

QueryFilter sets a function that approves which peers may be dialed in a query

func Resiliency added in v0.7.11 

func Resiliency(beta int) Option

Resiliency configures the number of peers closest to a target that must have responded in order for a given query path to complete.

The default value is 3.

func RoutingTableFilter added in v0.7.11 

func RoutingTableFilter(filter RouteTableFilterFunc) Option

RoutingTableFilter sets a function that approves which peers may be added to the routing table. The host should already have at least one connection to the peer under consideration.

func RoutingTableLatencyTolerance added in v0.7.11 

func RoutingTableLatencyTolerance(latency time.Duration) Option

RoutingTableLatencyTolerance sets the maximum acceptable latency for peers in the routing table's cluster.

func RoutingTableRefreshPeriod added in v0.7.11 

func RoutingTableRefreshPeriod(period time.Duration) Option

RoutingTableRefreshPeriod sets the period for refreshing buckets in the routing table. The DHT will refresh buckets every period by:

  1. First searching for nearby peers to figure out how many buckets we should try to fill.
  2. Then searching for a random key in each bucket that hasn't been queried in the last refresh period.

func RoutingTableRefreshQueryTimeout added in v0.7.11 

func RoutingTableRefreshQueryTimeout(timeout time.Duration) Option

RoutingTableRefreshQueryTimeout sets the timeout for routing table refresh queries.

func V1CompatibleMode added in v0.7.11 

func V1CompatibleMode(enable bool) Option

V1CompatibleMode sets the DHT to operate in V1 compatible mode. In this mode, the DHT node will act like a V1 DHT node (use the V1 protocol names) but will use the V2 query and routing table logic.

For now, this option defaults to true for backwards compatibility. In the near future, it will switch to false.

This option is perma-unstable and may be removed in the future.

func Validator added in v0.7.11 

func Validator(v record.Validator) Option

Validator configures the DHT to use the specified validator.

Defaults to a namespaced validator that can validate both public key (under the "pk" namespace) and IPNS records (under the "ipns" namespace). Setting the validator implies that the user wants to control the validators and therefore the default public key and IPNS validators will not be added.

type PeerKadID added in v0.7.11 

type PeerKadID struct {
	Peer peer.ID
	Kad  kbucket.ID
}

PeerKadID contains a libp2p Peer ID and a binary Kademlia ID.

func NewPeerKadID added in v0.7.11 

func NewPeerKadID(p peer.ID) *PeerKadID

NewPeerKadID creates a PeerKadID from a libp2p Peer ID.

func NewPeerKadIDSlice added in v0.7.11 

func NewPeerKadIDSlice(p []peer.ID) []*PeerKadID

NewPeerKadIDSlice creates a slice of PeerKadID from the passed slice of libp2p Peer IDs.

func OptPeerKadID added in v0.7.11 

func OptPeerKadID(p peer.ID) *PeerKadID

OptPeerKadID returns a pointer to a PeerKadID or nil if the passed Peer ID is it's default value.

type QueryFilterFunc added in v0.7.11 

type QueryFilterFunc func(dht *IpfsDHT, ai peer.AddrInfo) bool

QueryFilterFunc is a filter applied when considering peers to dial when querying

type RecvdVal 

type RecvdVal struct {
	Val  []byte
	From peer.ID
}

RecvdVal stores a value and the peer from which we got the value.

type RouteTableFilterFunc added in v0.7.11 

type RouteTableFilterFunc func(dht *IpfsDHT, conns []network.Conn) bool

RouteTableFilterFunc is a filter applied when considering connections to keep in the local route table.

Source Files

View all Source files

Directories

Path Synopsis
Package dual provides an implementaiton of a split or "dual" dht, where two parallel instances are maintained for the global internet and the local LAN respectively.
 Package dual provides an implementaiton of a split or "dual" dht, where two parallel instances are maintained for the global internet and the local LAN respectively.
internal
testing

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.