dht

package module
v0.0.0-...-0765461 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 1, 2023 License: MIT Imports: 58 Imported by: 0

README

go-libp2p-kad-dht

GoDoc Discourse posts

A Go implementation of libp2p Kademlia DHT specification

Table of Contents

Install

go get github.com/mikelsr/go-libp2p-kad-dht

Optimizations

Client-side optimizations are described in optimizations.md

Usage

Go to https://godoc.org/github.com/mikelsr/go-libp2p-kad-dht.

Contribute

Contributions welcome. Please check out the issues.

Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to libp2p are subject to the IPFS Code of Conduct.

Maintainers

See CODEOWNERS.

License

MIT © Protocol Labs Inc.

Documentation

Overview

Package dht implements a distributed hash table that satisfies the ipfs routing interface. This DHT is modeled after kademlia with S/Kademlia modifications.

Index

Constants

View Source 
const DefaultPrefix protocol.ID = "/ipfs"

DefaultPrefix is the application specific prefix attached to all DHT protocols by default.

Variables

View Source 
var (
	// ProtocolDHT is the default DHT protocol.
	ProtocolDHT protocol.ID = "/ipfs/kad/1.0.0"
	// DefaultProtocols spoken by the DHT.
	DefaultProtocols = []protocol.ID{ProtocolDHT}
)
View Source 
var DefaultBootstrapPeers []multiaddr.Multiaddr

DefaultBootstrapPeers is a set of public DHT bootstrap peers provided by libp2p.

View Source 
var ErrNoPeersQueried = errors.New("failed to query any peers")

ErrNoPeersQueried is returned when we failed to connect to any peers.

View Source 
var ErrReadTimeout = net.ErrReadTimeout

ErrReadTimeout is an error that occurs when no message is read within the timeout period.

View Source 
var LookupEventBufferSize = 16

LookupEventBufferSize is the number of events to buffer.

Functions

func GetDefaultBootstrapPeerAddrInfos 

func GetDefaultBootstrapPeerAddrInfos() []peer.AddrInfo

GetDefaultBootstrapPeerAddrInfos returns the peer.AddrInfos for the default bootstrap peers so we can use these for initializing the DHT by passing these to the BootstrapPeers(...) option.

func NewRTPeerDiversityFilter 

func NewRTPeerDiversityFilter(h host.Host, maxPerCpl, maxForTable int) *rtPeerIPGroupFilter

NewRTPeerDiversityFilter constructs the `PeerIPGroupFilter` that will be used to configure the diversity filter for the Routing Table. Please see the docs for `peerdiversity.PeerIPGroupFilter` AND `peerdiversity.Filter` for more details.

func PrivateQueryFilter 

func PrivateQueryFilter(_ interface{}, ai peer.AddrInfo) bool

PrivateQueryFilter doens't currently restrict which peers we are willing to query from the local DHT.

func PrivateRoutingTableFilter 

func PrivateRoutingTableFilter(dht interface{}, p peer.ID) bool

PrivateRoutingTableFilter allows a peer to be added to the routing table if the connections to that peer indicate that it is on a private network

func PublicQueryFilter 

func PublicQueryFilter(_ interface{}, ai peer.AddrInfo) bool

PublicQueryFilter returns true if the peer is suspected of being publicly accessible

func PublicRoutingTableFilter 

func PublicRoutingTableFilter(dht interface{}, p peer.ID) bool

PublicRoutingTableFilter allows a peer to be added to the routing table if the connections to that peer indicate that it is on a public network

func PublishLookupEvent 

func PublishLookupEvent(ctx context.Context, ev *LookupEvent)

PublishLookupEvent publishes a query event to the query event channel associated with the given context, if any.

func Quorum 

func Quorum(n int) routing.Option

Quorum is a DHT option that tells the DHT how many peers it needs to get values from before returning the best one. Zero means the DHT query should complete instead of returning early.

Default: 0

func RegisterForLookupEvents 

func RegisterForLookupEvents(ctx context.Context) (context.Context, <-chan *LookupEvent)

RegisterForLookupEvents registers a lookup event channel with the given context. The returned context can be passed to DHT queries to receive lookup events on the returned channels.

The passed context MUST be canceled when the caller is no longer interested in query events.

Types

type IpfsDHT 

type IpfsDHT struct {
	Validator record.Validator
	// contains filtered or unexported fields
}

IpfsDHT is an implementation of Kademlia with S/Kademlia modifications. It is used to implement the base Routing module.

func New 

func New(ctx context.Context, h host.Host, options ...Option) (*IpfsDHT, error)

New creates a new DHT with the specified host and options. Please note that being connected to a DHT peer does not necessarily imply that it's also in the DHT Routing Table. If the Routing Table has more than "minRTRefreshThreshold" peers, we consider a peer as a Routing Table candidate ONLY when we successfully get a query response from it OR if it send us a query.

func NewDHT 

func NewDHT(ctx context.Context, h host.Host, dstore ds.Batching) *IpfsDHT

NewDHT creates a new DHT object with the given peer as the 'local' host. IpfsDHT's initialized with this function will respond to DHT requests, whereas IpfsDHT's initialized with NewDHTClient will not.

func NewDHTClient 

func NewDHTClient(ctx context.Context, h host.Host, dstore ds.Batching) *IpfsDHT

NewDHTClient creates a new DHT object with the given peer as the 'local' host. IpfsDHT clients initialized with this function will not respond to DHT requests. If you need a peer to respond to DHT requests, use NewDHT instead.

func (*IpfsDHT) Bootstrap 

func (dht *IpfsDHT) Bootstrap(ctx context.Context) error

Bootstrap tells the DHT to get into a bootstrapped state satisfying the IpfsRouter interface.

func (*IpfsDHT) Close 

func (dht *IpfsDHT) Close() error

Close calls Process Close.

func (*IpfsDHT) Context 

func (dht *IpfsDHT) Context() context.Context

Context returns the DHT's context.

func (*IpfsDHT) FindLocal 

func (dht *IpfsDHT) FindLocal(ctx context.Context, id peer.ID) peer.AddrInfo

FindLocal looks for a peer with a given ID connected to this dht and returns the peer and the table it was found in.

func (*IpfsDHT) FindPeer 

func (dht *IpfsDHT) FindPeer(ctx context.Context, id peer.ID) (_ peer.AddrInfo, err error)

FindPeer searches for a peer with given ID.

func (*IpfsDHT) FindProviders 

func (dht *IpfsDHT) FindProviders(ctx context.Context, c cid.Cid) ([]peer.AddrInfo, error)

FindProviders searches until the context expires.

func (*IpfsDHT) FindProvidersAsync 

func (dht *IpfsDHT) FindProvidersAsync(ctx context.Context, key cid.Cid, count int) <-chan peer.AddrInfo

FindProvidersAsync is the same thing as FindProviders, but returns a channel. Peers will be returned on the channel as soon as they are found, even before the search query completes. If count is zero then the query will run until it completes. Note: not reading from the returned channel may block the query from progressing.

func (*IpfsDHT) ForceRefresh 

func (dht *IpfsDHT) ForceRefresh() <-chan error

ForceRefresh acts like RefreshRoutingTable but forces the DHT to refresh all buckets in the Routing Table irrespective of when they were last refreshed.

The returned channel will block until the refresh finishes, then yield the error and close. The channel is buffered and safe to ignore.

func (*IpfsDHT) GetClosestPeers 

func (dht *IpfsDHT) GetClosestPeers(ctx context.Context, key string) ([]peer.ID, error)

GetClosestPeers is a Kademlia 'node lookup' operation. Returns a channel of the K closest peers to the given key.

If the context is canceled, this function will return the context error along with the closest K peers it has found so far.

func (*IpfsDHT) GetPublicKey 

func (dht *IpfsDHT) GetPublicKey(ctx context.Context, p peer.ID) (ci.PubKey, error)

GetPublicKey gets the public key when given a Peer ID. It will extract from the Peer ID if inlined or ask the node it belongs to or ask the DHT.

func (*IpfsDHT) GetRoutingTableDiversityStats 

func (dht *IpfsDHT) GetRoutingTableDiversityStats() []peerdiversity.CplDiversityStats

GetRoutingTableDiversityStats returns the diversity stats for the Routing Table.

func (*IpfsDHT) GetValue 

func (dht *IpfsDHT) GetValue(ctx context.Context, key string, opts ...routing.Option) (_ []byte, err error)

GetValue searches for the value corresponding to given Key.

func (*IpfsDHT) Host 

func (dht *IpfsDHT) Host() host.Host

Host returns the libp2p host this DHT is operating with.

func (*IpfsDHT) Mode 

func (dht *IpfsDHT) Mode() ModeOpt

Mode allows introspection of the operation mode of the DHT

func (*IpfsDHT) NetworkSize 

func (dht *IpfsDHT) NetworkSize() (int32, error)

NetworkSize returns the most recent estimation of the DHT network size. EXPERIMENTAL: We do not provide any guarantees that this method will continue to exist in the codebase. Use it at your own risk.

func (*IpfsDHT) PeerID 

func (dht *IpfsDHT) PeerID() peer.ID

PeerID returns the DHT node's Peer ID.

func (*IpfsDHT) PeerKey 

func (dht *IpfsDHT) PeerKey() []byte

PeerKey returns a DHT key, converted from the DHT node's Peer ID.

func (*IpfsDHT) Ping 

func (dht *IpfsDHT) Ping(ctx context.Context, p peer.ID) error

Ping sends a ping message to the passed peer and waits for a response.

func (*IpfsDHT) Provide 

func (dht *IpfsDHT) Provide(ctx context.Context, key cid.Cid, brdcst bool) (err error)

Provide makes this node announce that it can provide a value for the given key

func (*IpfsDHT) ProviderStore 

func (dht *IpfsDHT) ProviderStore() providers.ProviderStore

ProviderStore returns the provider storage object for storing and retrieving provider records.

func (*IpfsDHT) PutValue 

func (dht *IpfsDHT) PutValue(ctx context.Context, key string, value []byte, opts ...routing.Option) (err error)

PutValue adds value corresponding to given Key. This is the top level "Store" operation of the DHT

func (*IpfsDHT) RefreshRoutingTable 

func (dht *IpfsDHT) RefreshRoutingTable() <-chan error

RefreshRoutingTable tells the DHT to refresh it's routing tables.

The returned channel will block until the refresh finishes, then yield the error and close. The channel is buffered and safe to ignore.

func (*IpfsDHT) RoutingTable 

func (dht *IpfsDHT) RoutingTable() *kb.RoutingTable

RoutingTable returns the DHT's routingTable.

func (*IpfsDHT) SearchValue 

func (dht *IpfsDHT) SearchValue(ctx context.Context, key string, opts ...routing.Option) (<-chan []byte, error)

SearchValue searches for the value corresponding to given Key and streams the results.

type KeyKadID 

type KeyKadID struct {
	Key string
	Kad kbucket.ID
}

KeyKadID contains the Kademlia key in string and binary form.

func NewKeyKadID 

func NewKeyKadID(k string) *KeyKadID

NewKeyKadID creates a KeyKadID from a string Kademlia ID.

type LookupEvent 

type LookupEvent struct {
	// Node is the ID of the node performing the lookup.
	Node *PeerKadID
	// ID is a unique identifier for the lookup instance.
	ID uuid.UUID
	// Key is the Kademlia key used as a lookup target.
	Key *KeyKadID
	// Request, if not nil, describes a state update event, associated with an outgoing query request.
	Request *LookupUpdateEvent
	// Response, if not nil, describes a state update event, associated with an outgoing query response.
	Response *LookupUpdateEvent
	// Terminate, if not nil, describe a termination event.
	Terminate *LookupTerminateEvent
}

LookupEvent is emitted for every notable event that happens during a DHT lookup. LookupEvent supports JSON marshalling because all of its fields do, recursively.

func NewLookupEvent 

func NewLookupEvent(
	node peer.ID,
	id uuid.UUID,
	key string,
	request *LookupUpdateEvent,
	response *LookupUpdateEvent,
	terminate *LookupTerminateEvent,
) *LookupEvent

NewLookupEvent creates a LookupEvent automatically converting the node libp2p Peer ID to a PeerKadID and the string Kademlia key to a KeyKadID.

type LookupTerminateEvent 

type LookupTerminateEvent struct {
	// Reason is the reason for lookup termination.
	Reason LookupTerminationReason
}

LookupTerminateEvent describes a lookup termination event.

func NewLookupTerminateEvent 

func NewLookupTerminateEvent(reason LookupTerminationReason) *LookupTerminateEvent

NewLookupTerminateEvent creates a new lookup termination event with a given reason.

type LookupTerminationReason 

type LookupTerminationReason int

LookupTerminationReason captures reasons for terminating a lookup. 

const (
	// LookupStopped indicates that the lookup was aborted by the user's stopFn.
	LookupStopped LookupTerminationReason = iota
	// LookupCancelled indicates that the lookup was aborted by the context.
	LookupCancelled
	// LookupStarvation indicates that the lookup terminated due to lack of unqueried peers.
	LookupStarvation
	// LookupCompleted indicates that the lookup terminated successfully, reaching the Kademlia end condition.
	LookupCompleted
)

func (LookupTerminationReason) MarshalJSON 

func (r LookupTerminationReason) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON encoding of the passed lookup termination reason.

func (LookupTerminationReason) String 

func (r LookupTerminationReason) String() string

type LookupUpdateEvent 

type LookupUpdateEvent struct {
	// Cause is the peer whose response (or lack of response) caused the update event.
	// If Cause is nil, this is the first update event in the lookup, caused by the seeding.
	Cause *PeerKadID
	// Source is the peer who informed us about the peer IDs in this update (below).
	Source *PeerKadID
	// Heard is a set of peers whose state in the lookup's peerset is being set to "heard".
	Heard []*PeerKadID
	// Waiting is a set of peers whose state in the lookup's peerset is being set to "waiting".
	Waiting []*PeerKadID
	// Queried is a set of peers whose state in the lookup's peerset is being set to "queried".
	Queried []*PeerKadID
	// Unreachable is a set of peers whose state in the lookup's peerset is being set to "unreachable".
	Unreachable []*PeerKadID
}

LookupUpdateEvent describes a lookup state update event.

func NewLookupUpdateEvent 

func NewLookupUpdateEvent(
	cause peer.ID,
	source peer.ID,
	heard []peer.ID,
	waiting []peer.ID,
	queried []peer.ID,
	unreachable []peer.ID,
) *LookupUpdateEvent

NewLookupUpdateEvent creates a new lookup update event, automatically converting the passed peer IDs to peer Kad IDs.

type ModeOpt 

type ModeOpt = dhtcfg.ModeOpt

ModeOpt describes what mode the dht should operate in 

const (
	// ModeAuto utilizes EvtLocalReachabilityChanged events sent over the event bus to dynamically switch the DHT
	// between Client and Server modes based on network conditions
	ModeAuto ModeOpt = iota
	// ModeClient operates the DHT as a client only, it cannot respond to incoming queries
	ModeClient
	// ModeServer operates the DHT as a server, it can both send and respond to queries
	ModeServer
	// ModeAutoServer operates in the same way as ModeAuto, but acts as a server when reachability is unknown
	ModeAutoServer
)

type Option 

type Option = dhtcfg.Option

func AddressFilter 

func AddressFilter(f func([]ma.Multiaddr) []ma.Multiaddr) Option

AddressFilter allows to configure the address filtering function. This function is run before addresses are added to the peerstore. It is most useful to avoid adding localhost / local addresses.

func BootstrapPeers 

func BootstrapPeers(bootstrappers ...peer.AddrInfo) Option

BootstrapPeers configures the bootstrapping nodes that we will connect to to seed and refresh our Routing Table if it becomes empty.

func BootstrapPeersFunc 

func BootstrapPeersFunc(getBootstrapPeers func() []peer.AddrInfo) Option

BootstrapPeersFunc configures the function that returns the bootstrapping nodes that we will connect to to seed and refresh our Routing Table if it becomes empty.

func BucketSize 

func BucketSize(bucketSize int) Option

BucketSize configures the bucket size (k in the Kademlia paper) of the routing table.

The default value is 20.

func Concurrency 

func Concurrency(alpha int) Option

Concurrency configures the number of concurrent requests (alpha in the Kademlia paper) for a given query path.

The default value is 10.

func Datastore 

func Datastore(ds ds.Batching) Option

Datastore configures the DHT to use the specified datastore.

Defaults to an in-memory (temporary) map.

func DisableAutoRefresh 

func DisableAutoRefresh() Option

DisableAutoRefresh completely disables 'auto-refresh' on the DHT routing table. This means that we will neither refresh the routing table periodically nor when the routing table size goes below the minimum threshold.

func DisableProviders 

func DisableProviders() Option

DisableProviders disables storing and retrieving provider records.

Defaults to enabled.

WARNING: do not change this unless you're using a forked DHT (i.e., a private network and/or distinct DHT protocols with the `Protocols` option).

func DisableValues 

func DisableValues() Option

DisableValues disables storing and retrieving value records (including public keys).

Defaults to enabled.

WARNING: do not change this unless you're using a forked DHT (i.e., a private network and/or distinct DHT protocols with the `Protocols` option).

func EnableOptimisticProvide 

func EnableOptimisticProvide() Option

EnableOptimisticProvide enables an optimization that skips the last hops of the provide process. This works by using the network size estimator (which uses the keyspace density of queries) to optimistically send ADD_PROVIDER requests when we most likely have found the last hop. It will also run some ADD_PROVIDER requests asynchronously in the background after returning, this allows to optimistically return earlier if some threshold number of RPCs have succeeded. The number of background/in-flight queries can be configured with the OptimisticProvideJobsPoolSize option.

EXPERIMENTAL: This is an experimental option and might be removed in the future. Use at your own risk.

func LookupCheckConcurrency 

func LookupCheckConcurrency(n int) Option

LookupInterval configures maximal number of go routines that can be used to perform a lookup check operation, before adding a new node to the routing table.

func MaxRecordAge 

func MaxRecordAge(maxAge time.Duration) Option

MaxRecordAge specifies the maximum time that any node will hold onto a record ("PutValue record") from the time its received. This does not apply to any other forms of validity that the record may contain. For example, a record may contain an ipns entry with an EOL saying its valid until the year 2020 (a great time in the future). For that record to stick around it must be rebroadcasted more frequently than once every 'MaxRecordAge'

func Mode 

func Mode(m ModeOpt) Option

Mode configures which mode the DHT operates in (Client, Server, Auto).

Defaults to ModeAuto.

func NamespacedValidator 

func NamespacedValidator(ns string, v record.Validator) Option

NamespacedValidator adds a validator namespaced under `ns`. This option fails if the DHT is not using a `record.NamespacedValidator` as its validator (it uses one by default but this can be overridden with the `Validator` option). Adding a namespaced validator without changing the `Validator` will result in adding a new validator in addition to the default public key and IPNS validators. The "pk" and "ipns" namespaces cannot be overridden here unless a new `Validator` has been set first.

Example: Given a validator registered as `NamespacedValidator("ipns", myValidator)`, all records with keys starting with `/ipns/` will be validated with `myValidator`.

func OptimisticProvideJobsPoolSize 

func OptimisticProvideJobsPoolSize(size int) Option

OptimisticProvideJobsPoolSize allows to configure the asynchronicity limit for in-flight ADD_PROVIDER RPCs. It makes sense to set it to a multiple of optProvReturnRatio * BucketSize. Check the description of EnableOptimisticProvide for more details.

EXPERIMENTAL: This is an experimental option and might be removed in the future. Use at your own risk.

func ProtocolExtension 

func ProtocolExtension(ext protocol.ID) Option

ProtocolExtension adds an application specific protocol to the DHT protocol. For example, /ipfs/lan/kad/1.0.0 instead of /ipfs/kad/1.0.0. extension should be of the form /lan.

func ProtocolPrefix 

func ProtocolPrefix(prefix protocol.ID) Option

ProtocolPrefix sets an application specific prefix to be attached to all DHT protocols. For example, /myapp/kad/1.0.0 instead of /ipfs/kad/1.0.0. Prefix should be of the form /myapp.

Defaults to dht.DefaultPrefix

func ProviderStore 

func ProviderStore(ps providers.ProviderStore) Option

ProviderStore sets the provider storage manager.

func QueryFilter 

func QueryFilter(filter QueryFilterFunc) Option

QueryFilter sets a function that approves which peers may be dialed in a query

func Resiliency 

func Resiliency(beta int) Option

Resiliency configures the number of peers closest to a target that must have responded in order for a given query path to complete.

The default value is 3.

func RoutingTableFilter 

func RoutingTableFilter(filter RouteTableFilterFunc) Option

RoutingTableFilter sets a function that approves which peers may be added to the routing table. The host should already have at least one connection to the peer under consideration.

func RoutingTableLatencyTolerance 

func RoutingTableLatencyTolerance(latency time.Duration) Option

RoutingTableLatencyTolerance sets the maximum acceptable latency for peers in the routing table's cluster.

func RoutingTablePeerDiversityFilter 

func RoutingTablePeerDiversityFilter(pg peerdiversity.PeerIPGroupFilter) Option

RoutingTablePeerDiversityFilter configures the implementation of the `PeerIPGroupFilter` that will be used to construct the diversity filter for the Routing Table. Please see the docs for `peerdiversity.PeerIPGroupFilter` AND `peerdiversity.Filter` for more details.

func RoutingTableRefreshPeriod 

func RoutingTableRefreshPeriod(period time.Duration) Option

RoutingTableRefreshPeriod sets the period for refreshing buckets in the routing table. The DHT will refresh buckets every period by:

  1. First searching for nearby peers to figure out how many buckets we should try to fill.
  2. Then searching for a random key in each bucket that hasn't been queried in the last refresh period.

func RoutingTableRefreshQueryTimeout 

func RoutingTableRefreshQueryTimeout(timeout time.Duration) Option

RoutingTableRefreshQueryTimeout sets the timeout for routing table refresh queries.

func V1ProtocolOverride 

func V1ProtocolOverride(proto protocol.ID) Option

V1ProtocolOverride overrides the protocolID used for /kad/1.0.0 with another. This is an advanced feature, and should only be used to handle legacy networks that have not been using protocolIDs of the form /app/kad/1.0.0.

This option will override and ignore the ProtocolPrefix and ProtocolExtension options

func Validator 

func Validator(v record.Validator) Option

Validator configures the DHT to use the specified validator.

Defaults to a namespaced validator that can validate both public key (under the "pk" namespace) and IPNS records (under the "ipns" namespace). Setting the validator implies that the user wants to control the validators and therefore the default public key and IPNS validators will not be added.

type PeerKadID 

type PeerKadID struct {
	Peer peer.ID
	Kad  kbucket.ID
}

PeerKadID contains a libp2p Peer ID and a binary Kademlia ID.

func NewPeerKadID 

func NewPeerKadID(p peer.ID) *PeerKadID

NewPeerKadID creates a PeerKadID from a libp2p Peer ID.

func NewPeerKadIDSlice 

func NewPeerKadIDSlice(p []peer.ID) []*PeerKadID

NewPeerKadIDSlice creates a slice of PeerKadID from the passed slice of libp2p Peer IDs.

func OptPeerKadID 

func OptPeerKadID(p peer.ID) *PeerKadID

OptPeerKadID returns a pointer to a PeerKadID or nil if the passed Peer ID is it's default value.

type QueryFilterFunc 

type QueryFilterFunc = dhtcfg.QueryFilterFunc

QueryFilterFunc is a filter applied when considering peers to dial when querying

type RouteTableFilterFunc 

type RouteTableFilterFunc = dhtcfg.RouteTableFilterFunc

RouteTableFilterFunc is a filter applied when considering connections to keep in the local route table.

Source Files

View all Source files

Directories

Path Synopsis
Package dual provides an implementation of a split or "dual" dht, where two parallel instances are maintained for the global internet and the local LAN respectively.
 Package dual provides an implementation of a split or "dual" dht, where two parallel instances are maintained for the global internet and the local LAN respectively.
config
net
testing

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.