dht

README

This is a golang Kademlia/Bittorrent DHT library that implements BEP 5.

It's typically used by a torrent client such as Taipei-Torrent, but it could also be used by a standalone DHT routers, or for other more creative purposes.

The DHT performs well and supports the most important features despite its simple API.

A multi-node deployment is able to process more than 5000 incoming packets per second in a single core of a very old AMD Athlon(tm) 64 Processor 3700+, when the optional rate-limiting feature is disabled.

By default, if left running for several days the DHT node should use approx. 30MB of RAM. This can be adjusted by changing MaxInfoHashes and MaxInfoHashPeers accordingly.

For usage details, see the online documentation at: http://godoc.org/github.com/nictuku/dht

A full example is at: find_infohash_and_wait

Documentation

Index

• Variables
• func DecodePeerAddress(x string) string
• func RegisterFlags(c *Config)
• type Config
  • func NewConfig() *Config
• type DHT
  • func New(config *Config) (node *DHT, err error)
  • func (d *DHT) AddNode(addr string)
  • func (d *DHT) Bootstrap()
  • func (d *DHT) PeersRequest(ih string, announce bool)
  • func (d *DHT) PeersRequestPort(ih string, announce bool, port int)
  • func (d *DHT) Port() int
  • func (d *DHT) RemoveInfoHash(ih string)
  • func (d *DHT) Run() error
  • func (d *DHT) Start() (err error)
  • func (d *DHT) Stop()
• type DebugLogger
• type InfoHash
  • func DecodeInfoHash(x string) (b InfoHash, err error)
  • func (i InfoHash) String() string
• type Logger
• type PeerStore

Examples

• DHT

Variables

var DefaultConfig = NewConfig()

Functions

func DecodePeerAddress

func DecodePeerAddress(x string) string

DecodePeerAddress transforms the binary-encoded host:port address into a human-readable format. So, "abcdef" becomes 97.98.99.100:25958.

func RegisterFlags

func RegisterFlags(c *Config)

Registers Config fields as command line flags. If c is nil, DefaultConfig is used.

Types

type Config

type Config struct {
	// IP Address to listen on.  If left blank, one is chosen automatically.
	Address string
	// UDP port the DHT node should listen on. If zero, it picks a random port.
	Port int
	// Number of peers that DHT will try to find for each infohash being searched. This might
	// later be moved to a per-infohash option. Default value: 5.
	NumTargetPeers int
	// Comma separated list of DHT routers used for bootstrapping the network.
	DHTRouters string
	// Maximum number of nodes to store in the routing table. Default value: 100.
	MaxNodes int
	// How often to ping nodes in the network to see if they are reachable. Default value: 15 min.
	CleanupPeriod time.Duration
	//  If true, the node will read the routing table from disk on startup and save routing
	//  table snapshots on disk every few minutes. Default value: true.
	SaveRoutingTable bool
	// How often to save the routing table to disk. Default value: 5 minutes.
	SavePeriod time.Duration
	// Maximum packets per second to be processed. Disabled if negative. Default value: 100.
	RateLimit int64
	// MaxInfoHashes is the limit of number of infohashes for which we should keep a peer list.
	// If this and MaxInfoHashPeers are unchanged, it should consume around 25 MB of RAM. Larger
	// values help keeping the DHT network healthy. Default value: 2048.
	MaxInfoHashes int
	// MaxInfoHashPeers is the limit of number of peers to be tracked for each infohash. A
	// single peer contact typically consumes 6 bytes. Default value: 256.
	MaxInfoHashPeers int
	// ClientPerMinuteLimit protects against spammy clients. Ignore their requests if exceeded
	// this number of packets per minute. Default value: 50.
	ClientPerMinuteLimit int
	// ThrottlerTrackedClients is the number of hosts the client throttler remembers. An LRU is used to
	// track the most interesting ones. Default value: 1000.
	ThrottlerTrackedClients int64
	//Protocol for UDP connections, udp4= IPv4, udp6 = IPv6
	UDPProto string
}

Config for the DHT Node. Use NewConfig to create a configuration with default values.

func NewConfig

func NewConfig() *Config

Creates a *Config populated with default values.

type DHT

type DHT struct {
	// PeersRequestResults receives results after user calls PeersRequest method.
	// Map key contains the 20 bytes infohash string, value contains the list of peer addresses.
	// Peer addresses are in binary format. You can use DecodePeerAddress function to decode peer addresses.
	PeersRequestResults chan map[InfoHash][]string
	// Logger contains hooks for a client to attach for certain RPCs.
	// Hooks is a better name for the job but we don't want to change it and break existing users.
	Logger Logger
	// DebugLogger is called with log messages.
	// By default, nothing is printed to the output from the library.
	// If you want to see log messages, you have to provide a DebugLogger implementation.
	DebugLogger DebugLogger
	// contains filtered or unexported fields
}

DHT should be created by New(). It provides DHT features to a torrent client, such as finding new peers for torrent downloads without requiring a tracker.

Example

ExampleDHT is a simple example that searches for a particular infohash and exits when it finds any peers. A stand-alone version can be found in the examples/ directory.

if testing.Short() {
	fmt.Println("Peer found for the requested infohash or the test was skipped")
	return
}
d, err := New(nil)
if err != nil {
	fmt.Println(err)
	return
}
if err = d.Start(); err != nil {
	fmt.Println(err)
	return
}

infoHash, err := DecodeInfoHash("d1c5676ae7ac98e8b19f63565905105e3c4c37a2")
if err != nil {
	fmt.Printf("DecodeInfoHash faiure: %v", err)
	return
}

tick := time.Tick(time.Second)

var infoHashPeers map[InfoHash][]string
timer := time.NewTimer(30 * time.Second)
defer timer.Stop()
M:
for {
	select {
	case <-tick:
		// Repeat the request until a result appears, querying nodes that haven't been
		// consulted before and finding close-by candidates for the infohash.
		d.PeersRequest(string(infoHash), false)
	case infoHashPeers = <-d.PeersRequestResults:
		break M
	case <-timer.C:
		fmt.Printf("Could not find new peers: timed out")
		return
	}
}
for ih, peers := range infoHashPeers {
	if len(peers) > 0 {
		// Peers are encoded in binary format. Decoding example using github.com/nictuku/nettools:
		//for _, peer := range peers {
		//	fmt.Println(DecodePeerAddress(peer))
		//}

		if fmt.Sprintf("%x", ih) == "d1c5676ae7ac98e8b19f63565905105e3c4c37a2" {
			fmt.Println("Peer found for the requested infohash or the test was skipped")
			return
		}
	}
}

Output:

Peer found for the requested infohash or the test was skipped

func New

func New(config *Config) (node *DHT, err error)

New creates a DHT node. If config is nil, DefaultConfig will be used. Changing the config after calling this function has no effect.

This method replaces NewDHTNode.

func (*DHT) AddNode

func (d *DHT) AddNode(addr string)

AddNode informs the DHT of a new node it should add to its routing table. addr is a string containing the target node's "host:port" UDP address.

func (*DHT) Bootstrap

func (d *DHT) Bootstrap()

func (*DHT) PeersRequest

func (d *DHT) PeersRequest(ih string, announce bool)

PeersRequest asks the DHT to search for more peers for the infoHash provided. announce should be true if the connected peer is actively downloading this infohash, which is normally the case - unless this DHT node is just a router that doesn't downloads torrents. The infoHash added to the store can be deleted with RemoveInfoHash method.

func (*DHT) PeersRequestPort

func (d *DHT) PeersRequestPort(ih string, announce bool, port int)

PeersRequestPort is same as PeersRequest but it takes additional port argument to use in "announce_peer" request.

func (*DHT) Port

func (d *DHT) Port() int

Port returns the port number assigned to the DHT. This is useful when when initialising the DHT with port 0, i.e. automatic port assignment, in order to retrieve the actual port number used.

func (*DHT) RemoveInfoHash

func (d *DHT) RemoveInfoHash(ih string)

RemoveInfoHash removes infoHash from local store. This method should be called when the peer is no longer downloading this infoHash.

func (*DHT) Run

func (d *DHT) Run() error

Run launches the dht node. It starts a listener on the desired address, then runs the main loop in the same go routine. If initSocket fails, Run returns with the error. If initSocket succeeds, Run blocks until d.Stop() is called. DEPRECATED - Start should be used instead of Run

func (*DHT) Start

func (d *DHT) Start() (err error)

Start launches the dht node. It starts a listener on the desired address, then runs the main loop in a separate go routine - Start replaces Run and will always return, with nil if the dht successfully started or with an error either. d.Stop() is expected by the caller to stop the dht

func (*DHT) Stop

func (d *DHT) Stop()

Stop the DHT node.

type DebugLogger

type DebugLogger interface {
	Debugf(format string, args ...interface{})
	Infof(format string, args ...interface{})
	Errorf(format string, args ...interface{})
}

type InfoHash

type InfoHash string

func DecodeInfoHash

func DecodeInfoHash(x string) (b InfoHash, err error)

DecodeInfoHash transforms a hex-encoded 20-characters string to a binary infohash.

func (InfoHash) String

func (i InfoHash) String() string

type Logger

type Logger interface {
	GetPeers(addr net.UDPAddr, queryID string, infoHash InfoHash)
}

Logger allows the DHT client to attach hooks for certain RPCs so it can log interesting events any way it wants.

type PeerStore

type PeerStore interface {
	Get(ih InfoHash) *peerContactsSet
	Count(ih InfoHash) int
	Alive(ih InfoHash) int
	PeerContacts(ih InfoHash) []string
	AddContact(ih InfoHash, peerContact string) bool
	KillContact(peerContact string)
	AddLocalDownload(ih InfoHash, port int)
	HasLocalDownload(ih InfoHash) (port int)
	RemoveLocalDownload(ih InfoHash)
}