torrent

package
v0.0.0-...-add6754 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 31, 2016 License: MPL-2.0, MIT Imports: 43 Imported by: 0

README

torrent

Build Status GoDoc Coverage

This repository implements BitTorrent-related packages and command-line utilities in Go. The emphasis is on use as a library from other projects. It's been used 24/7 in production by a downstream, private service since late 2014.

There is support for protocol encryption, DHT, PEX, uTP, and various extensions. See the package documentation for a more complete list. There are several data storage backends provided: blob, file, and mmap, and you can write your own, such as to store data on S3, or in a database. You can use the provided binaries in ./cmd, or use torrent as a library for your own applications.

Many of the sub-packages can be used for other purposes: dht, bencode, and tracker, in particular.

Installation

Install the library package with go get github.com/anacrolix/torrent, or the provided cmds with go get github.com/anacrolix/torrent/cmd/....

Library example

There is a small example in the package documentation.

Other projects using Torrent

Commands

Here I'll describe what some of the provided commands in ./cmd do.

Note that the godo command which is invoked in the following examples builds and executes a Go import path, like go run. It's easier to use this convention than to spell out the install/invoke cycle for every single example.

torrent

Downloads torrents from the command-line.

$ go get github.com/anacrolix/torrent/cmd/torrent
$ torrent 'magnet:?xt=urn:btih:KRWPCX3SJUM4IMM4YF5RPHL6ANPYTQPU'
2015/04/01 02:08:20 main.go:137: downloaded ALL the torrents
$ md5sum ubuntu-14.04.2-desktop-amd64.iso
1b305d585b1918f297164add46784116  ubuntu-14.04.2-desktop-amd64.iso
$ echo such amaze
wow
torrentfs

torrentfs mounts a FUSE filesystem at -mountDir. The contents are the torrents described by the torrent files and magnet links at -torrentPath. Data for read requests is fetched only as required from the torrent network, and stored at -downloadDir.

$ mkdir mnt torrents
$ godo github.com/anacrolix/torrent/cmd/torrentfs -mountDir mnt -torrentPath torrents &
$ cd torrents
$ wget http://releases.ubuntu.com/14.04.2/ubuntu-14.04.2-desktop-amd64.iso.torrent
$ cd ..
$ ls mnt
ubuntu-14.04.2-desktop-amd64.iso
$ pv mnt/ubuntu-14.04.2-desktop-amd64.iso | md5sum
996MB 0:04:40 [3.55MB/s] [========================================>] 100%
1b305d585b1918f297164add46784116  -
torrent-magnet

Creates a magnet link from a torrent file. Note the extracted trackers, display name, and info hash.

$ godo github.com/anacrolix/torrent/cmd/torrent-magnet < ubuntu-14.04.2-desktop-amd64.iso.torrent
magnet:?xt=urn:btih:546cf15f724d19c4319cc17b179d7e035f89c1f4&dn=ubuntu-14.04.2-desktop-amd64.iso&tr=http%3A%2F%2Ftorrent.ubuntu.com%3A6969%2Fannounce&tr=http%3A%2F%2Fipv6.torrent.ubuntu.com%3A6969%2Fannounce
dht-ping

Pings DHT nodes with the given network addresses.

$ godo ./cmd/dht-ping router.bittorrent.com:6881 router.utorrent.com:6881
2015/04/01 17:21:23 main.go:33: dht server on [::]:60058
32f54e697351ff4aec29cdbaabf2fbe3467cc267 (router.bittorrent.com:6881): 648.218621ms
ebff36697351ff4aec29cdbaabf2fbe3467cc267 (router.utorrent.com:6881): 873.864706ms
2/2 responses (100.000000%)

Documentation

Overview

Package torrent implements a torrent client. Goals include:

  • Configurable data storage, such as file, mmap, and piece-based.
  • Downloading on demand: torrent.Reader will request only the data required to satisfy Reads, which is ideal for streaming and torrentfs.

BitTorrent features implemented include:

  • Protocol obfuscation
  • DHT
  • uTP
  • PEX
  • Magnet
  • IP Blocklists
  • Some IPv6
  • UDP Trackers

ConfigDir

A Client has a configurable ConfigDir that defaults to $HOME/.config/torrent. Torrent metainfo files are cached at $CONFIGDIR/torrents/$infohash.torrent. Infohashes in $CONFIGDIR/banned_infohashes cannot be added to the Client. A P2P Plaintext Format blocklist is loaded from a file at the location specified by the environment variable TORRENT_BLOCKLIST_FILE if set. otherwise from $CONFIGDIR/blocklist. If $CONFIGDIR/packed-blocklist exists, this is memory- mapped as a packed IP blocklist, saving considerable memory.

Example
package main

import (
	"log"

	"github.com/anacrolix/torrent"
)

func main() {
	c, _ := torrent.NewClient(nil)
	defer c.Close()
	t, _ := c.AddMagnet("magnet:?xt=urn:btih:ZOCMZQIPFFW7OLLMIC5HUB6BPCSDEOQU")
	<-t.GotInfo()
	t.DownloadAll()
	c.WaitAll()
	log.Print("ermahgerd, torrent downloaded")
}
Output:

Example (FileReader)
package main

import (
	"io"

	"github.com/anacrolix/torrent"
)

func main() {
	var (
		t torrent.Torrent
		f torrent.File
	)
	r := t.NewReader()
	defer r.Close()
	// Access the parts of the torrent pertaining to f. Data will be
	// downloaded as required, per the configuration of the torrent.Reader.
	_ = io.NewSectionReader(r, f.Offset(), f.Length())
}
Output:

Index

Examples

Constants

View Source
const (
	PiecePriorityNone      piecePriority = iota // Not wanted.
	PiecePriorityNormal                         // Wanted.
	PiecePriorityReadahead                      // May be required soon.
	PiecePriorityNext                           // Succeeds a piece where a read occurred.
	PiecePriorityNow                            // A read occurred in this piece.
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Clients contain zero or more Torrents. A client manages a blocklist, the TCP/UDP protocol ports, and DHT as desired.

func NewClient

func NewClient(cfg *Config) (cl *Client, err error)

Creates a new client.

func (*Client) AddMagnet

func (me *Client) AddMagnet(uri string) (T Torrent, err error)

func (*Client) AddTorrent

func (me *Client) AddTorrent(mi *metainfo.MetaInfo) (T Torrent, err error)

func (*Client) AddTorrentFromFile

func (me *Client) AddTorrentFromFile(filename string) (T Torrent, err error)

func (*Client) AddTorrentSpec

func (cl *Client) AddTorrentSpec(spec *TorrentSpec) (T Torrent, new bool, err error)

Add or merge a torrent spec. If the torrent is already present, the trackers will be merged with the existing ones. If the Info isn't yet known, it will be set. The display name is replaced if the new spec provides one. Returns new if the torrent wasn't already in the client.

func (*Client) Close

func (me *Client) Close()

Stops the client. All connections to peers are closed and all activity will come to a halt.

func (*Client) ConfigDir

func (cl *Client) ConfigDir() string

The directory where the Client expects to find and store configuration data. Defaults to $HOME/.config/torrent.

func (*Client) DHT

func (me *Client) DHT() *dht.Server

func (*Client) IPBlockList

func (me *Client) IPBlockList() iplist.Ranger

func (*Client) ListenAddr

func (me *Client) ListenAddr() (addr net.Addr)

func (*Client) PeerID

func (me *Client) PeerID() string

func (*Client) SetIPBlockList

func (me *Client) SetIPBlockList(list iplist.Ranger)

func (*Client) Torrent

func (cl *Client) Torrent(ih InfoHash) (T Torrent, ok bool)

Returns a handle to the given torrent, if it's present in the client.

func (*Client) Torrents

func (me *Client) Torrents() (ret []Torrent)

Returns handles to all the torrents loaded in the Client.

func (*Client) WaitAll

func (me *Client) WaitAll() bool

Returns true when all torrents are completely downloaded and false if the client is stopped before that.

func (*Client) WriteStatus

func (cl *Client) WriteStatus(_w io.Writer)

Writes out a human readable status of the client, such as for writing to a HTTP status page.

type Config

type Config struct {
	// Store torrent file data in this directory unless TorrentDataOpener is
	// specified.
	DataDir string `long:"data-dir" description:"directory to store downloaded torrent data"`
	// The address to listen for new uTP and TCP bittorrent protocol
	// connections. DHT shares a UDP socket with uTP unless configured
	// otherwise.
	ListenAddr string `long:"listen-addr" value-name:"HOST:PORT"`
	// Don't announce to trackers. This only leaves DHT to discover peers.
	DisableTrackers bool `long:"disable-trackers"`
	DisablePEX      bool `long:"disable-pex"`
	// Don't create a DHT.
	NoDHT bool `long:"disable-dht"`
	// Overrides the default DHT configuration.
	DHTConfig dht.ServerConfig
	// Don't ever send chunks to peers.
	NoUpload bool `long:"no-upload"`
	// Upload even after there's nothing in it for us. By default uploading is
	// not altruistic.
	Seed bool `long:"seed"`
	// User-provided Client peer ID. If not present, one is generated automatically.
	PeerID string
	// For the bittorrent protocol.
	DisableUTP bool
	// For the bittorrent protocol.
	DisableTCP bool `long:"disable-tcp"`
	// Don't automatically load "$ConfigDir/blocklist".
	NoDefaultBlocklist bool
	// Defaults to "$HOME/.config/torrent". This is where "blocklist",
	// "torrents" and other operational files are stored.
	ConfigDir string
	// Don't save or load to a cache of torrent files stored in
	// "$ConfigDir/torrents".
	DisableMetainfoCache bool
	// Called to instantiate storage for each added torrent. Provided backends
	// are in $REPO/data. If not set, the "file" implementation is used.
	TorrentDataOpener
	DisableEncryption bool `long:"disable-encryption"`

	IPBlocklist *iplist.IPList
	DisableIPv6 bool `long:"disable-ipv6"`
	// Perform logging and any other behaviour that will help debug.
	Debug bool `help:"enable debug logging"`
}

Override Client defaults.

type Data

type Data interface {
	io.ReaderAt
	io.WriterAt
	// Bro, do you even io.Closer?
	Close()
	// If the data isn't available, err should be io.ErrUnexpectedEOF.
	WriteSectionTo(w io.Writer, off, n int64) (written int64, err error)
	// We believe the piece data will pass a hash check.
	PieceCompleted(index int) error
	// Returns true if the piece is complete.
	PieceComplete(index int) bool
}

Represents data storage for a Torrent.

type File

type File struct {
	// contains filtered or unexported fields
}

Provides access to regions of torrent data that correspond to its files.

func (*File) DisplayPath

func (f *File) DisplayPath() string

The relative file path for a multi-file torrent, and the torrent name for a single-file torrent.

func (File) FileInfo

func (f File) FileInfo() metainfo.FileInfo

func (*File) Length

func (f *File) Length() int64

func (*File) Offset

func (f *File) Offset() int64

Data for this file begins this far into the torrent.

func (File) Path

func (f File) Path() string

func (*File) PrioritizeRegion

func (f *File) PrioritizeRegion(off, len int64)

Marks pieces in the region of the file for download. This is a helper wrapping Torrent.SetRegionPriority.

func (*File) State

func (f *File) State() (ret []FilePieceState)

Returns the state of pieces in this file.

func (*File) Torrent

func (f *File) Torrent() Torrent

type FilePieceState

type FilePieceState struct {
	Bytes int64 // Bytes within the piece that are part of this File.
	PieceState
}

type Handle

type Handle interface {
	io.Reader
	io.Seeker
	io.Closer
	io.ReaderAt
}

A file-like handle to some torrent data resource.

type InfoHash

type InfoHash [20]byte

func (InfoHash) AsString

func (ih InfoHash) AsString() string

func (InfoHash) HexString

func (ih InfoHash) HexString() string

type Magnet

type Magnet struct {
	InfoHash    [20]byte
	Trackers    []string
	DisplayName string
}

func ParseMagnetURI

func ParseMagnetURI(uri string) (m Magnet, err error)

ParseMagnetURI parses Magnet-formatted URIs into a Magnet instance

func (*Magnet) String

func (m *Magnet) String() (ret string)

type Peer

type Peer struct {
	Id     [20]byte
	IP     net.IP
	Port   int
	Source peerSource
	// Peer is known to support encryption.
	SupportsEncryption bool
}

type PieceState

type PieceState struct {
	Priority piecePriority
	// The piece is available in its entirety.
	Complete bool
	// The piece is being hashed, or is queued for hash.
	Checking bool
	// Some of the piece has been obtained.
	Partial bool
}

The current state of a piece.

type PieceStateRun

type PieceStateRun struct {
	PieceState
	Length int // How many consecutive pieces have this state.
}

Represents a series of consecutive pieces with the same state.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Accesses torrent data via a client.

func (*Reader) Close

func (r *Reader) Close() error

func (*Reader) Read

func (r *Reader) Read(b []byte) (n int, err error)

func (*Reader) ReadAt

func (r *Reader) ReadAt(b []byte, off int64) (n int, err error)

func (*Reader) Seek

func (r *Reader) Seek(off int64, whence int) (ret int64, err error)

func (*Reader) SetReadahead

func (r *Reader) SetReadahead(readahead int64)

Configure the number of bytes ahead of a read that should also be prioritized in preparation for further reads.

func (*Reader) SetResponsive

func (r *Reader) SetResponsive()

Don't wait for pieces to complete and be verified. Read calls return as soon as they can when the underlying chunks become available.

type Torrent

type Torrent struct {
	// contains filtered or unexported fields
}

The public handle to a live torrent within a Client.

func (Torrent) AddPeers

func (t Torrent) AddPeers(pp []Peer) error

func (Torrent) BytesCompleted

func (t Torrent) BytesCompleted() int64

Number of bytes of the entire torrent we have completed.

func (Torrent) DownloadAll

func (t Torrent) DownloadAll()

Marks the entire torrent for download. Requires the info first, see GotInfo.

func (Torrent) Drop

func (t Torrent) Drop()

Drop the torrent from the client, and close it.

func (Torrent) Files

func (t Torrent) Files() (ret []File)

Returns handles to the files in the torrent. This requires the metainfo is available first.

func (Torrent) GotInfo

func (t Torrent) GotInfo() <-chan struct{}

Closed when the info (.Info()) for the torrent has become available. Using features of Torrent that require the info before it is available will have undefined behaviour.

func (Torrent) Info

func (t Torrent) Info() *metainfo.Info

Returns the metainfo info dictionary, or nil if it's not yet available.

func (Torrent) InfoHash

func (t Torrent) InfoHash() InfoHash

The torrent's infohash. This is fixed and cannot change. It uniquely identifies a torrent.

func (Torrent) Length

func (t Torrent) Length() int64

func (Torrent) MetaInfo

func (t Torrent) MetaInfo() *metainfo.MetaInfo

Returns a run-time generated metainfo for the torrent that includes the info bytes and announce-list as currently known to the client.

func (Torrent) Name

func (t Torrent) Name() string

The current working name for the torrent. Either the name in the info dict, or a display name given such as by the dn value in a magnet link, or "".

func (Torrent) NewReader

func (t Torrent) NewReader() (ret *Reader)

Returns a Reader bound to the torrent's data. All read calls block until the data requested is actually available. Priorities are set to ensure the data requested will be downloaded as soon as possible.

func (Torrent) NumPieces

func (t Torrent) NumPieces() int

The number of pieces in the torrent. This requires that the info has been obtained first.

func (Torrent) PieceStateRuns

func (t Torrent) PieceStateRuns() []PieceStateRun

Returns the state of pieces of the torrent. They are grouped into runs of same state. The sum of the state run lengths is the number of pieces in the torrent.

func (Torrent) Seeding

func (t Torrent) Seeding() bool

Returns true if the torrent is currently being seeded. This occurs when the client is willing to upload without wanting anything in return.

func (Torrent) SetDisplayName

func (t Torrent) SetDisplayName(dn string)

Clobbers the torrent display name. The display name is used as the torrent name if the metainfo is not available.

func (Torrent) SetRegionPriority

func (t Torrent) SetRegionPriority(off, len int64)

Marks the pieces in the given region for download.

func (Torrent) SubscribePieceStateChanges

func (t Torrent) SubscribePieceStateChanges() *pubsub.Subscription

The subscription emits as (int) the index of pieces as their state changes. A state change is when the PieceState for a piece alters in value.

type TorrentDataOpener

type TorrentDataOpener func(*metainfo.Info) Data

type TorrentSpec

type TorrentSpec struct {
	// The tiered tracker URIs.
	Trackers [][]string
	InfoHash InfoHash
	Info     *metainfo.InfoEx
	// The name to use if the Name field from the Info isn't available.
	DisplayName string
	// The chunk size to use for outbound requests. Defaults to 16KiB if not
	// set.
	ChunkSize int
}

Specifies a new torrent for adding to a client. There are helpers for magnet URIs and torrent metainfo files.

func TorrentSpecFromMagnetURI

func TorrentSpecFromMagnetURI(uri string) (spec *TorrentSpec, err error)

func TorrentSpecFromMetaInfo

func TorrentSpecFromMetaInfo(mi *metainfo.MetaInfo) (spec *TorrentSpec)

Directories

Path Synopsis
cmd
dht-ping
Pings DHT nodes with the given network addresses.
Pings DHT nodes with the given network addresses.
magnet-metainfo
Converts magnet URIs and info hashes into torrent metainfo files.
Converts magnet URIs and info hashes into torrent metainfo files.
torrent
Downloads torrents from the command-line.
Downloads torrents from the command-line.
torrent-pick
Downloads torrents from the command-line.
Downloads torrents from the command-line.
torrentfs
Mounts a FUSE filesystem backed by torrents and magnet links.
Mounts a FUSE filesystem backed by torrents and magnet links.
data
Standard use involves creating a Server, and calling Announce on it with the details of your local torrent client and infohash of interest.
Standard use involves creating a Server, and calling Announce on it with the details of your local torrent client and infohash of interest.
internal
pieceordering
Implements ordering of torrent piece indices for such purposes as download prioritization.
Implements ordering of torrent piece indices for such purposes as download prioritization.
Package iplist handles the P2P Plaintext Format described by https://en.wikipedia.org/wiki/PeerGuardian#P2P_plaintext_format.
Package iplist handles the P2P Plaintext Format described by https://en.wikipedia.org/wiki/PeerGuardian#P2P_plaintext_format.
cmd/pack-blocklist
Takes P2P blocklist text format in stdin, and outputs the packed format from the iplist package.
Takes P2P blocklist text format in stdin, and outputs the packed format from the iplist package.
Package logonce implements an io.Writer facade that only performs distinct writes.
Package logonce implements an io.Writer facade that only performs distinct writes.
dirwatch
Package dirwatch provides filesystem-notification based tracking of torrent info files and magnet URIs in a directory.
Package dirwatch provides filesystem-notification based tracking of torrent info files and magnet URIs in a directory.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL