Documentation ¶
Overview ¶
Package torrent implements a torrent client. Goals include:
- Configurable data storage, such as file, mmap, and piece-based.
- Downloading on demand: torrent.Reader will request only the data required to satisfy Reads, which is ideal for streaming and torrentfs.
BitTorrent features implemented include:
- Protocol obfuscation
- DHT
- uTP
- PEX
- Magnet links
- IP Blocklists
- Some IPv6
- HTTP and UDP tracker clients
- BEPs:
- 3: Basic BitTorrent protocol
- 5: DHT
- 6: Fast Extension (have all/none only)
- 7: IPv6 Tracker Extension
- 9: ut_metadata
- 10: Extension protocol
- 11: PEX
- 12: Multitracker metadata extension
- 15: UDP Tracker Protocol
- 20: Peer ID convention ("-GTnnnn-")
- 23: Tracker Returns Compact Peer Lists
- 27: Private torrents
- 29: uTorrent transport protocol
- 41: UDP Tracker Protocol Extensions
- 42: DHT Security extension
- 43: Read-only DHT Nodes
Example ¶
package main import ( "log" "git.anonify.eu/x0x/torrent" ) func main() { c, _ := torrent.NewClient(nil) defer c.Close() t, _ := c.AddMagnet("magnet:?xt=urn:btih:ZOCMZQIPFFW7OLLMIC5HUB6BPCSDEOQU") <-t.GotInfo() t.DownloadAll() c.WaitAll() log.Print("ermahgerd, torrent downloaded") }
Output:
Example (FileReader) ¶
package main import ( "git.anonify.eu/x0x/torrent" ) func main() { var f torrent.File // Accesses the parts of the torrent pertaining to f. Data will be // downloaded as required, per the configuration of the torrent.Reader. r := f.NewReader() defer r.Close() }
Output:
Index ¶
- Constants
- Variables
- func LoopbackListenHost(network string) string
- func NewUtpSocket(network, addr string, fc firewallCallback) (utpSocket, error)
- type Client
- func (cl *Client) AddDHTNodes(nodes []string)
- func (cl *Client) AddMagnet(uri string) (T *Torrent, err error)
- func (cl *Client) AddTorrent(mi *metainfo.MetaInfo) (T *Torrent, err error)
- func (cl *Client) AddTorrentFromFile(filename string) (T *Torrent, err error)
- func (cl *Client) AddTorrentInfoHash(infoHash metainfo.Hash) (t *Torrent, new bool)
- func (cl *Client) AddTorrentInfoHashWithStorage(infoHash metainfo.Hash, specStorage storage.ClientImpl) (t *Torrent, new bool)
- func (cl *Client) AddTorrentSpec(spec *TorrentSpec) (t *Torrent, new bool, err error)
- func (cl *Client) BadPeerIPs() []string
- func (cl *Client) Close()
- func (cl *Client) Closed() <-chan struct{}
- func (cl *Client) DhtServers() []*dht.Server
- func (cl *Client) ListenAddrs() (ret []net.Addr)
- func (cl *Client) LocalPort() (port int)
- func (cl *Client) PeerID() PeerID
- func (cl *Client) String() string
- func (cl *Client) Torrent(ih metainfo.Hash) (t *Torrent, ok bool)
- func (cl *Client) Torrents() []*Torrent
- func (cl *Client) WaitAll() bool
- func (cl *Client) WriteStatus(_w io.Writer)
- type ClientConfig
- type ConnStats
- type Count
- type File
- func (f *File) BytesCompleted() int64
- func (f *File) Cancel()deprecated
- func (f *File) DisplayPath() string
- func (f *File) Download()
- func (f File) FileInfo() metainfo.FileInfo
- func (f *File) Length() int64
- func (f *File) NewReader() Reader
- func (f *File) Offset() int64
- func (f File) Path() string
- func (f *File) Priority() piecePriority
- func (f *File) SetPriority(prio piecePriority)
- func (f *File) State() (ret []FilePieceState)
- func (f *File) Torrent() *Torrent
- type FilePieceState
- type Handle
- type HeaderObfuscationPolicy
- type InfoHash
- type IpPort
- type Peer
- type PeerExtensionBits
- type PeerID
- type Peers
- type Piece
- type PieceState
- type PieceStateChange
- type PieceStateRun
- type Reader
- type RequestStrategyMaker
- type Torrent
- func (t *Torrent) AddClientPeer(cl *Client)
- func (t *Torrent) AddPeers(pp []Peer)
- func (t *Torrent) AddTrackers(announceList [][]string)
- func (t *Torrent) BytesCompleted() int64
- func (t *Torrent) BytesMissing() int64
- func (t *Torrent) CancelPieces(begin, end pieceIndex)
- func (t *Torrent) Closed() <-chan struct{}
- func (t *Torrent) DownloadAll()
- func (t *Torrent) DownloadPieces(begin, end pieceIndex)
- func (t *Torrent) Drop()
- func (t *Torrent) Files() []*File
- func (t *Torrent) GotInfo() <-chan struct{}
- func (t *Torrent) Info() *metainfo.Info
- func (t *Torrent) InfoHash() metainfo.Hash
- func (t *Torrent) KnownSwarm() (ks []Peer)
- func (t *Torrent) Length() int64
- func (t *Torrent) Metainfo() metainfo.MetaInfo
- func (t *Torrent) Name() string
- func (t *Torrent) NewReader() Reader
- func (t *Torrent) NumPieces() pieceIndex
- func (t *Torrent) Piece(i pieceIndex) *Piece
- func (t *Torrent) PieceBytesMissing(piece int) int64
- func (t *Torrent) PieceState(piece pieceIndex) PieceState
- func (t *Torrent) PieceStateRuns() []PieceStateRun
- func (t *Torrent) Seeding() bool
- func (t *Torrent) SetDisplayName(dn string)
- func (t *Torrent) SetInfoBytes(b []byte) (err error)
- func (t *Torrent) SetMaxEstablishedConns(max int) (oldMax int)
- func (t *Torrent) Stats() TorrentStats
- func (t *Torrent) String() string
- func (t *Torrent) SubscribePieceStateChanges() *pubsub.Subscription
- func (t *Torrent) VerifyData()
- type TorrentSpec
- type TorrentStats
Examples ¶
Constants ¶
const ( PiecePriorityNone piecePriority = iota // Not wanted. Must be the zero value. PiecePriorityNormal // Wanted. PiecePriorityHigh // Wanted a lot. PiecePriorityReadahead // May be required soon. // Succeeds a piece where a read occurred. Currently the same as Now, // apparently due to issues with caching. PiecePriorityNext PiecePriorityNow // A Reader is reading in this piece. Highest urgency. )
Variables ¶
var DefaultHTTPUserAgent = "Go-Torrent/1.0"
Functions ¶
func LoopbackListenHost ¶
func NewUtpSocket ¶
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Clients contain zero or more Torrents. A Client manages a blocklist, the TCP/UDP protocol ports, and DHT as desired.
func NewClient ¶
func NewClient(cfg *ClientConfig) (cl *Client, err error)
func (*Client) AddDHTNodes ¶
func (*Client) AddTorrent ¶
func (*Client) AddTorrentFromFile ¶
func (*Client) AddTorrentInfoHash ¶
func (*Client) AddTorrentInfoHashWithStorage ¶
func (cl *Client) AddTorrentInfoHashWithStorage(infoHash metainfo.Hash, specStorage storage.ClientImpl) (t *Torrent, new bool)
Adds a torrent by InfoHash with a custom Storage implementation. If the torrent already exists then this Storage is ignored and the existing torrent returned with `new` set to `false`
func (*Client) AddTorrentSpec ¶
func (cl *Client) AddTorrentSpec(spec *TorrentSpec) (t *Torrent, new bool, err error)
Add or merge a torrent spec. If the torrent is already present, the trackers will be merged with the existing ones. If the Info isn't yet known, it will be set. The display name is replaced if the new spec provides one. Returns new if the torrent wasn't already in the client. Note that any `Storage` defined on the spec will be ignored if the torrent is already present (i.e. `new` return value is `true`)
func (*Client) BadPeerIPs ¶
func (*Client) Close ¶
func (cl *Client) Close()
Stops the client. All connections to peers are closed and all activity will come to a halt.
func (*Client) DhtServers ¶
func (cl *Client) DhtServers() []*dht.Server
func (*Client) ListenAddrs ¶
ListenAddrs addresses currently being listened to.
func (*Client) WaitAll ¶
Returns true when all torrents are completely downloaded and false if the client is stopped before that.
func (*Client) WriteStatus ¶
Writes out a human readable status of the client, such as for writing to a HTTP status page.
type ClientConfig ¶
type ClientConfig struct { // Store torrent file data in this directory unless .DefaultStorage is // specified. DataDir string `long:"data-dir" description:"directory to store downloaded torrent data"` // The address to listen for new uTP and TCP bittorrent protocol // connections. DHT shares a UDP socket with uTP unless configured // otherwise. ListenHost func(network string) string ListenPort int NoDefaultPortForwarding bool UpnpID string // Don't announce to trackers. This only leaves DHT to discover peers. DisableTrackers bool `long:"disable-trackers"` DisablePEX bool `long:"disable-pex"` // Don't create a DHT. NoDHT bool `long:"disable-dht"` DhtStartingNodes dht.StartingNodesGetter // Never send chunks to peers. NoUpload bool `long:"no-upload"` // Disable uploading even when it isn't fair. DisableAggressiveUpload bool `long:"disable-aggressive-upload"` // Upload even after there's nothing in it for us. By default uploading is // not altruistic, we'll only upload to encourage the peer to reciprocate. Seed bool `long:"seed"` // Only applies to chunks uploaded to peers, to maintain responsiveness // communicating local Client state to peers. Each limiter token // represents one byte. The Limiter's burst must be large enough to fit a // whole chunk, which is usually 16 KiB (see TorrentSpec.ChunkSize). UploadRateLimiter *rate.Limiter // Rate limits all reads from connections to peers. Each limiter token // represents one byte. The Limiter's burst must be bigger than the // largest Read performed on a the underlying rate-limiting io.Reader // minus one. This is likely to be the larger of the main read loop buffer // (~4096), and the requested chunk size (~16KiB, see // TorrentSpec.ChunkSize). DownloadRateLimiter *rate.Limiter // User-provided Client peer ID. If not present, one is generated automatically. PeerID string // For the bittorrent protocol. DisableUTP bool // For the bittorrent protocol. DisableTCP bool `long:"disable-tcp"` // Called to instantiate storage for each added torrent. Builtin backends // are in the storage package. If not set, the "file" implementation is // used. DefaultStorage storage.ClientImpl HeaderObfuscationPolicy HeaderObfuscationPolicy // The crypto methods to offer when initiating connections with header obfuscation. CryptoProvides mse.CryptoMethod // Chooses the crypto method to use when receiving connections with header obfuscation. CryptoSelector mse.CryptoSelector // Sets usage of Socks5 Proxy. Authentication should be included in the url if needed. // Examples: socks5://demo:demo@192.168.99.100:1080 // http://proxy.domain.com:3128 ProxyURL string IPBlocklist iplist.Ranger DisableIPv6 bool `long:"disable-ipv6"` DisableIPv4 bool DisableIPv4Peers bool // Perform logging and any other behaviour that will help debug. Debug bool `help:"enable debugging"` Logger log.Logger // HTTPProxy defines proxy for HTTP requests. // Format: func(*Request) (*url.URL, error), // or result of http.ProxyURL(HTTPProxy). // By default, it is composed from ClientConfig.ProxyURL, // if not set explicitly in ClientConfig struct HTTPProxy func(*http.Request) (*url.URL, error) // HTTPUserAgent changes default UserAgent for HTTP requests HTTPUserAgent string // Updated occasionally to when there's been some changes to client // behaviour in case other clients are assuming anything of us. See also // `bep20`. ExtendedHandshakeClientVersion string // Peer ID client identifier prefix. We'll update this occasionally to // reflect changes to client behaviour that other clients may depend on. // Also see `extendedHandshakeClientVersion`. Bep20 string // Peer dial timeout to use when there are limited peers. NominalDialTimeout time.Duration // Minimum peer dial timeout to use (even if we have lots of peers). MinDialTimeout time.Duration EstablishedConnsPerTorrent int HalfOpenConnsPerTorrent int // Maximum number of peer addresses in reserve. TorrentPeersHighWater int // Minumum number of peers before effort is made to obtain more peers. TorrentPeersLowWater int // Limit how long handshake can take. This is to reduce the lingering // impact of a few bad apples. 4s loses 1% of successful handshakes that // are obtained with 60s timeout, and 5% of unsuccessful handshakes. HandshakesTimeout time.Duration // The IP addresses as our peers should see them. May differ from the // local interfaces due to NAT or other network configurations. PublicIp4 net.IP PublicIp6 net.IP DisableAcceptRateLimiting bool ConnTracker *conntrack.Instance // OnQuery hook func DHTOnQuery func(query *krpc.Msg, source net.Addr) (propagate bool) DefaultRequestStrategy RequestStrategyMaker // contains filtered or unexported fields }
Probably not safe to modify this after it's given to a Client.
func NewDefaultClientConfig ¶
func NewDefaultClientConfig() *ClientConfig
func (*ClientConfig) SetListenAddr ¶
func (cfg *ClientConfig) SetListenAddr(addr string) *ClientConfig
type ConnStats ¶
type ConnStats struct { // Total bytes on the wire. Includes handshakes and encryption. BytesWritten Count BytesWrittenData Count BytesRead Count BytesReadData Count BytesReadUsefulData Count ChunksWritten Count ChunksRead Count ChunksReadUseful Count ChunksReadWasted Count MetadataChunksRead Count // Number of pieces data was written to, that subsequently passed verification. PiecesDirtiedGood Count // Number of pieces data was written to, that subsequently failed verification. Note that a // connection may not have been the sole dirtier of a piece. PiecesDirtiedBad Count }
Various connection-level metrics. At the Torrent level these are aggregates. Chunks are messages with data payloads. Data is actual torrent content without any overhead. Useful is something we needed locally. Unwanted is something we didn't ask for (but may still be useful). Written is things sent to the peer, and Read is stuff received from them.
type Count ¶
type Count struct {
// contains filtered or unexported fields
}
func (*Count) MarshalJSON ¶
type File ¶
type File struct {
// contains filtered or unexported fields
}
Provides access to regions of torrent data that correspond to its files.
func (*File) BytesCompleted ¶
Number of bytes of the entire file we have completed. This is the sum of completed pieces, and dirtied chunks of incomplete pieces.
func (*File) DisplayPath ¶
The relative file path for a multi-file torrent, and the torrent name for a single-file torrent.
func (*File) Download ¶
func (f *File) Download()
Requests that all pieces containing data in the file be downloaded.
func (*File) Priority ¶
func (f *File) Priority() piecePriority
Returns the priority per File.SetPriority.
func (*File) SetPriority ¶
func (f *File) SetPriority(prio piecePriority)
Sets the minimum priority for pieces in the File.
func (*File) State ¶
func (f *File) State() (ret []FilePieceState)
Returns the state of pieces in this file.
type FilePieceState ¶
type FilePieceState struct { Bytes int64 // Bytes within the piece that are part of this File. PieceState }
The download status of a piece that comprises part of a File.
type HeaderObfuscationPolicy ¶
type Peer ¶
type Peer struct { Id [20]byte IP net.IP Port int Source peerSource // Peer is known to support encryption. SupportsEncryption bool peer_protocol.PexPeerFlags // Whether we can ignore poor or bad behaviour from the peer. Trusted bool }
Peer connection info, handed about publicly.
func (*Peer) FromPex ¶
func (me *Peer) FromPex(na krpc.NodeAddr, fs peer_protocol.PexPeerFlags)
FromPex generate Peer from peer exchange
type PeerExtensionBits ¶
type PeerExtensionBits = pp.PeerExtensionBits
type Peers ¶
type Peers []Peer
func (*Peers) AppendFromPex ¶
func (me *Peers) AppendFromPex(nas []krpc.NodeAddr, fs []peer_protocol.PexPeerFlags)
type Piece ¶
type Piece struct {
// contains filtered or unexported fields
}
func (*Piece) SetPriority ¶
func (p *Piece) SetPriority(prio piecePriority)
type PieceState ¶
type PieceState struct { Priority piecePriority storage.Completion // The piece is being hashed, or is queued for hash. Checking bool // Some of the piece has been obtained. Partial bool }
The current state of a piece.
type PieceStateChange ¶
type PieceStateChange struct { Index int PieceState }
type PieceStateRun ¶
type PieceStateRun struct { PieceState Length int // How many consecutive pieces have this state. }
Represents a series of consecutive pieces with the same state.
type RequestStrategyMaker ¶
type RequestStrategyMaker func(callbacks requestStrategyCallbacks, clientLocker sync.Locker) requestStrategy
func RequestStrategyDuplicateRequestTimeout ¶
func RequestStrategyDuplicateRequestTimeout(duplicateRequestTimeout time.Duration) RequestStrategyMaker
func RequestStrategyFastest ¶
func RequestStrategyFastest() RequestStrategyMaker
func RequestStrategyFuzzing ¶
func RequestStrategyFuzzing() RequestStrategyMaker
type Torrent ¶
type Torrent struct {
// contains filtered or unexported fields
}
Maintains state of torrent within a Client. Many methods should not be called before the info is available, see .Info and .GotInfo.
func (*Torrent) AddClientPeer ¶
Adds each a trusted, pending peer for each of the Client's addresses.
func (*Torrent) AddTrackers ¶
func (*Torrent) BytesCompleted ¶
Number of bytes of the entire torrent we have completed. This is the sum of completed pieces, and dirtied chunks of incomplete pieces. Do not use this for download rate, as it can go down when pieces are lost or fail checks. Sample Torrent.Stats.DataBytesRead for actual file data download rate.
func (*Torrent) BytesMissing ¶
func (*Torrent) CancelPieces ¶
func (t *Torrent) CancelPieces(begin, end pieceIndex)
func (*Torrent) Closed ¶
func (t *Torrent) Closed() <-chan struct{}
Returns a channel that is closed when the Torrent is closed.
func (*Torrent) DownloadAll ¶
func (t *Torrent) DownloadAll()
Marks the entire torrent for download. Requires the info first, see GotInfo. Sets piece priorities for historical reasons.
func (*Torrent) DownloadPieces ¶
func (t *Torrent) DownloadPieces(begin, end pieceIndex)
Raise the priorities of pieces in the range [begin, end) to at least Normal priority. Piece indexes are not the same as bytes. Requires that the info has been obtained, see Torrent.Info and Torrent.GotInfo.
func (*Torrent) Drop ¶
func (t *Torrent) Drop()
Drop the torrent from the client, and close it. It's always safe to do this. No data corruption can, or should occur to either the torrent's data, or connected peers.
func (*Torrent) Files ¶
Returns handles to the files in the torrent. This requires that the Info is available first.
func (*Torrent) GotInfo ¶
func (t *Torrent) GotInfo() <-chan struct{}
Returns a channel that is closed when the info (.Info()) for the torrent has become available.
func (*Torrent) InfoHash ¶
The torrent's infohash. This is fixed and cannot change. It uniquely identifies a torrent.
func (*Torrent) KnownSwarm ¶
KnownSwarm returns the known subset of the peers in the Torrent's swarm, including active, pending, and half-open peers.
func (*Torrent) Length ¶
The completed length of all the torrent data, in all its files. This is derived from the torrent info, when it is available.
func (*Torrent) Metainfo ¶
Returns a run-time generated metainfo for the torrent that includes the info bytes and announce-list as currently known to the client.
func (*Torrent) Name ¶
The current working name for the torrent. Either the name in the info dict, or a display name given such as by the dn value in a magnet link, or "".
func (*Torrent) NewReader ¶
Returns a Reader bound to the torrent's data. All read calls block until the data requested is actually available.
func (*Torrent) NumPieces ¶
func (t *Torrent) NumPieces() pieceIndex
The number of pieces in the torrent. This requires that the info has been obtained first.
func (*Torrent) PieceBytesMissing ¶
Get missing bytes count for specific piece.
func (*Torrent) PieceState ¶
func (t *Torrent) PieceState(piece pieceIndex) PieceState
func (*Torrent) PieceStateRuns ¶
func (t *Torrent) PieceStateRuns() []PieceStateRun
Returns the state of pieces of the torrent. They are grouped into runs of same state. The sum of the state run lengths is the number of pieces in the torrent.
func (*Torrent) Seeding ¶
Returns true if the torrent is currently being seeded. This occurs when the client is willing to upload without wanting anything in return.
func (*Torrent) SetDisplayName ¶
Clobbers the torrent display name. The display name is used as the torrent name if the metainfo is not available.
func (*Torrent) SetInfoBytes ¶
func (*Torrent) SetMaxEstablishedConns ¶
func (*Torrent) Stats ¶
func (t *Torrent) Stats() TorrentStats
func (*Torrent) SubscribePieceStateChanges ¶
func (t *Torrent) SubscribePieceStateChanges() *pubsub.Subscription
The subscription emits as (int) the index of pieces as their state changes. A state change is when the PieceState for a piece alters in value.
func (*Torrent) VerifyData ¶
func (t *Torrent) VerifyData()
Forces all the pieces to be re-hashed. See also Piece.VerifyData. This should not be called before the Info is available.
type TorrentSpec ¶
type TorrentSpec struct { // The tiered tracker URIs. Trackers [][]string InfoHash metainfo.Hash InfoBytes []byte // The name to use if the Name field from the Info isn't available. DisplayName string // The chunk size to use for outbound requests. Defaults to 16KiB if not // set. ChunkSize int Storage storage.ClientImpl }
Specifies a new torrent for adding to a client. There are helpers for magnet URIs and torrent metainfo files.
func TorrentSpecFromMagnetURI ¶
func TorrentSpecFromMagnetURI(uri string) (spec *TorrentSpec, err error)
func TorrentSpecFromMetaInfo ¶
func TorrentSpecFromMetaInfo(mi *metainfo.MetaInfo) (spec *TorrentSpec)
type TorrentStats ¶
type TorrentStats struct { // Aggregates stats over all connections past and present. Some values may // not have much meaning in the aggregate context. ConnStats // Ordered by expected descending quantities (if all is well). TotalPeers int PendingPeers int ActivePeers int ConnectedSeeders int HalfOpenPeers int }
Source Files ¶
- Peer.go
- Peers.go
- bad_storage.go
- bep40.go
- client.go
- closewrapper.go
- config.go
- conn_stats.go
- connection.go
- deferrwl.go
- doc.go
- file.go
- global.go
- handshake.go
- listen.go
- misc.go
- multiless.go
- networks.go
- peerid.go
- piece.go
- piecestate.go
- portfwd.go
- prioritized_peers.go
- protocol.go
- ratelimitreader.go
- reader.go
- request_strategy.go
- socket.go
- spec.go
- t.go
- torrent.go
- torrent_stats.go
- tracker_scraper.go
- utp.go
- utp_libutp.go
- worst_conns.go
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
magnet-metainfo
Converts magnet URIs and info hashes into torrent metainfo files.
|
Converts magnet URIs and info hashes into torrent metainfo files. |
torrent
Downloads torrents from the command-line.
|
Downloads torrents from the command-line. |
torrent-pick
Downloads torrents from the command-line.
|
Downloads torrents from the command-line. |
torrentfs
Mounts a FUSE filesystem backed by torrents and magnet links.
|
Mounts a FUSE filesystem backed by torrents and magnet links. |
internal
|
|
Package iplist handles the P2P Plaintext Format described by https://en.wikipedia.org/wiki/PeerGuardian#P2P_plaintext_format.
|
Package iplist handles the P2P Plaintext Format described by https://en.wikipedia.org/wiki/PeerGuardian#P2P_plaintext_format. |
cmd/pack-blocklist
Takes P2P blocklist text format in stdin, and outputs the packed format from the iplist package.
|
Takes P2P blocklist text format in stdin, and outputs the packed format from the iplist package. |
Package logonce implements an io.Writer facade that only performs distinct writes.
|
Package logonce implements an io.Writer facade that only performs distinct writes. |
Package storage implements storage backends for package torrent.
|
Package storage implements storage backends for package torrent. |
util
|
|
dirwatch
Package dirwatch provides filesystem-notification based tracking of torrent info files and magnet URIs in a directory.
|
Package dirwatch provides filesystem-notification based tracking of torrent info files and magnet URIs in a directory. |