storage

package
v0.0.0-...-384af84 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 24, 2023 License: BSD-2-Clause Imports: 12 Imported by: 0

Documentation

Overview

Package storage provides abstractions and implementations for storing and retrieving peers from their announced infohashes.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// PromGCDurationMilliseconds is a histogram used by storage to record the
	// durations of execution time required for removing expired peers.
	PromGCDurationMilliseconds = prometheus.NewHistogram(prometheus.HistogramOpts{
		Name:    "chihaya_storage_gc_duration_milliseconds",
		Help:    "The time it takes to perform storage garbage collection",
		Buckets: prometheus.ExponentialBuckets(9.375, 2, 10),
	})

	// PromInfohashesCount is a gauge used to hold the current total amount of
	// unique swarms being tracked by a storage.
	PromInfohashesCount = prometheus.NewGauge(prometheus.GaugeOpts{
		Name: "chihaya_storage_infohashes_count",
		Help: "The number of Infohashes tracked",
	})

	// PromSeedersCount is a gauge used to hold the current total amount of
	// unique seeders per swarm.
	PromSeedersCount = prometheus.NewGauge(prometheus.GaugeOpts{
		Name: "chihaya_storage_seeders_count",
		Help: "The number of seeders tracked",
	})

	// PromLeechersCount is a gauge used to hold the current total amount of
	// unique leechers per swarm.
	PromLeechersCount = prometheus.NewGauge(prometheus.GaugeOpts{
		Name: "chihaya_storage_leechers_count",
		Help: "The number of leechers tracked",
	})
)
View Source 
var ErrDriverDoesNotExist = errors.New("peer store driver with that name does not exist")

ErrDriverDoesNotExist is the error returned by NewPeerStore when a peer store driver with that name does not exist.

View Source 
var ErrResourceDoesNotExist = bittorrent.ClientError("resource does not exist")

ErrResourceDoesNotExist is the error returned by all delete methods and the AnnouncePeers method of the PeerStore interface if the requested resource does not exist.

View Source 
var PeerEqualityFunc = func(p1, p2 bittorrent.Peer) bool { return p1.Equal(p2) }

PeerEqualityFunc is the boolean function to use to check two Peers for equality. Depending on the implementation of the PeerStore, this can be changed to use (Peer).EqualEndpoint instead.

Functions

func AnnounceLeecher 

func AnnounceLeecher(b *testing.B, ps PeerStore)

AnnounceLeecher benchmarks the AnnouncePeers method of a PeerStore for announcing a leecher. The swarm announced to has 500 seeders and 500 leechers.

AnnounceLeecher can run in parallel.

func AnnounceLeecher1kInfohash 

func AnnounceLeecher1kInfohash(b *testing.B, ps PeerStore)

AnnounceLeecher1kInfohash behaves like AnnounceLeecher with one of 1000 infohashes.

AnnounceLeecher1kInfohash can run in parallel.

func AnnounceSeeder 

func AnnounceSeeder(b *testing.B, ps PeerStore)

AnnounceSeeder behaves like AnnounceLeecher with a seeder instead of a leecher.

AnnounceSeeder can run in parallel.

func AnnounceSeeder1kInfohash 

func AnnounceSeeder1kInfohash(b *testing.B, ps PeerStore)

AnnounceSeeder1kInfohash behaves like AnnounceSeeder with one of 1000 infohashes.

AnnounceSeeder1kInfohash can run in parallel.

func DeleteNonexist 

func DeleteNonexist(b *testing.B, ps PeerStore)

DeleteNonexist benchmarks the DeleteSeeder method of a PeerStore by attempting to delete a Peer that is nonexistent.

DeleteNonexist can run in parallel.

func DeleteNonexist1k 

func DeleteNonexist1k(b *testing.B, ps PeerStore)

DeleteNonexist1k benchmarks the DeleteSeeder method of a PeerStore by attempting to delete one of 1000 nonexistent Peers.

DeleteNonexist can run in parallel.

func DeleteNonexist1kInfohash 

func DeleteNonexist1kInfohash(b *testing.B, ps PeerStore)

DeleteNonexist1kInfohash benchmarks the DeleteSeeder method of a PeerStore by attempting to delete one Peer from one of 1000 infohashes.

DeleteNonexist1kInfohash can run in parallel.

func DeleteNonexist1kInfohash1k 

func DeleteNonexist1kInfohash1k(b *testing.B, ps PeerStore)

DeleteNonexist1kInfohash1k benchmarks the Delete method of a PeerStore by attempting to delete one of 1000 Peers from one of 1000 Infohashes.

DeleteNonexist1kInfohash1k can run in parallel.

func GradNonexist 

func GradNonexist(b *testing.B, ps PeerStore)

GradNonexist benchmarks the GraduateLeecher method of a PeerStore by attempting to graduate a nonexistent Peer.

GradNonexist can run in parallel.

func GradNonexist1k 

func GradNonexist1k(b *testing.B, ps PeerStore)

GradNonexist1k benchmarks the GraduateLeecher method of a PeerStore by attempting to graduate one of 1000 nonexistent Peers.

GradNonexist1k can run in parallel.

func GradNonexist1kInfohash 

func GradNonexist1kInfohash(b *testing.B, ps PeerStore)

GradNonexist1kInfohash benchmarks the GraduateLeecher method of a PeerStore by attempting to graduate a nonexistent Peer for one of 100 Infohashes.

GradNonexist1kInfohash can run in parallel.

func GradNonexist1kInfohash1k 

func GradNonexist1kInfohash1k(b *testing.B, ps PeerStore)

GradNonexist1kInfohash1k benchmarks the GraduateLeecher method of a PeerStore by attempting to graduate one of 1000 nonexistent Peers for one of 1000 infohashes.

GradNonexist1kInfohash1k can run in parallel.

func Nop 

func Nop(b *testing.B, ps PeerStore)

Nop executes a no-op for each iteration. It should produce the same results for each PeerStore. This can be used to get an estimate of the impact of the benchmark harness on benchmark results and an estimate of the general performance of the system benchmarked on.

Nop can run in parallel.

func Put 

func Put(b *testing.B, ps PeerStore)

Put benchmarks the PutSeeder method of a PeerStore by repeatedly Putting the same Peer for the same InfoHash.

Put can run in parallel.

func Put1k 

func Put1k(b *testing.B, ps PeerStore)

Put1k benchmarks the PutSeeder method of a PeerStore by cycling through 1000 Peers and Putting them into the swarm of one infohash.

Put1k can run in parallel.

func Put1kInfohash 

func Put1kInfohash(b *testing.B, ps PeerStore)

Put1kInfohash benchmarks the PutSeeder method of a PeerStore by cycling through 1000 infohashes and putting the same peer into their swarms.

Put1kInfohash can run in parallel.

func Put1kInfohash1k 

func Put1kInfohash1k(b *testing.B, ps PeerStore)

Put1kInfohash1k benchmarks the PutSeeder method of a PeerStore by cycling through 1000 infohashes and 1000 Peers and calling Put with them.

Put1kInfohash1k can run in parallel.

func PutDelete 

func PutDelete(b *testing.B, ps PeerStore)

PutDelete benchmarks the PutSeeder and DeleteSeeder methods of a PeerStore by calling PutSeeder followed by DeleteSeeder for one Peer and one infohash.

PutDelete can not run in parallel.

func PutDelete1k 

func PutDelete1k(b *testing.B, ps PeerStore)

PutDelete1k benchmarks the PutSeeder and DeleteSeeder methods in the same way PutDelete does, but with one from 1000 Peers per iteration.

PutDelete1k can not run in parallel.

func PutDelete1kInfohash 

func PutDelete1kInfohash(b *testing.B, ps PeerStore)

PutDelete1kInfohash behaves like PutDelete1k with 1000 infohashes instead of 1000 Peers.

PutDelete1kInfohash can not run in parallel.

func PutDelete1kInfohash1k 

func PutDelete1kInfohash1k(b *testing.B, ps PeerStore)

PutDelete1kInfohash1k behaves like PutDelete1k with 1000 infohashes in addition to 1000 Peers.

PutDelete1kInfohash1k can not run in parallel.

func PutGradDelete 

func PutGradDelete(b *testing.B, ps PeerStore)

PutGradDelete benchmarks the PutLeecher, GraduateLeecher and DeleteSeeder methods of a PeerStore by adding one leecher to a swarm, promoting it to a seeder and deleting the seeder.

PutGradDelete can not run in parallel.

func PutGradDelete1k 

func PutGradDelete1k(b *testing.B, ps PeerStore)

PutGradDelete1k behaves like PutGradDelete with one of 1000 Peers.

PutGradDelete1k can not run in parallel.

func PutGradDelete1kInfohash 

func PutGradDelete1kInfohash(b *testing.B, ps PeerStore)

PutGradDelete1kInfohash behaves like PutGradDelete with one of 1000 infohashes.

PutGradDelete1kInfohash can not run in parallel.

func PutGradDelete1kInfohash1k 

func PutGradDelete1kInfohash1k(b *testing.B, ps PeerStore)

PutGradDelete1kInfohash1k behaves like PutGradDelete with one of 1000 Peers and one of 1000 infohashes.

PutGradDelete1kInfohash can not run in parallel.

func RegisterDriver 

func RegisterDriver(name string, d Driver)

RegisterDriver makes a Driver available by the provided name.

If called twice with the same name, the name is blank, or if the provided Driver is nil, this function panics.

func ScrapeSwarm 

func ScrapeSwarm(b *testing.B, ps PeerStore)

ScrapeSwarm benchmarks the ScrapeSwarm method of a PeerStore. The swarm scraped has 500 seeders and 500 leechers.

ScrapeSwarm can run in parallel.

func ScrapeSwarm1kInfohash 

func ScrapeSwarm1kInfohash(b *testing.B, ps PeerStore)

ScrapeSwarm1kInfohash behaves like ScrapeSwarm with one of 1000 infohashes.

ScrapeSwarm1kInfohash can run in parallel.

func TestPeerStore 

func TestPeerStore(t *testing.T, p PeerStore)

TestPeerStore tests a PeerStore implementation against the interface.

Types

type Driver 

type Driver interface {
	NewPeerStore(cfg interface{}) (PeerStore, error)
}

Driver is the interface used to initialize a new type of PeerStore.

type PeerStore 

type PeerStore interface {
	// PutSeeder adds a Seeder to the Swarm identified by the provided
	// InfoHash.
	PutSeeder(infoHash bittorrent.InfoHash, p bittorrent.Peer) error

	// DeleteSeeder removes a Seeder from the Swarm identified by the
	// provided InfoHash.
	//
	// If the Swarm or Peer does not exist, this function returns
	// ErrResourceDoesNotExist.
	DeleteSeeder(infoHash bittorrent.InfoHash, p bittorrent.Peer) error

	// PutLeecher adds a Leecher to the Swarm identified by the provided
	// InfoHash.
	// If the Swarm does not exist already, it is created.
	PutLeecher(infoHash bittorrent.InfoHash, p bittorrent.Peer) error

	// DeleteLeecher removes a Leecher from the Swarm identified by the
	// provided InfoHash.
	//
	// If the Swarm or Peer does not exist, this function returns
	// ErrResourceDoesNotExist.
	DeleteLeecher(infoHash bittorrent.InfoHash, p bittorrent.Peer) error

	// GraduateLeecher promotes a Leecher to a Seeder in the Swarm
	// identified by the provided InfoHash.
	//
	// If the given Peer is not present as a Leecher or the swarm does not exist
	// already, the Peer is added as a Seeder and no error is returned.
	GraduateLeecher(infoHash bittorrent.InfoHash, p bittorrent.Peer) error

	// AnnouncePeers is a best effort attempt to return Peers from the Swarm
	// identified by the provided InfoHash.
	// The numWant parameter indicates the number of peers requested by the
	// announcing Peer p. The seeder flag determines whether the Peer announced
	// as a Seeder.
	// The returned Peers are required to be either all IPv4 or all IPv6.
	//
	// The returned Peers should strive be:
	// - as close to length equal to numWant as possible without going over
	// - all IPv4 or all IPv6 depending on the provided peer
	// - if seeder is true, should ideally return more leechers than seeders
	// - if seeder is false, should ideally return more seeders than
	//   leechers
	//
	// Returns ErrResourceDoesNotExist if the provided InfoHash is not tracked.
	AnnouncePeers(infoHash bittorrent.InfoHash, seeder bool, numWant int, p bittorrent.Peer) (peers []bittorrent.Peer, err error)

	// ScrapeSwarm returns information required to answer a Scrape request
	// about a Swarm identified by the given InfoHash.
	// The AddressFamily indicates whether or not the IPv6 swarm should be
	// scraped.
	// The Complete and Incomplete fields of the Scrape must be filled,
	// filling the Snatches field is optional.
	//
	// If the Swarm does not exist, an empty Scrape and no error is returned.
	ScrapeSwarm(infoHash bittorrent.InfoHash, addressFamily bittorrent.AddressFamily) bittorrent.Scrape

	// stop.Stopper is an interface that expects a Stop method to stop the
	// PeerStore.
	// For more details see the documentation in the stop package.
	stop.Stopper

	// log.Fielder returns a loggable version of the data used to configure and
	// operate a particular PeerStore.
	log.Fielder
}

PeerStore is an interface that abstracts the interactions of storing and manipulating Peers such that it can be implemented for various data stores.

Implementations of the PeerStore interface must do the following in addition to implementing the methods of the interface in the way documented:

  • Implement a garbage-collection strategy that ensures stale data is removed. For example, a timestamp on each InfoHash/Peer combination can be used to track the last activity for that Peer. The entire database can then be scanned periodically and too old Peers removed. The intervals and durations involved should be configurable.
  • IPv4 and IPv6 swarms must be isolated from each other. A PeerStore must be able to transparently handle IPv4 and IPv6 Peers, but must separate them. AnnouncePeers and ScrapeSwarm must return information about the Swarm matching the given AddressFamily only.

Implementations can be tested against this interface using the tests in storage_tests.go and the benchmarks in storage_bench.go.

func NewPeerStore 

func NewPeerStore(name string, cfg interface{}) (ps PeerStore, err error)

NewPeerStore attempts to initialize a new PeerStore instance from the list of registered Drivers.

If a driver does not exist, returns ErrDriverDoesNotExist.

Source Files

View all Source files

Directories

Path Synopsis
Package memory implements the storage interface for a Chihaya BitTorrent tracker keeping peer data in memory.
 Package memory implements the storage interface for a Chihaya BitTorrent tracker keeping peer data in memory.
Package redis implements the storage interface for a Chihaya BitTorrent tracker keeping peer data in redis with hash.
 Package redis implements the storage interface for a Chihaya BitTorrent tracker keeping peer data in redis with hash.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.