p2plab

package module
v0.0.0-...-300ac43 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2020 License: Apache-2.0 Imports: 8 Imported by: 0

README

p2plab

Build Status GoDoc

p2plab is infrastructure to benchmark IPFS throughput in reproducible and quantifiable way.

asciicast

Key features:

  • IPFS infrastructure as code
  • Cluster-agnostic benchmarking scenarios
  • Live update IPFS infrastructure to different commit
  • Distributed tracing

Getting started

By default, p2plab runs with a in-memory driver and can deploy a cluster of IPFS nodes as subprocesses.

First, compile and run labd, the main daemon orchestrating p2plab:

export GO111MODULE=on
go get -u github.com/Netflix/p2plab/cmd/labd
labd

In a new terminal, compile labctl, the CLI to manage the infrastructure and run benchmarks:

export GO111MODULE=on
go get -u github.com/Netflix/p2plab/cmd/labctl

Now you can create your first local cluster using one of the examples:

$ labctl cluster create --definition ./examples/cluster/same-region.json my-cluster
6:52PM INF Creating node group name=my-cluster
6:52PM INF Updating metadata with new nodes name=my-cluster
6:52PM INF Waiting for healthy nodes name=my-cluster
6:52PM INF Updating cluster metadata name=my-cluster
6:52PM INF Created cluster "my-cluster"
my-cluster

$ labctl node ls my-cluster
+----------------------+-----------+--------------+---------------------------------------------------+----------------+----------------+
|          ID          |  ADDRESS  | GITREFERENCE |                      LABELS                       |   CREATEDAT    |   UPDATEDAT    |
+----------------------+-----------+--------------+---------------------------------------------------+----------------+----------------+
| bp3ept7ic6vdctur3dag | 127.0.0.1 | HEAD         | bp3ept7ic6vdctur3dag,t2.micro,us-west-2           | 20 seconds ago | 20 seconds ago |
| bp3eptfic6vdctur3db0 | 127.0.0.1 | HEAD         | bp3eptfic6vdctur3db0,neighbors,t2.micro,us-west-2 | 20 seconds ago | 20 seconds ago |
| bp3eptvic6vdctur3dbg | 127.0.0.1 | HEAD         | bp3eptvic6vdctur3dbg,neighbors,t2.micro,us-west-2 | 20 seconds ago | 20 seconds ago |
+----------------------+-----------+--------------+---------------------------------------------------+----------------+----------------+

In the labels column, notice how two out of three of the nodes have the label neighbors. This will come in handy when running our benchmark. Benchmarks are executed from scenarios, and scenarios are decoupled from the cluster we benchmark because they operate on labels. Whether we're running in us-west-1 or us-east-2, or our cluster has 3 or 50 nodes, you can still execute the same scenario given that the appropriate nodes are labelled.

Let's create our first scenario using one of the examples:

$ labctl scenario create ./examples/scenario/neighbors.json
6:59PM INF Created scenario "neighbors"
neighbors

$ labctl scenario inspect neighbors
{
    "ID": "neighbors",
    "Definition": {
        "objects": {
            "golang": {
                "type": "oci",
                "source": "docker.io/library/golang:latest",
                "layout": "",
                "chunker": "",
                "rawLeaves": false,
                "hashFunc": "",
                "maxLinks": 0
            }
        },
        "seed": {
            "neighbors": "golang"
        },
        "benchmark": {
            "(not 'neighbors')": "golang"
        }
    },
    "Labels": [
        "neighbors"
    ],
    "CreatedAt": "2020-02-14T18:59:44.698063252Z",
    "UpdatedAt": "2020-02-14T18:59:44.698063252Z"
}

When we finally run our benchmark, labd will download the objects in the scenario, in this case the golang OCI image and convert it into a IPFS DAG. Then it will follow the seed stage and distribute the object golang to nodes matching the label neighbors. The benchmark will then measure how long it takes for nodes that don't match the label neighbors with the object golang.

$ labctl benchmark create my-cluster neighbors
7:02PM INF Retrieving nodes in cluster bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Resolving git references bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Building p2p app(s) bid=my-cluster-neighbors-1581706936119660719 commits=["5f7c8e0d9104c76974db9640c05beec429f56e36"]
7:02PM INF Updating cluster bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Retrieving peer infos bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Connecting cluster bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Creating scenario plan bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Transforming objects into IPLD DAGs bid=my-cluster-neighbors-1581706936119660719
7:02PM INF Resolving OCI reference bid=my-cluster-neighbors-1581706936119660719 source=docker.io/library/golang:latest
7:02PM INF Resolved reference to digest bid=my-cluster-neighbors-1581706936119660719 digest=sha256:9295ba678e3764d79ac0aeabdbcf281a91933c81c8de29387d8a2f557e256cdb source=docker.io/library/golang:latest
7:02PM INF Converting manifest recursively to IPLD DAG bid=my-cluster-neighbors-1581706936119660719 digest=sha256:9295ba678e3764d79ac0aeabdbcf281a91933c81c8de29387d8a2f557e256cdb
7:03PM INF Constructing Unixfs directory over manifest blobs bid=my-cluster-neighbors-1581706936119660719 target=sha256:29c7ea58b504cee59a6f4e442867151f0763be246d5c9d06f499ac841118f93f
7:03PM INF Planning scenario seed bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Planning scenario benchmark bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Creating benchmark metadata bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Executing scenario plan bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Seeding cluster bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Seeding completed bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Starting a session for benchmarking bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Waiting for healthy nodes bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Retrieving peer infos bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Connecting cluster bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Benchmarking cluster bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Benchmark completed bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Retrieving reports bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Ending the session bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Updating benchmark metadata bid=my-cluster-neighbors-1581706936119660719
7:03PM INF Completed benchmark "my-cluster-neighbors-1581706936119660719"
# Summary
Total time: 4 seconds 971 milliseconds
Trace:

# Bandwidth
+-------------------+----------------------+---------+----------+----------+----------+
|       QUERY       |         NODE         | TOTALIN | TOTALOUT |  RATEIN  | RATEOUT  |
+-------------------+----------------------+---------+----------+----------+----------+
| (not 'neighbors') | bp3ept7ic6vdctur3dag | 397 MB  |  204 kB  | 106 MB/s | 54 kB/s  |
+-------------------+----------------------+---------+----------+----------+----------+
|         -         | bp3eptfic6vdctur3db0 |  96 kB  |  188 MB  | 26 kB/s  | 52 MB/s  |
+                   +----------------------+---------+----------+----------+----------+
|                   | bp3eptvic6vdctur3dbg | 109 kB  |  212 MB  | 28 kB/s  | 55 MB/s  |
+-------------------+----------------------+---------+----------+----------+----------+
|                            TOTAL         | 397 MB  |  400 MB  | 106 MB/s | 107 MB/s |
+-------------------+----------------------+---------+----------+----------+----------+

# Bitswap
+-------------------+----------------------+------------+------------+-----------+----------+----------+---------+
|       QUERY       |         NODE         | BLOCKSRECV | BLOCKSSENT | DUPBLOCKS | DATARECV | DATASENT | DUPDATA |
+-------------------+----------------------+------------+------------+-----------+----------+----------+---------+
| (not 'neighbors') | bp3ept7ic6vdctur3dag |   1,866    |     0      |     2     |  481 MB  |   0 B    | 263 kB  |
+-------------------+----------------------+------------+------------+-----------+----------+----------+---------+
|         -         | bp3eptfic6vdctur3db0 |     0      |    888     |     0     |   0 B    |  228 MB  |   0 B   |
+                   +----------------------+            +------------+           +          +----------+         +
|                   | bp3eptvic6vdctur3dbg |            |    978     |           |          |  252 MB  |         |
+-------------------+----------------------+------------+------------+-----------+----------+----------+---------+
|                            TOTAL         |   1,866    |   1,866    |     2     |  481 MB  |  481 MB  | 263 kB  |
+-------------------+----------------------+------------+------------+-----------+----------+----------+---------+

Well done! You've ran your first benchmark and transferred a container image over IPFS.

Live updating the cluster

Now you have a control group, we may want to compare it against a different configuration of IPFS. For example, let's compare the TCP vs QUIC transport of libp2p.

First, let's find out what the default configuration of a node is:

labctl node inspect my-cluster bp3ept7ic6vdctur3dag
{
    "ID": "bp3ept7ic6vdctur3dag",
    "Address": "127.0.0.1",
    "AgentPort": 42509,
    "AppPort": 37727,
    "Peer": {
        "GitReference": "HEAD",
        "Transports": [
            "tcp"
        ],
        "Muxers": [
            "mplex"
        ],
        "SecurityTransports": [
            "secio"
        ],
        "Routing": "nil"
    },
    "Labels": [
        "bp3ept7ic6vdctur3dag",
        "t2.micro",
        "us-west-2"
    ],
    "CreatedAt": "2020-02-14T18:59:44.698063252Z",
    "UpdatedAt": "2020-02-14T18:59:44.698063252Z"
}

By default, the libp2p hosts are created with tcp+secio and muxed using mplex. Since QUIC already has TLS, we want to update tcp to quic and not use secio.

labctl node update --transports quic --security-transports "" my-cluster

Now, all our nodes are now using quic:

labctl --output json node ls my-cluster | jq '.[].Peer'
{
  "GitReference": "HEAD",
  "Transports": [
    "quic"
  ],
  "Muxers": [
    "mplex"
  ],
  "SecurityTransports": null,
  "Routing": "nil"
}
{
  "GitReference": "HEAD",
  "Transports": [
    "quic"
  ],
  "Muxers": [
    "mplex"
  ],
  "SecurityTransports": null,
  "Routing": "nil"
}
{
  "GitReference": "HEAD",
  "Transports": [
    "quic"
  ],
  "Muxers": [
    "mplex"
  ],
  "SecurityTransports": null,
  "Routing": "nil"
}

And then run our benchmark again to see how it compares to tcp+secio:

labctl benchmark create my-cluster neighbors

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Action 

type Action interface {
	String() string

	Tasks(ctx context.Context, ns []Node) (map[string]metadata.Task, error)
}

type AddOption 

type AddOption func(*AddSettings) error

AddOption is an option for AddSettings.

func WithChunker 

func WithChunker(chunker string) AddOption

WithChunker sets the chunking strategy for the content.

func WithHashFunc 

func WithHashFunc(hashFunc string) AddOption

WithHashFunc sets the hashing function for the blocks.

func WithLayout 

func WithLayout(layout string) AddOption

WithLayout sets the format for DAG generation. 

func WithMaxLinks(maxLinks int) AddOption

WithMaxLinks sets the maximum children each block can have.

func WithRawLeaves 

func WithRawLeaves(rawLeaves bool) AddOption

WithRawLeaves sets whether to use raw blocks for leaf nodes.

type AddSettings 

type AddSettings struct {
	Layout    string
	Chunker   string
	RawLeaves bool
	Hidden    bool
	Shard     bool
	NoCopy    bool
	HashFunc  string
	MaxLinks  int
}

AddSettings describe the settings for adding content to the peer.

type AgentAPI 

type AgentAPI interface {
	Healthcheck(ctx context.Context) bool

	Update(ctx context.Context, id, link string, pdef metadata.PeerDefinition) error

	// SSH creates a SSH connection to the node.
	SSH(ctx context.Context, opts ...SSHOption) error
}

type AppAPI 

type AppAPI interface {
	PeerInfo(ctx context.Context) (peer.AddrInfo, error)

	Report(ctx context.Context) (metadata.ReportNode, error)

	// Run executes an task on the node.
	Run(ctx context.Context, task metadata.Task) error
}

type Benchmark 

type Benchmark interface {
	Labeled

	Metadata() metadata.Benchmark

	Report(ctx context.Context) (metadata.Report, error)
}

Benchmark is an execution of a scenario on a cluster.

type BenchmarkAPI 

type BenchmarkAPI interface {
	// Create creates a benchmark of a scenario on a cluster.
	Create(ctx context.Context, cluster, scenario string, opts ...StartBenchmarkOption) (id string, err error)

	// Get returns a benchmark.
	Get(ctx context.Context, id string) (Benchmark, error)

	Label(ctx context.Context, ids, adds, removes []string) ([]Benchmark, error)

	// List returns available benchmarks.
	List(ctx context.Context, opts ...ListOption) ([]Benchmark, error)

	Remove(ctx context.Context, ids ...string) error
}

BenchmarkAPI defines API for benchmark operations.

type Build 

type Build interface {
	// ID returns a uniquely identifiable string.
	ID() string

	Metadata() metadata.Build

	// Open creates a reader for the build's binary.
	Open(ctx context.Context) (io.ReadCloser, error)
}

Build is an compiled peer ready to be deployed.

type BuildAPI 

type BuildAPI interface {
	// Get returns a build.
	Get(ctx context.Context, id string) (Build, error)

	// List returns available builds.
	List(ctx context.Context) ([]Build, error)

	// Upload uploads a binary for a build.
	Upload(ctx context.Context, r io.Reader) (Build, error)
}

BuildAPI defines the API for build operations.

type Builder 

type Builder interface {
	// Init initializes the builder.
	Init(ctx context.Context) error

	// Resolve resolves a git-ref to a commit it can uniquely build.
	Resolve(ctx context.Context, ref string) (commit string, err error)

	// Build compiles the peer at the given commit.
	Build(ctx context.Context, commit string) (link string, err error)
}

Builder compiles the peer to hotswap the underlying implementation on a live cluster.

type Cluster 

type Cluster interface {
	Labeled

	Metadata() metadata.Cluster

	// Update updates nodes in a cluster matching the list options with the given
	// peer definition.
	Update(ctx context.Context, pdef metadata.PeerDefinition, opts ...ListOption) ([]Node, error)
}

Cluster is a group of instances connected in a p2p network. They can be provisioned by developers, or CI. Clusters may span multiple regions and have heterogeneous nodes.

type ClusterAPI 

type ClusterAPI interface {
	// Create deploys a cluster.
	Create(ctx context.Context, name string, opts ...CreateClusterOption) (id string, err error)

	// Get returns a cluster.
	Get(ctx context.Context, name string) (Cluster, error)

	// Label adds/removes labels to/from clusters.
	Label(ctx context.Context, names, adds, removes []string) ([]Cluster, error)

	// List returns available clusters.
	List(ctx context.Context, opts ...ListOption) ([]Cluster, error)

	// Remove destroys clusters permanently.
	Remove(ctx context.Context, names ...string) error
}

ClusterAPI defines API for cluster operations.

type ControlAPI 

type ControlAPI interface {
	// Cluster returns an implementaiton of Cluster API.
	Cluster() ClusterAPI

	// Node returns an implementation of Node API.
	Node() NodeAPI

	// Scenario returns an implementation of Scenario API.
	Scenario() ScenarioAPI

	// Benchmark returns an implementation of Benchmark API.
	Benchmark() BenchmarkAPI

	// Experiment returns an implementation of Experiment API.
	Experiment() ExperimentAPI

	// Build returns an implementation of Build API.
	Build() BuildAPI
}

ControlAPI defines APIs for labd.

type CreateClusterOption 

type CreateClusterOption func(*CreateClusterSettings) error

CreateClusterOption is an option to modify create cluster settings.

func WithClusterDefinition 

func WithClusterDefinition(definition string) CreateClusterOption

func WithClusterInstanceType 

func WithClusterInstanceType(instanceType string) CreateClusterOption

func WithClusterRegion 

func WithClusterRegion(region string) CreateClusterOption

func WithClusterSize 

func WithClusterSize(size int) CreateClusterOption

type CreateClusterSettings 

type CreateClusterSettings struct {
	Definition        string
	Size              int
	InstanceType      string
	Region            string
	ClusterDefinition metadata.ClusterDefinition
}

CreateClusterSettings specify cluster properties for creation.

type Downloader 

type Downloader interface {
	// Download downloads an artifact with an abstract link.
	Download(ctx context.Context, link string) (io.ReadCloser, error)
}

Download downlaods artifacts from an external distribution mechanism.

type Experiment 

type Experiment interface {
	Labeled

	Metadata() metadata.Experiment
}

type ExperimentAPI 

type ExperimentAPI interface {
	Create(ctx context.Context, name string, edef metadata.ExperimentDefinition) (id string, err error)

	Get(ctx context.Context, id string) (Experiment, error)

	Label(ctx context.Context, ids, adds, removes []string) ([]Experiment, error)

	List(ctx context.Context, opts ...ListOption) ([]Experiment, error)

	Remove(ctx context.Context, ids ...string) error
}

ExperimentAPI is an unimplemented layer to run experiments, a collection of benchmarks while varying some aspect.

type Labeled 

type Labeled interface {
	// ID returns a uniquely identifiable string.
	ID() string

	// Labels returns a unique list of labels.
	Labels() []string
}

Labeled defines a resource that has labels.

type LabeledSet 

type LabeledSet interface {
	// Add adds a labeled resource to the set.
	Add(labeled Labeled)

	// Remove removes a labeled resource from the set.
	Remove(id string)

	// Get returns a labeled resource from the set.
	Get(id string) Labeled

	// Contains returns whether a labeled resource with the id exists in the set.
	Contains(id string) bool

	// Slice returns the labeled resources as a slice.
	Slice() []Labeled
}

LabeledSet is a set of labeled resources, duplicate resources are detected by the ID of the labeled resource.

type ListOption 

type ListOption func(*ListSettings) error

func WithQuery 

func WithQuery(q string) ListOption

type ListSettings 

type ListSettings struct {
	Query string
}

type Node 

type Node interface {
	Labeled

	AgentAPI

	AppAPI

	Metadata() metadata.Node
}

Node is an instance running the P2P application to be benchmarked.

type NodeAPI 

type NodeAPI interface {
	// Get returns a node.
	Get(ctx context.Context, cluster, id string) (Node, error)

	Label(ctx context.Context, cluster string, ids, adds, removes []string) ([]Node, error)

	List(ctx context.Context, cluster string, opts ...ListOption) ([]Node, error)
}

NodeAPI defines the API for node operations.

type NodeGroup 

type NodeGroup struct {
	ID    string
	Nodes []metadata.Node
}

NodeGroup is a cluster of nodes.

type NodeProvider 

type NodeProvider interface {
	// CreateNodeGroup returns a healthy cluster of nodes.
	CreateNodeGroup(ctx context.Context, id string, cdef metadata.ClusterDefinition) (*NodeGroup, error)

	// DestroyNodeGroup destroys a cluster of nodes.
	DestroyNodeGroup(ctx context.Context, ng *NodeGroup) error
}

NodeProvider is a service that can provision nodes.

type Peer 

type Peer interface {
	// Host returns the libp2p host.
	Host() host.Host

	// DAGService returns the IPLD DAG service.
	DAGService() ipld.DAGService

	// Connect connects to the libp2p peers.
	Connect(ctx context.Context, infos []peer.AddrInfo) error

	// Disconnect disconnects from libp2p peers.
	Disconnect(ctx context.Context, infos []peer.AddrInfo) error

	// Add adds content from an io.Reader into the Peer's storage.
	Add(ctx context.Context, r io.Reader, opts ...AddOption) (ipld.Node, error)

	// Get returns an Unixfsv1 file from a given cid.
	Get(ctx context.Context, c cid.Cid) (files.Node, error)

	// FetchGraph fetches the full DAG rooted at a given cid.
	FetchGraph(ctx context.Context, c cid.Cid) error

	// Report returns all the metrics collected from the peer.
	Report(ctx context.Context) (metadata.ReportNode, error)
}

Peer is a minimal IPFS node that can distribute IPFS DAGs.

type Query 

type Query interface {
	// String returns the original query.
	String() string

	// Match returns the subset of lset that matches the query.
	Match(ctx context.Context, lset LabeledSet) (LabeledSet, error)
}

Query is an executable function against a cluster to match a set of nodes. Queries are used to group nodes to perform actions in either the seeding or benchmarking stage of a scenario.

type QueryOption 

type QueryOption func(*QuerySettings) error

func WithAddLabels 

func WithAddLabels(labels ...string) QueryOption

func WithRemoveLabels 

func WithRemoveLabels(labels ...string) QueryOption

type QuerySettings 

type QuerySettings struct {
	AddLabels    []string
	RemoveLabels []string
}

type SSHOption 

type SSHOption func(SSHSettings) error

SSHOption is an option to modify SSH settings.

type SSHSettings 

type SSHSettings struct {
}

SSHSetttings specify ssh settings when connecting to a node.

type Scenario 

type Scenario interface {
	Labeled

	Metadata() metadata.Scenario
}

Scenario is a schema for benchmarks that describes objects to benchmark, how the cluster is initially seeded, and what to benchmark.

type ScenarioAPI 

type ScenarioAPI interface {
	// Create saves a scenario for the given scenario definition.
	Create(ctx context.Context, name string, sdef metadata.ScenarioDefinition) (Scenario, error)

	// Get returns a scenario.
	Get(ctx context.Context, name string) (Scenario, error)

	// Label adds and removes labels from nodes identified by the list of names.
	Label(ctx context.Context, names, adds, removes []string) ([]Scenario, error)

	// List returns available scenarios.
	List(ctx context.Context, opts ...ListOption) ([]Scenario, error)

	Remove(ctx context.Context, names ...string) error
}

ScenarioAPI defines API for scenario operations.

type StartBenchmarkOption 

type StartBenchmarkOption func(*StartBenchmarkSettings) error

func WithBenchmarkNoReset 

func WithBenchmarkNoReset() StartBenchmarkOption

type StartBenchmarkSettings 

type StartBenchmarkSettings struct {
	NoReset bool
}

type Transformer 

type Transformer interface {
	// Transform adds a resource defined by source into an IPFS DAG stored in
	// peer.
	Transform(ctx context.Context, peer Peer, source string, opts ...AddOption) (cid.Cid, error)

	// Close releases any resources held by the Transformer.
	Close() error
}

Transformer defines a way to convert an external resource into IPFS DAGs.

type Uploader 

type Uploader interface {
	// Upload uploads artifacts to a registry.
	Upload(ctx context.Context, r io.Reader) (link string, err error)

	Close() error
}

Uploader uploads artifacts to an external distribution mechanism.

Source Files

View all Source files

Directories

Path Synopsis
cmd
labagent
labagent/command
labapp
labapp/command
labctl
labctl/command
labd
labd/command
ociadd
ociget
s3build
s3download
s3upload
cue
parser
Package parser enables parsing of cue sources
 Package parser enables parsing of cue sources
healthcheckrouter
filedownloader
httpdownloader
s3downloader
agentapi
agentrouter
supervisor
appapi
approuter
controlapi
routers/benchmarkrouter
routers/buildrouter
routers/clusterrouter
routers/experimentrouter
routers/helpers
Package helpers contains helper functions to be reused by multiple routers
 Package helpers contains helper functions to be reused by multiple routers
routers/noderouter
routers/scenariorouter
pkg
cliutil
digestconv
httputil
logutil
stringutil
traceutil
inmemory
terraform
oci
fileuploader
s3uploader

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.