testplanet

package
v0.34.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 26, 2020 License: AGPL-3.0 Imports: 88 Imported by: 6

Documentation

Overview

Package testplanet implements the full network wiring for testing

Index

Constants

This section is empty.

Variables

View Source 
var DisablePeerCAWhitelist = Reconfigure{
	Satellite: func(log *zap.Logger, index int, config *satellite.Config) {
		config.Server.UsePeerCAWhitelist = false
	},
	StorageNode: func(index int, config *storagenode.Config) {
		config.Server.UsePeerCAWhitelist = false
	},
}

DisablePeerCAWhitelist returns a `Reconfigure` that sets `UsePeerCAWhitelist` for all node types that use kademlia.

View Source 
var ReconfigureRS = func(minThreshold, repairThreshold, successThreshold, totalThreshold int) func(log *zap.Logger, index int, config *satellite.Config) {
	return func(log *zap.Logger, index int, config *satellite.Config) {
		config.Metainfo.RS.MinThreshold = minThreshold
		config.Metainfo.RS.RepairThreshold = repairThreshold
		config.Metainfo.RS.SuccessThreshold = successThreshold
		config.Metainfo.RS.TotalThreshold = totalThreshold
	}
}

ReconfigureRS returns function to change satellite redundancy scheme values

View Source 
var ShortenOnlineWindow = Reconfigure{
	Satellite: func(log *zap.Logger, index int, config *satellite.Config) {
		config.Overlay.Node.OnlineWindow = 1 * time.Second
	},
}

ShortenOnlineWindow returns a `Reconfigure` that sets the NodeSelection OnlineWindow to 1 second, meaning a connection failure leads to marking the nodes as offline

Functions

func Run 

func Run(t *testing.T, config Config, test func(t *testing.T, ctx *testcontext.Context, planet *Planet))

Run runs testplanet in multiple configurations.

Types

type Config 

type Config struct {
	SatelliteCount   int
	StorageNodeCount int
	UplinkCount      int

	IdentityVersion *storj.IDVersion
	Reconfigure     Reconfigure

	Name string
}

Config describes planet configuration

type DefaultReferralManagerServer added in v0.27.0 

type DefaultReferralManagerServer struct {
	// contains filtered or unexported fields
}

DefaultReferralManagerServer implements the default behavior of a mock referral manager

func (*DefaultReferralManagerServer) GetTokens added in v0.27.0 

func (server *DefaultReferralManagerServer) GetTokens(ctx context.Context, req *pb.GetTokensRequest) (*pb.GetTokensResponse, error)

GetTokens implements a mock GetTokens endpoint that returns a number of referral tokens. By default, it returns 0 tokens.

func (*DefaultReferralManagerServer) RedeemToken added in v0.27.0 

func (server *DefaultReferralManagerServer) RedeemToken(ctx context.Context, req *pb.RedeemTokenRequest) (*pb.RedeemTokenResponse, error)

RedeemToken implements a mock RedeemToken endpoint.

func (*DefaultReferralManagerServer) SetTokenCount added in v0.27.0 

func (server *DefaultReferralManagerServer) SetTokenCount(tokenCount int)

SetTokenCount sets the number of tokens GetTokens endpoint should return.

type Peer 

type Peer interface {
	ID() storj.NodeID
	Addr() string
	URL() storj.NodeURL
	Local() overlay.NodeDossier

	Run(context.Context) error
	Close() error
}

Peer represents one of StorageNode or Satellite

type Planet 

type Planet struct {
	VersionControl *versioncontrol.Peer
	Satellites     []*SatelliteSystem
	StorageNodes   []*storagenode.Peer
	Uplinks        []*Uplink

	ReferralManager *server.Server
	// contains filtered or unexported fields
}

Planet is a full storj system setup.

func NewCustom 

func NewCustom(log *zap.Logger, config Config) (*Planet, error)

NewCustom creates a new full system with the specified configuration.

func (*Planet) Identities 

func (planet *Planet) Identities() *testidentity.Identities

Identities returns the identity provider for this planet.

func (*Planet) NewIdentity 

func (planet *Planet) NewIdentity() (*identity.FullIdentity, error)

NewIdentity creates a new identity for a node

func (*Planet) NewListener 

func (planet *Planet) NewListener() (net.Listener, error)

NewListener creates a new listener

func (*Planet) NewVersionConfig 

func (planet *Planet) NewVersionConfig() checker.Config

NewVersionConfig returns the Version Config for this planet with tuned metrics.

func (*Planet) NewVersionInfo 

func (planet *Planet) NewVersionInfo() version.Info

NewVersionInfo returns the Version Info for this planet with tuned metrics.

func (*Planet) Shutdown 

func (planet *Planet) Shutdown() error

Shutdown shuts down all the nodes and deletes temporary directories.

func (*Planet) Size 

func (planet *Planet) Size() int

Size returns number of nodes in the network

func (*Planet) Start 

func (planet *Planet) Start(ctx context.Context)

Start starts all the nodes.

func (*Planet) StopPeer 

func (planet *Planet) StopPeer(peer Peer) error

StopPeer stops a single peer in the planet

func (*Planet) WriteWhitelist 

func (planet *Planet) WriteWhitelist(version storj.IDVersion) (string, error)

WriteWhitelist writes the pregenerated signer's CA cert to a "CA whitelist", PEM-encoded.

type Reconfigure 

type Reconfigure struct {
	NewSatelliteDB        func(log *zap.Logger, index int) (satellite.DB, error)
	NewSatellitePointerDB func(log *zap.Logger, index int) (metainfo.PointerDB, error)
	Satellite             func(log *zap.Logger, index int, config *satellite.Config)
	ReferralManagerServer func(log *zap.Logger) pb.ReferralManagerServer

	NewStorageNodeDB func(index int, db storagenode.DB, log *zap.Logger) (storagenode.DB, error)
	StorageNode      func(index int, config *storagenode.Config)
	UniqueIPCount    int

	Identities func(log *zap.Logger, version storj.IDVersion) *testidentity.Identities
}

Reconfigure allows to change node configurations

type SatelliteSystem 

type SatelliteSystem struct {
	Core     *satellite.Core
	API      *satellite.API
	Repairer *satellite.Repairer
	Admin    *satellite.Admin

	Log      *zap.Logger
	Identity *identity.FullIdentity
	DB       satellite.DB

	Dialer rpc.Dialer

	Server *server.Server

	Version *versionchecker.Service

	Contact struct {
		Service  *contact.Service
		Endpoint *contact.Endpoint
	}

	Overlay struct {
		DB        overlay.DB
		Service   *overlay.Service
		Inspector *overlay.Inspector
	}

	Metainfo struct {
		Database  metainfo.PointerDB
		Service   *metainfo.Service
		Endpoint2 *metainfo.Endpoint
		Loop      *metainfo.Loop
	}

	Inspector struct {
		Endpoint *inspector.Endpoint
	}

	Orders struct {
		DB       orders.DB
		Endpoint *orders.Endpoint
		Service  *orders.Service
		Chore    *orders.Chore
	}

	Repair struct {
		Checker   *checker.Checker
		Repairer  *repairer.Service
		Inspector *irreparable.Inspector
	}
	Audit struct {
		Queue    *audit.Queue
		Worker   *audit.Worker
		Chore    *audit.Chore
		Verifier *audit.Verifier
		Reporter *audit.Reporter
	}

	GarbageCollection struct {
		Service *gc.Service
	}

	DBCleanup struct {
		Chore *dbcleanup.Chore
	}

	Accounting struct {
		Tally          *tally.Service
		Rollup         *rollup.Service
		ProjectUsage   *accounting.Service
		ReportedRollup *reportedrollup.Chore
	}

	LiveAccounting struct {
		Cache accounting.Cache
	}

	Mail struct {
		Service *mailservice.Service
	}

	Vouchers struct {
		Endpoint *vouchers.Endpoint
	}

	Console struct {
		Listener net.Listener
		Service  *console.Service
		Endpoint *consoleweb.Server
	}

	Marketing struct {
		Listener net.Listener
		Endpoint *marketingweb.Server
	}

	NodeStats struct {
		Endpoint *nodestats.Endpoint
	}

	GracefulExit struct {
		Chore    *gracefulexit.Chore
		Endpoint *gracefulexit.Endpoint
	}

	Metrics struct {
		Chore *metrics.Chore
	}

	DowntimeTracking struct {
		DetectionChore  *downtime.DetectionChore
		EstimationChore *downtime.EstimationChore
		Service         *downtime.Service
	}
}

SatelliteSystem contains all the processes needed to run a full Satellite setup

func (*SatelliteSystem) Addr 

func (system *SatelliteSystem) Addr() string

Addr returns the public address from the Satellite system API.

func (*SatelliteSystem) Close 

func (system *SatelliteSystem) Close() error

Close closes all the subsystems in the Satellite system

func (*SatelliteSystem) ID 

func (system *SatelliteSystem) ID() storj.NodeID

ID returns the ID of the Satellite system.

func (*SatelliteSystem) Local 

func (system *SatelliteSystem) Local() overlay.NodeDossier

Local returns the peer local node info from the Satellite system API.

func (*SatelliteSystem) PrivateAddr 

func (system *SatelliteSystem) PrivateAddr() string

PrivateAddr returns the private address from the Satellite system API.

func (*SatelliteSystem) Run 

func (system *SatelliteSystem) Run(ctx context.Context) (err error)

Run runs all the subsystems in the Satellite system

func (*SatelliteSystem) URL 

func (system *SatelliteSystem) URL() storj.NodeURL

URL returns the storj.NodeURL from the Satellite system API. 

type Uplink struct {
	Log              *zap.Logger
	Info             pb.Node
	Identity         *identity.FullIdentity
	Dialer           rpc.Dialer
	StorageNodeCount int

	APIKey    map[storj.NodeID]*macaroon.APIKey
	ProjectID map[storj.NodeID]uuid.UUID
}

Uplink is a general purpose

func (*Uplink) Addr 

func (client *Uplink) Addr() string

Addr returns uplink address

func (*Uplink) CreateBucket 

func (client *Uplink) CreateBucket(ctx context.Context, satellite *SatelliteSystem, bucketName string) error

CreateBucket creates a new bucket

func (*Uplink) DeleteBucket added in v0.33.2 

func (client *Uplink) DeleteBucket(ctx context.Context, satellite *SatelliteSystem, bucketName string) error

DeleteBucket deletes a bucket.

func (*Uplink) DeleteObject added in v0.33.2 

func (client *Uplink) DeleteObject(ctx context.Context, satellite *SatelliteSystem, bucketName string, path storj.Path) error

DeleteObject deletes an object at the path in a bucket

func (*Uplink) DialMetainfo 

func (client *Uplink) DialMetainfo(ctx context.Context, destination Peer, apikey *macaroon.APIKey) (*metainfo.Client, error)

DialMetainfo dials destination with apikey and returns metainfo Client

func (*Uplink) DialPiecestore 

func (client *Uplink) DialPiecestore(ctx context.Context, destination Peer) (*piecestore.Client, error)

DialPiecestore dials destination storagenode and returns a piecestore client.

func (*Uplink) Download 

func (client *Uplink) Download(ctx context.Context, satellite *SatelliteSystem, bucketName string, path storj.Path) ([]byte, error)

Download data from specific satellite

func (*Uplink) DownloadStream 

func (client *Uplink) DownloadStream(ctx context.Context, satellite *SatelliteSystem, bucketName string, path storj.Path) (_ io.ReadCloser, cleanup func() error, err error)

DownloadStream returns stream for downloading data

func (*Uplink) DownloadStreamRange 

func (client *Uplink) DownloadStreamRange(ctx context.Context, satellite *SatelliteSystem, bucketName string, path storj.Path, start, limit int64) (_ io.ReadCloser, cleanup func() error, err error)

DownloadStreamRange returns stream for downloading data

func (*Uplink) GetConfig 

func (client *Uplink) GetConfig(satellite *SatelliteSystem) cmd.Config

GetConfig returns a default config for a given satellite.

func (*Uplink) GetProject 

func (client *Uplink) GetProject(ctx context.Context, satellite *SatelliteSystem) (*libuplink.Project, error)

GetProject returns a libuplink.Project which allows interactions with a specific project

func (*Uplink) GetProjectAndBucket 

func (client *Uplink) GetProjectAndBucket(ctx context.Context, satellite *SatelliteSystem, bucketName string, clientCfg cmd.Config) (_ *libuplink.Project, _ *libuplink.Bucket, err error)

GetProjectAndBucket returns a libuplink.Project and Bucket which allows interactions with a specific project and its buckets

func (*Uplink) ID 

func (client *Uplink) ID() storj.NodeID

ID returns uplink id

func (*Uplink) Local 

func (client *Uplink) Local() pb.Node

Local returns uplink info 

func (client *Uplink) NewLibuplink(ctx context.Context) (*libuplink.Uplink, error)

NewLibuplink creates a libuplink.Uplink object with the testplanet Uplink config

func (*Uplink) Shutdown 

func (client *Uplink) Shutdown() error

Shutdown shuts down all uplink dependencies

func (*Uplink) Upload 

func (client *Uplink) Upload(ctx context.Context, satellite *SatelliteSystem, bucket string, path storj.Path, data []byte) error

Upload data to specific satellite

func (*Uplink) UploadWithClientConfig 

func (client *Uplink) UploadWithClientConfig(ctx context.Context, satellite *SatelliteSystem, clientConfig cmd.Config, bucketName string, path storj.Path, data []byte) (err error)

UploadWithClientConfig uploads data to specific satellite with custom client configuration

func (*Uplink) UploadWithExpiration 

func (client *Uplink) UploadWithExpiration(ctx context.Context, satellite *SatelliteSystem, bucketName string, path storj.Path, data []byte, expiration time.Time) error

UploadWithExpiration data to specific satellite and expiration time

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.