testplanet

package
v1.24.1-rc Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 18, 2021 License: AGPL-3.0 Imports: 95 Imported by: 6

Documentation

Overview

Package testplanet implements the full network wiring for testing

Index

Constants

This section is empty.

Variables

View Source 
var Combine = func(elements ...func(log *zap.Logger, index int, config *satellite.Config)) func(log *zap.Logger, index int, config *satellite.Config) {
	return func(log *zap.Logger, index int, config *satellite.Config) {
		for _, f := range elements {
			f(log, index, config)
		}
	}
}

Combine combines satellite reconfigure functions.

View Source 
var DisablePeerCAWhitelist = Reconfigure{
	Satellite: func(log *zap.Logger, index int, config *satellite.Config) {
		config.Server.UsePeerCAWhitelist = false
	},
	StorageNode: func(index int, config *storagenode.Config) {
		config.Server.UsePeerCAWhitelist = false
	},
}

DisablePeerCAWhitelist returns a `Reconfigure` that sets `UsePeerCAWhitelist` for all node types that use kademlia.

View Source 
var DisableQUIC = Reconfigure{
	Satellite: func(log *zap.Logger, index int, config *satellite.Config) {
		config.Server.DisableQUIC = true
	},
	StorageNode: func(index int, config *storagenode.Config) {
		config.Server.DisableQUIC = true
	},
}

DisableQUIC prevents both satellite and storagenode being able to accept new quic connections.

View Source 
var DisableTCP = Reconfigure{
	Satellite: func(log *zap.Logger, index int, config *satellite.Config) {
		config.Server.DisableTCPTLS = true
	},
	StorageNode: func(index int, config *storagenode.Config) {
		config.Server.DisableTCPTLS = true
	},
}

DisableTCP prevents both satellite and storagenode being able to accept new tcp connections.

View Source 
var MaxMetadataSize = func(maxMetadataSize memory.Size) func(log *zap.Logger, index int, config *satellite.Config) {
	return func(log *zap.Logger, index int, config *satellite.Config) {
		config.Metainfo.MaxMetadataSize = maxMetadataSize
	}
}

MaxMetadataSize returns function to change satellite max metadata size value.

View Source 
var MaxSegmentSize = func(maxSegmentSize memory.Size) func(log *zap.Logger, index int, config *satellite.Config) {
	return func(log *zap.Logger, index int, config *satellite.Config) {
		config.Metainfo.MaxSegmentSize = maxSegmentSize
	}
}

MaxSegmentSize returns function to change satellite max segment size value.

View Source 
var ReconfigureRS = func(minThreshold, repairThreshold, successThreshold, totalThreshold int) func(log *zap.Logger, index int, config *satellite.Config) {
	return func(log *zap.Logger, index int, config *satellite.Config) {
		config.Metainfo.RS.Min = minThreshold
		config.Metainfo.RS.Repair = repairThreshold
		config.Metainfo.RS.Success = successThreshold
		config.Metainfo.RS.Total = totalThreshold
	}
}

ReconfigureRS returns function to change satellite redundancy scheme values.

View Source 
var ShortenOnlineWindow = Reconfigure{
	Satellite: func(log *zap.Logger, index int, config *satellite.Config) {
		config.Overlay.Node.OnlineWindow = 1 * time.Second
	},
}

ShortenOnlineWindow returns a `Reconfigure` that sets the NodeSelection OnlineWindow to 1 second, meaning a connection failure leads to marking the nodes as offline.

Functions

func Run 

func Run(t *testing.T, config Config, test func(t *testing.T, ctx *testcontext.Context, planet *Planet))

Run runs testplanet in multiple configurations.

Types

type Config 

type Config struct {
	SatelliteCount   int
	StorageNodeCount int
	UplinkCount      int

	IdentityVersion *storj.IDVersion
	Reconfigure     Reconfigure

	Name        string
	NonParallel bool
}

Config describes planet configuration.

type DatabaseConfig added in v1.1.1 

type DatabaseConfig struct {
	SatelliteDB        string
	SatellitePointerDB string
}

DatabaseConfig defines connection strings for database.

type Peer 

type Peer interface {
	Label() string

	ID() storj.NodeID
	Addr() string
	URL() string
	NodeURL() storj.NodeURL

	Run(context.Context) error
	Close() error
}

Peer represents one of StorageNode or Satellite.

type Planet 

type Planet struct {
	VersionControl *versioncontrol.Peer
	Satellites     []*Satellite
	StorageNodes   []*StorageNode
	Uplinks        []*Uplink
	// contains filtered or unexported fields
}

Planet is a full storj system setup.

func NewCustom 

func NewCustom(ctx context.Context, log *zap.Logger, config Config, satelliteDatabases satellitedbtest.SatelliteDatabases) (*Planet, error)

NewCustom creates a new full system with the specified configuration.

func (*Planet) FindNode added in v0.35.2 

func (planet *Planet) FindNode(nodeID storj.NodeID) *StorageNode

FindNode is a helper to retrieve a storage node record by its node ID.

func (*Planet) Identities 

func (planet *Planet) Identities() *testidentity.Identities

Identities returns the identity provider for this planet.

func (*Planet) NewIdentity 

func (planet *Planet) NewIdentity() (*identity.FullIdentity, error)

NewIdentity creates a new identity for a node.

func (*Planet) NewListener 

func (planet *Planet) NewListener() (net.Listener, error)

NewListener creates a new listener.

func (*Planet) NewVersionConfig 

func (planet *Planet) NewVersionConfig() checker.Config

NewVersionConfig returns the Version Config for this planet with tuned metrics.

func (*Planet) NewVersionInfo 

func (planet *Planet) NewVersionInfo() version.Info

NewVersionInfo returns the Version Info for this planet with tuned metrics.

func (*Planet) Shutdown 

func (planet *Planet) Shutdown() error

Shutdown shuts down all the nodes and deletes temporary directories.

func (*Planet) Size 

func (planet *Planet) Size() int

Size returns number of nodes in the network.

func (*Planet) Start 

func (planet *Planet) Start(ctx context.Context)

Start starts all the nodes.

func (*Planet) StopNodeAndUpdate added in v1.5.2 

func (planet *Planet) StopNodeAndUpdate(ctx context.Context, node *StorageNode) error

StopNodeAndUpdate stops storage node and updates satellite overlay.

func (*Planet) StopPeer 

func (planet *Planet) StopPeer(peer Peer) error

StopPeer stops a single peer in the planet.

func (*Planet) WaitForStorageNodeDeleters added in v1.4.1 

func (planet *Planet) WaitForStorageNodeDeleters(ctx context.Context)

WaitForStorageNodeDeleters calls the Wait method on each storagenode's PieceDeleter.

func (*Planet) WaitForStorageNodeEndpoints added in v0.35.2 

func (planet *Planet) WaitForStorageNodeEndpoints(ctx context.Context) error

WaitForStorageNodeEndpoints waits for storage node endpoints to finish their work. The call will return an error if they have not been completed after 1 minute.

func (*Planet) WriteWhitelist 

func (planet *Planet) WriteWhitelist(version storj.IDVersion) (string, error)

WriteWhitelist writes the pregenerated signer's CA cert to a "CA whitelist", PEM-encoded.

type Project added in v1.3.2 

type Project struct {
	ID    uuid.UUID
	Owner ProjectOwner

	Satellite Peer
	APIKey    string

	RawAPIKey *macaroon.APIKey
	// contains filtered or unexported fields
}

Project contains all necessary information about a user.

func (*Project) DialMetainfo added in v1.3.2 

func (project *Project) DialMetainfo(ctx context.Context) (*metainfo.Client, error)

DialMetainfo dials the satellite with the appropriate api key.

type ProjectOwner added in v1.3.2 

type ProjectOwner struct {
	ID    uuid.UUID
	Email string
}

ProjectOwner contains information about the project owner.

type Reconfigure 

type Reconfigure struct {
	SatelliteDB        func(log *zap.Logger, index int, db satellite.DB) (satellite.DB, error)
	SatellitePointerDB func(log *zap.Logger, index int, db metainfo.PointerDB) (metainfo.PointerDB, error)
	Satellite          func(log *zap.Logger, index int, config *satellite.Config)

	StorageNodeDB func(index int, db storagenode.DB, log *zap.Logger) (storagenode.DB, error)
	StorageNode   func(index int, config *storagenode.Config)
	UniqueIPCount int

	VersionControl func(config *versioncontrol.Config)

	Identities func(log *zap.Logger, version storj.IDVersion) *testidentity.Identities
}

Reconfigure allows to change node configurations.

type Satellite added in v1.1.1 

type Satellite struct {
	Name   string
	Config satellite.Config

	Core     *satellite.Core
	API      *satellite.API
	Repairer *satellite.Repairer
	Admin    *satellite.Admin
	GC       *satellite.GarbageCollection

	Log      *zap.Logger
	Identity *identity.FullIdentity
	DB       satellite.DB

	Dialer rpc.Dialer

	Server *server.Server

	Version *versionchecker.Service

	Contact struct {
		Service  *contact.Service
		Endpoint *contact.Endpoint
	}

	Overlay struct {
		DB           overlay.DB
		Service      *overlay.Service
		Inspector    *overlay.Inspector
		DQStrayNodes *straynodes.Chore
	}

	Metainfo struct {
		Database  metainfo.PointerDB
		Service   *metainfo.Service
		Endpoint2 *metainfo.Endpoint
		Loop      *metainfo.Loop
		Metabase  *metainfo.PointerDBMetabase
	}

	Inspector struct {
		Endpoint *inspector.Endpoint
	}

	Orders struct {
		DB       orders.DB
		Endpoint *orders.Endpoint
		Service  *orders.Service
		Chore    *orders.Chore
	}

	Repair struct {
		Checker   *checker.Checker
		Repairer  *repairer.Service
		Inspector *irreparable.Inspector
	}

	Audit struct {
		Queues   *audit.Queues
		Worker   *audit.Worker
		Chore    *audit.Chore
		Verifier *audit.Verifier
		Reporter *audit.Reporter
	}

	GarbageCollection struct {
		Service *gc.Service
	}

	ExpiredDeletion struct {
		Chore *expireddeletion.Chore
	}

	Accounting struct {
		Tally            *tally.Service
		Rollup           *rollup.Service
		ProjectUsage     *accounting.Service
		ProjectBWCleanup *projectbwcleanup.Chore
		RollupArchive    *rolluparchive.Chore
	}

	LiveAccounting struct {
		Cache accounting.Cache
	}

	ProjectLimits struct {
		Cache *accounting.ProjectLimitCache
	}

	Mail struct {
		Service *mailservice.Service
	}

	Console struct {
		Listener net.Listener
		Service  *console.Service
		Endpoint *consoleweb.Server
	}

	NodeStats struct {
		Endpoint *nodestats.Endpoint
	}

	GracefulExit struct {
		Chore    *gracefulexit.Chore
		Endpoint *gracefulexit.Endpoint
	}

	Metrics struct {
		Chore *metrics.Chore
	}
}

Satellite contains all the processes needed to run a full Satellite setup.

func (*Satellite) AddProject added in v1.6.1 

func (system *Satellite) AddProject(ctx context.Context, ownerID uuid.UUID, name string) (*console.Project, error)

AddProject adds project to a satellite and makes specified user an owner.

func (*Satellite) AddUser added in v1.6.1 

func (system *Satellite) AddUser(ctx context.Context, newUser console.CreateUser, maxNumberOfProjects int) (*console.User, error)

AddUser adds user to a satellite. Password from newUser will be always overridden by FullName to have known password which can be used automatically.

func (*Satellite) Addr added in v1.1.1 

func (system *Satellite) Addr() string

Addr returns the public address from the Satellite system API.

func (*Satellite) AuthenticatedContext added in v1.11.1 

func (system *Satellite) AuthenticatedContext(ctx context.Context, userID uuid.UUID) (context.Context, error)

AuthenticatedContext creates context with authentication date for given user.

func (*Satellite) Close added in v1.1.1 

func (system *Satellite) Close() error

Close closes all the subsystems in the Satellite system.

func (*Satellite) ID added in v1.1.1 

func (system *Satellite) ID() storj.NodeID

ID returns the ID of the Satellite system.

func (*Satellite) Label added in v1.17.1 

func (system *Satellite) Label() string

Label returns name for debugger.

func (*Satellite) NodeURL added in v1.3.2 

func (system *Satellite) NodeURL() storj.NodeURL

NodeURL returns the storj.NodeURL from the Satellite system API.

func (*Satellite) PrivateAddr added in v1.1.1 

func (system *Satellite) PrivateAddr() string

PrivateAddr returns the private address from the Satellite system API.

func (*Satellite) Run added in v1.1.1 

func (system *Satellite) Run(ctx context.Context) (err error)

Run runs all the subsystems in the Satellite system.

func (*Satellite) URL added in v1.1.1 

func (system *Satellite) URL() string

URL returns the node url from the Satellite system API.

type StorageNode added in v1.1.1 

type StorageNode struct {
	Name   string
	Config storagenode.Config
	*storagenode.Peer
}

StorageNode contains all the processes needed to run a full StorageNode setup.

func (*StorageNode) Label added in v1.17.1 

func (system *StorageNode) Label() string

Label returns name for debugger.

func (*StorageNode) NodeURL added in v1.3.2 

func (system *StorageNode) NodeURL() storj.NodeURL

NodeURL returns the storj.NodeURL.

func (*StorageNode) URL added in v1.1.1 

func (system *StorageNode) URL() string

URL returns the node url as a string. 

type Uplink struct {
	Log      *zap.Logger
	Identity *identity.FullIdentity
	Dialer   rpc.Dialer
	Config   uplink.Config

	APIKey map[storj.NodeID]*macaroon.APIKey
	Access map[storj.NodeID]*uplink.Access

	// Projects is indexed by the satellite number.
	Projects []*Project
}

Uplink is a registered user on all satellites, which contains the necessary accesses and project info.

func (*Uplink) Addr 

func (client *Uplink) Addr() string

Addr returns uplink address.

func (*Uplink) CreateBucket 

func (client *Uplink) CreateBucket(ctx context.Context, satellite *Satellite, bucketName string) error

CreateBucket creates a new bucket.

func (*Uplink) DeleteBucket added in v0.33.2 

func (client *Uplink) DeleteBucket(ctx context.Context, satellite *Satellite, bucketName string) error

DeleteBucket deletes a bucket.

func (*Uplink) DeleteObject added in v0.33.2 

func (client *Uplink) DeleteObject(ctx context.Context, satellite *Satellite, bucketName string, path storj.Path) error

DeleteObject deletes an object at the path in a bucket.

func (*Uplink) DialMetainfo 

func (client *Uplink) DialMetainfo(ctx context.Context, destination Peer, apikey *macaroon.APIKey) (*metainfo.Client, error)

DialMetainfo dials destination with apikey and returns metainfo Client.

func (*Uplink) DialPiecestore 

func (client *Uplink) DialPiecestore(ctx context.Context, destination Peer) (*piecestore.Client, error)

DialPiecestore dials destination storagenode and returns a piecestore client.

func (*Uplink) Download 

func (client *Uplink) Download(ctx context.Context, satellite *Satellite, bucketName string, path storj.Path) ([]byte, error)

Download data from specific satellite.

func (*Uplink) DownloadStream 

func (client *Uplink) DownloadStream(ctx context.Context, satellite *Satellite, bucketName string, path storj.Path) (_ io.ReadCloser, cleanup func() error, err error)

DownloadStream returns stream for downloading data.

func (*Uplink) DownloadStreamRange 

func (client *Uplink) DownloadStreamRange(ctx context.Context, satellite *Satellite, bucketName string, path storj.Path, start, limit int64) (_ io.ReadCloser, cleanup func() error, err error)

DownloadStreamRange returns stream for downloading data.

func (*Uplink) GetProject 

func (client *Uplink) GetProject(ctx context.Context, satellite *Satellite) (*uplink.Project, error)

GetProject returns a uplink.Project which allows interactions with a specific project.

func (*Uplink) ID 

func (client *Uplink) ID() storj.NodeID

ID returns uplink id.

func (*Uplink) ListBuckets added in v1.21.1 

func (client *Uplink) ListBuckets(ctx context.Context, satellite *Satellite) ([]*uplink.Bucket, error)

ListBuckets returns a list of all buckets in a project.

func (*Uplink) OpenProject added in v1.19.1 

func (client *Uplink) OpenProject(ctx context.Context, satellite *Satellite) (*uplink.Project, error)

OpenProject opens project with predefined access grant and gives access to pure uplink API.

func (*Uplink) Shutdown 

func (client *Uplink) Shutdown() error

Shutdown shuts down all uplink dependencies.

func (*Uplink) Upload 

func (client *Uplink) Upload(ctx context.Context, satellite *Satellite, bucket string, path storj.Path, data []byte) error

Upload data to specific satellite.

func (*Uplink) UploadWithExpiration 

func (client *Uplink) UploadWithExpiration(ctx context.Context, satellite *Satellite, bucketName string, path storj.Path, data []byte, expiration time.Time) error

UploadWithExpiration data to specific satellite and expiration time.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.