gracefulexit

Documentation

Index

• Variables
• type BlobsCleaner
  • func NewBlobsCleaner(log *zap.Logger, store *pieces.Store, trust *trust.Pool, ...) *BlobsCleaner
  • func (blobsCleaner *BlobsCleaner) RemoveBlobs(ctx context.Context) (err error)
• type Chore
  • func NewChore(log *zap.Logger, service Service, transferService piecetransfer.Service, ...) *Chore
  • func (chore *Chore) Close() error
  • func (chore *Chore) Run(ctx context.Context) (err error)
  • func (chore *Chore) TestWaitForWorkers()
• type Config
• type Endpoint
  • func NewEndpoint(log *zap.Logger, trust *trust.Pool, satellites satellites.DB, ...) *Endpoint
  • func (e *Endpoint) GetExitProgress(ctx context.Context, req *internalpb.GetExitProgressRequest) (*internalpb.GetExitProgressResponse, error)
  • func (e *Endpoint) GetNonExitingSatellites(ctx context.Context, req *internalpb.GetNonExitingSatellitesRequest) (*internalpb.GetNonExitingSatellitesResponse, error)
  • func (e *Endpoint) GracefulExitFeasibility(ctx context.Context, request *internalpb.GracefulExitFeasibilityRequest) (*internalpb.GracefulExitFeasibilityResponse, error)
  • func (e *Endpoint) InitiateGracefulExit(ctx context.Context, req *internalpb.InitiateGracefulExitRequest) (*internalpb.ExitProgress, error)
• type ExitingSatellite
• type Service
  • func NewService(log *zap.Logger, store *pieces.Store, trust *trust.Pool, ...) Service
• type Worker
  • func NewWorker(log *zap.Logger, service Service, transferService piecetransfer.Service, ...) *Worker
  • func (worker *Worker) Close() error
  • func (worker *Worker) Run(ctx context.Context, done func()) (err error)

Variables

var (
	// Error is the default error class for graceful exit package.
	Error = errs.Class("gracefulexit")
)

Types

type BlobsCleaner

type BlobsCleaner struct {
	// contains filtered or unexported fields
}

BlobsCleaner checks for satellites that the node has completed exit successfully and clear blobs of it.

architecture: Chore

func NewBlobsCleaner

func NewBlobsCleaner(log *zap.Logger, store *pieces.Store, trust *trust.Pool, satelliteDB satellites.DB) *BlobsCleaner

NewBlobsCleaner instantiates BlobsCleaner.

func (*BlobsCleaner) RemoveBlobs

func (blobsCleaner *BlobsCleaner) RemoveBlobs(ctx context.Context) (err error)

RemoveBlobs runs blobs cleaner for satellites on which GE is completed. On node's restart checks if any of trusted satellites has GE status "successfully exited" Deletes blobs/satellite folder if exists, so if garbage collector didn't clean all SNO won't keep trash.

type Chore

type Chore struct {
	Loop *sync2.Cycle
	// contains filtered or unexported fields
}

Chore checks for satellites that the node is exiting and creates a worker per satellite to complete the process.

architecture: Chore

func NewChore

func NewChore(log *zap.Logger, service Service, transferService piecetransfer.Service, dialer rpc.Dialer, config Config) *Chore

NewChore instantiates Chore.

func (*Chore) Close

func (chore *Chore) Close() error

Close closes chore.

func (*Chore) Run

func (chore *Chore) Run(ctx context.Context) (err error)

Run starts the chore.

func (*Chore) TestWaitForWorkers

func (chore *Chore) TestWaitForWorkers()

TestWaitForWorkers waits for any pending worker to finish.

type Config

type Config struct {
	ChoreInterval          time.Duration `help:"how often to run the chore to check for satellites for the node to exit." releaseDefault:"1m" devDefault:"10s"`
	NumWorkers             int           `help:"number of workers to handle satellite exits" default:"4"`
	NumConcurrentTransfers int           `help:"number of concurrent transfers per graceful exit worker" default:"5"`
	MinBytesPerSecond      memory.Size   `help:"the minimum acceptable bytes that an exiting node can transfer per second to the new node" default:"5KB"`
	MinDownloadTimeout     time.Duration `help:"the minimum duration for downloading a piece from storage nodes before timing out" default:"2m"`
}

Config for graceful exit.

type Endpoint

type Endpoint struct {
	// contains filtered or unexported fields
}

Endpoint implements private inspector for Graceful Exit.

func NewEndpoint

func NewEndpoint(log *zap.Logger, trust *trust.Pool, satellites satellites.DB, dialer rpc.Dialer, usageCache *pieces.BlobsUsageCache) *Endpoint

NewEndpoint creates a new graceful exit endpoint.

func (*Endpoint) GetExitProgress

func (e *Endpoint) GetExitProgress(ctx context.Context, req *internalpb.GetExitProgressRequest) (*internalpb.GetExitProgressResponse, error)

GetExitProgress returns graceful exit progress on each satellite that a storagde node has started exiting.

func (*Endpoint) GetNonExitingSatellites

func (e *Endpoint) GetNonExitingSatellites(ctx context.Context, req *internalpb.GetNonExitingSatellitesRequest) (*internalpb.GetNonExitingSatellitesResponse, error)

GetNonExitingSatellites returns a list of satellites that the storagenode has not begun a graceful exit for.

func (*Endpoint) GracefulExitFeasibility

func (e *Endpoint) GracefulExitFeasibility(ctx context.Context, request *internalpb.GracefulExitFeasibilityRequest) (*internalpb.GracefulExitFeasibilityResponse, error)

GracefulExitFeasibility returns graceful exit feasibility by node's age on chosen satellite.

func (*Endpoint) InitiateGracefulExit

func (e *Endpoint) InitiateGracefulExit(ctx context.Context, req *internalpb.InitiateGracefulExitRequest) (*internalpb.ExitProgress, error)

InitiateGracefulExit updates one or more satellites in the storagenode's database to be gracefully exiting.

type ExitingSatellite

type ExitingSatellite struct {
	satellites.ExitProgress
	NodeURL storj.NodeURL
}

ExitingSatellite encapsulates a node address with its graceful exit progress.

type Service

type Service interface {
	// ListPendingExits returns a slice with one record for every satellite
	// from which this node is gracefully exiting. Each record includes the
	// satellite's ID/address and information about the graceful exit status
	// and progress.
	ListPendingExits(ctx context.Context) ([]ExitingSatellite, error)

	// DeletePiece deletes one piece stored for a satellite, and updates
	// the deleted byte count for the corresponding graceful exit operation.
	DeletePiece(ctx context.Context, satelliteID storj.NodeID, pieceID storj.PieceID) error

	// DeleteSatellitePieces deletes all pieces stored for a satellite, and updates
	// the deleted byte count for the corresponding graceful exit operation.
	DeleteSatellitePieces(ctx context.Context, satelliteID storj.NodeID) error

	// ExitFailed updates the database when a graceful exit has failed.
	ExitFailed(ctx context.Context, satelliteID storj.NodeID, reason pb.ExitFailed_Reason, exitFailedBytes []byte) error

	// ExitCompleted updates the database when a graceful exit is completed. It also
	// deletes all pieces and blobs for that satellite.
	ExitCompleted(ctx context.Context, satelliteID storj.NodeID, completionReceipt []byte, wait func()) error

	// ExitNotPossible deletes the entry for the corresponding graceful exit operation.
	// This is intended to be called when a graceful exit operation was initiated but
	// the satellite rejected it.
	ExitNotPossible(ctx context.Context, satelliteID storj.NodeID) error
}

Service acts as the gateway to the `satellites` db for graceful exit code (querying and updating that db as necessary).

architecture: Service

func NewService

func NewService(log *zap.Logger, store *pieces.Store, trust *trust.Pool, satelliteDB satellites.DB, dialer rpc.Dialer, config Config) Service

NewService is a constructor for a GE service.

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker is responsible for completing the graceful exit for a given satellite.

func NewWorker

func NewWorker(log *zap.Logger, service Service, transferService piecetransfer.Service, dialer rpc.Dialer, satelliteURL storj.NodeURL, config Config) *Worker

NewWorker instantiates Worker.

func (*Worker) Close

func (worker *Worker) Close() error

Close halts the worker.

func (*Worker) Run

func (worker *Worker) Run(ctx context.Context, done func()) (err error)

Run calls the satellite endpoint, transfers pieces, validates, and responds with success or failure. It also marks the satellite finished once all the pieces have been transferred.