repairer

Documentation

Index

• Variables
• type Config
• type ECRepairer
  • func NewECRepairer(log *zap.Logger, dialer rpc.Dialer, satelliteSignee signing.Signee) *ECRepairer
  • func (ec *ECRepairer) Get(ctx context.Context, limits []*pb.AddressedOrderLimit, ...) (_ io.ReadCloser, failedPieces []*pb.RemotePiece, err error)
  • func (ec *ECRepairer) Repair(ctx context.Context, limits []*pb.AddressedOrderLimit, ...) (successfulNodes []*pb.Node, successfulHashes []*pb.PieceHash, err error)
• type SegmentRepairer
  • func NewSegmentRepairer(log *zap.Logger, metainfo *metainfo.Service, orders *orders.Service, ...) *SegmentRepairer
  • func (repairer *SegmentRepairer) Repair(ctx context.Context, path storj.Path) (shouldDelete bool, err error)
• type Service
  • func NewService(log *zap.Logger, queue queue.RepairQueue, config *Config, ...) *Service
  • func (service *Service) Close() error
  • func (service *Service) Run(ctx context.Context) (err error)

Variables

var ErrPieceHashVerifyFailed = errs.Class("piece hashes don't match")

ErrPieceHashVerifyFailed is the errs class when a piece hash downloaded from storagenode fails to match the original hash.

var (
	Error = errs.Class("repairer error")
)

Error is a standard error class for this package.

var IrreparableError = errs.Class("irreparable error")

IrreparableError is the errs class of irreparable segment errors

Types

type Config

type Config struct {
	MaxRepair                     int           `help:"maximum segments that can be repaired concurrently" releaseDefault:"5" devDefault:"1"`
	Interval                      time.Duration `help:"how frequently repairer should try and repair more data" releaseDefault:"1h" devDefault:"0h5m0s"`
	Timeout                       time.Duration `help:"time limit for uploading repaired pieces to new storage nodes" default:"10m0s"`
	MaxBufferMem                  memory.Size   `help:"maximum buffer memory (in bytes) to be allocated for read buffers" default:"4M"`
	MaxExcessRateOptimalThreshold float64       `` /* 135-byte string literal not displayed */
}

Config contains configurable values for repairer

type ECRepairer

type ECRepairer struct {
	// contains filtered or unexported fields
}

ECRepairer allows the repairer to download, verify, and upload pieces from storagenodes.

func NewECRepairer

func NewECRepairer(log *zap.Logger, dialer rpc.Dialer, satelliteSignee signing.Signee) *ECRepairer

NewECRepairer creates a new repairer for interfacing with storagenodes.

func (*ECRepairer) Get

func (ec *ECRepairer) Get(ctx context.Context, limits []*pb.AddressedOrderLimit, privateKey storj.PiecePrivateKey, es eestream.ErasureScheme, dataSize int64, path storj.Path) (_ io.ReadCloser, failedPieces []*pb.RemotePiece, err error)

Get downloads pieces from storagenodes using the provided order limits, and decodes those pieces into a segment. It attempts to download from the minimum required number based on the redundancy scheme. After downloading a piece, the ECRepairer will verify the hash and original order limit for that piece. If verification fails, another piece will be downloaded until we reach the minimum required or run out of order limits. If piece hash verification fails, it will return all failed node IDs.

func (*ECRepairer) Repair

func (ec *ECRepairer) Repair(ctx context.Context, limits []*pb.AddressedOrderLimit, privateKey storj.PiecePrivateKey, rs eestream.RedundancyStrategy, data io.Reader, expiration time.Time, timeout time.Duration, path storj.Path) (successfulNodes []*pb.Node, successfulHashes []*pb.PieceHash, err error)

Repair takes a provided segment, encodes it with the provided redundancy strategy, and uploads the pieces in need of repair to new nodes provided by order limits.

type SegmentRepairer

type SegmentRepairer struct {
	// contains filtered or unexported fields
}

SegmentRepairer for segments

func NewSegmentRepairer

func NewSegmentRepairer(
	log *zap.Logger, metainfo *metainfo.Service, orders *orders.Service,
	overlay *overlay.Service, dialer rpc.Dialer, timeout time.Duration,
	excessOptimalThreshold float64, repairOverride int, satelliteSignee signing.Signee,
) *SegmentRepairer

NewSegmentRepairer creates a new instance of SegmentRepairer.

excessPercentageOptimalThreshold is the percentage to apply over the optimal threshould to determine the maximum limit of nodes to upload repaired pieces, when negative, 0 is applied.

func (*SegmentRepairer) Repair

func (repairer *SegmentRepairer) Repair(ctx context.Context, path storj.Path) (shouldDelete bool, err error)

Repair retrieves an at-risk segment and repairs and stores lost pieces on new nodes note that shouldDelete is used even in the case where err is not null note that it will update audit status as failed for nodes that failed piece hash verification during repair downloading

type Service

type Service struct {
	Limiter *sync2.Limiter
	Loop    sync2.Cycle
	// contains filtered or unexported fields
}

Service contains the information needed to run the repair service

architecture: Worker

func NewService

func NewService(log *zap.Logger, queue queue.RepairQueue, config *Config, repairer *SegmentRepairer) *Service

NewService creates repairing service

func (*Service) Close

func (service *Service) Close() error

Close closes resources

func (*Service) Run

func (service *Service) Run(ctx context.Context) (err error)

Run runs the repairer service