repair

Documentation

Index

• func FindClumpedPieces(pieces metabase.Pieces, lastNets []string) (clumped metabase.Pieces)
• func SegmentHealth(numHealthy, minPieces, totalNodes int, failureRate float64) float64
• type PiecesCheckResult
  • func ClassifySegmentPieces(pieces metabase.Pieces, nodes []nodeselection.SelectedNode, ...) (result PiecesCheckResult)

Functions

func FindClumpedPieces

func FindClumpedPieces(pieces metabase.Pieces, lastNets []string) (clumped metabase.Pieces)

FindClumpedPieces finds pieces that are stored in the same last_net (i.e., the same /24 network in the IPv4 case). The first piece for a given last_net is fine, but any subsequent pieces in the same last_net will be returned as part of the 'clumped' list.

lastNets must be a slice of the same length as pieces; lastNets[i] corresponds to pieces[i].

func SegmentHealth

func SegmentHealth(numHealthy, minPieces, totalNodes int, failureRate float64) float64

SegmentHealth returns a value corresponding to the health of a segment in the repair queue. Lower health segments should be repaired first.

This calculation purports to find the number of iterations for which a segment can be expected to survive, with the given failureRate. The number of iterations for the segment to survive (X) can be modeled with the negative binomial distribution, with the number of pieces that must be lost as the success threshold r, and the chance of losing a single piece in a round as the trial success probability p.

First, we calculate the expected number of iterations for a segment to survive if we were to lose exactly one node every iteration:

r = numHealthy - minPieces + 1
p = (totalNodes - numHealthy) / totalNodes
X ~ NB(r, p)

Then we take the mean of that distribution to use as our expected value, which is pr/(1-p).

Finally, to get away from the "one node per iteration" simplification, we just scale the magnitude of the iterations in the model so that there really is one node being lost. For example, if our failureRate and totalNodes imply a churn rate of 3 nodes per day, we just take 1/3 of a day and call that an "iteration" for purposes of the model. To convert iterations in the model to days, we divide the mean of the negative binomial distribution (X, above) by the number of nodes that we estimate will churn in one day.

Types

type PiecesCheckResult

type PiecesCheckResult struct {
	// ExcludeNodeIDs is a list of all node IDs holding pieces of this segment.
	ExcludeNodeIDs []storj.NodeID

	// Missing is a set of Piece Numbers which are to be considered as lost and irretrievable.
	// (They reside on offline/disqualified/unknown nodes.)
	Missing map[uint16]struct{}
	// Retrievable contains all Piece Numbers that are retrievable; that is, all piece numbers
	// from the segment that are NOT in Missing.
	Retrievable map[uint16]struct{}

	// Suspended is a set of Piece Numbers which reside on nodes which are suspended.
	Suspended map[uint16]struct{}
	// Clumped is a set of Piece Numbers which are to be considered unhealthy because of IP
	// clumping. (If DoDeclumping is disabled, this set will be empty.)
	Clumped map[uint16]struct{}
	// Exiting is a set of Piece Numbers which are considered unhealthy because the node on
	// which they reside has initiated graceful exit.
	Exiting map[uint16]struct{}
	// OutOfPlacement is a set of Piece Numbers which are unhealthy because of placement rules.
	// (If DoPlacementCheck is disabled, this set will be empty.)
	OutOfPlacement map[uint16]struct{}
	// InExcludedCountry is a set of Piece Numbers which are unhealthy because they are in
	// Excluded countries.
	InExcludedCountry map[uint16]struct{}

	// ForcingRepair is the set of pieces which force a repair operation for this segment (that
	// includes, currently, only pieces in OutOfPlacement).
	ForcingRepair map[uint16]struct{}
	// Unhealthy contains all Piece Numbers which are in Missing OR Suspended OR Clumped OR
	// Exiting OR OutOfPlacement OR InExcludedCountry.
	Unhealthy map[uint16]struct{}
	// UnhealthyRetrievable is the set of pieces that are "unhealthy-but-retrievable". That is,
	// pieces that are in Unhealthy AND Retrievable.
	UnhealthyRetrievable map[uint16]struct{}
	// Healthy contains all Piece Numbers from the segment which are not in Unhealthy.
	// (Equivalently: all Piece Numbers from the segment which are NOT in Missing OR
	// Suspended OR Clumped OR Exiting OR OutOfPlacement OR InExcludedCountry).
	Healthy map[uint16]struct{}
}

PiecesCheckResult contains all necessary aggregate information about the state of pieces in a segment. The node that should be holding each piece is evaluated to see if it is online and whether it is in a clumped IP network, in an excluded country, or out of placement for the segment.

func ClassifySegmentPieces

func ClassifySegmentPieces(pieces metabase.Pieces, nodes []nodeselection.SelectedNode, excludedCountryCodes map[location.CountryCode]struct{}, doPlacementCheck, doDeclumping bool, filter nodeselection.NodeFilter) (result PiecesCheckResult)

ClassifySegmentPieces classifies the pieces of a segment into the categories represented by a PiecesCheckResult. Pieces may be put into multiple categories.