repair

Documentation

Index

• func FindClumpedPieces(pieces metabase.Pieces, lastNets []string) (clumped metabase.Pieces)
• func SegmentHealth(numHealthy, minPieces, totalNodes int, failureRate float64) float64

Functions

func FindClumpedPieces

func FindClumpedPieces(pieces metabase.Pieces, lastNets []string) (clumped metabase.Pieces)

FindClumpedPieces finds pieces that are stored in the same last_net (i.e., the same /24 network in the IPv4 case). The first piece for a given last_net is fine, but any subsequent pieces in the same last_net will be returned as part of the 'clumped' list.

lastNets must be a slice of the same length as pieces; lastNets[i] corresponds to pieces[i].

func SegmentHealth

func SegmentHealth(numHealthy, minPieces, totalNodes int, failureRate float64) float64

SegmentHealth returns a value corresponding to the health of a segment in the repair queue. Lower health segments should be repaired first.

This calculation purports to find the number of iterations for which a segment can be expected to survive, with the given failureRate. The number of iterations for the segment to survive (X) can be modeled with the negative binomial distribution, with the number of pieces that must be lost as the success threshold r, and the chance of losing a single piece in a round as the trial success probability p.

First, we calculate the expected number of iterations for a segment to survive if we were to lose exactly one node every iteration:

r = numHealthy - minPieces + 1
p = (totalNodes - numHealthy) / totalNodes
X ~ NB(r, p)

Then we take the mean of that distribution to use as our expected value, which is pr/(1-p).

Finally, to get away from the "one node per iteration" simplification, we just scale the magnitude of the iterations in the model so that there really is one node being lost. For example, if our failureRate and totalNodes imply a churn rate of 3 nodes per day, we just take 1/3 of a day and call that an "iteration" for purposes of the model. To convert iterations in the model to days, we divide the mean of the negative binomial distribution (X, above) by the number of nodes that we estimate will churn in one day.