checker

Documentation

Index

• Variables
• type Checker
  • func NewChecker(logger *zap.Logger, repairQueue queue.RepairQueue, irrdb irreparable.DB, ...) *Checker
  • func (checker *Checker) Close() error
  • func (checker *Checker) IdentifyInjuredSegments(ctx context.Context) (err error)
  • func (checker *Checker) IrreparableProcess(ctx context.Context) (err error)
  • func (checker *Checker) RefreshReliabilityCache(ctx context.Context) error
  • func (checker *Checker) Run(ctx context.Context) (err error)
• type Config
• type ReliabilityCache
  • func NewReliabilityCache(overlay *overlay.Service, staleness time.Duration) *ReliabilityCache
  • func (cache *ReliabilityCache) LastUpdate() time.Time
  • func (cache *ReliabilityCache) MissingPieces(ctx context.Context, created time.Time, pieces []*pb.RemotePiece) (_ []int32, err error)
  • func (cache *ReliabilityCache) NumNodes(ctx context.Context) (numNodes int, err error)
  • func (cache *ReliabilityCache) Refresh(ctx context.Context) (err error)
• type RepairOverride
  • func (ro *RepairOverride) Set(s string) error
  • func (ro *RepairOverride) String() string
  • func (RepairOverride) Type() string
• type RepairOverrides
  • func (ros *RepairOverrides) GetMap() RepairOverridesMap
  • func (ros *RepairOverrides) Set(s string) error
  • func (ros *RepairOverrides) String() string
  • func (RepairOverrides) Type() string
• type RepairOverridesMap
  • func (rom *RepairOverridesMap) GetOverrideValue(rs storj.RedundancyScheme) int32
  • func (rom *RepairOverridesMap) GetOverrideValuePB(rs *pb.RedundancyScheme) int32

Variables

var (
	Error = errs.Class("checker error")
)

Error is a standard error class for this package.

Types

type Checker

type Checker struct {
	Loop            *sync2.Cycle
	IrreparableLoop *sync2.Cycle
	// contains filtered or unexported fields
}

Checker contains the information needed to do checks for missing pieces.

architecture: Chore

func NewChecker

func NewChecker(logger *zap.Logger, repairQueue queue.RepairQueue, irrdb irreparable.DB, metainfo *metainfo.Service, metaLoop *metainfo.Loop, overlay *overlay.Service, config Config) *Checker

NewChecker creates a new instance of checker.

func (*Checker) Close

func (checker *Checker) Close() error

Close halts the Checker loop.

func (*Checker) IdentifyInjuredSegments

func (checker *Checker) IdentifyInjuredSegments(ctx context.Context) (err error)

IdentifyInjuredSegments checks for missing pieces off of the metainfo and overlay.

func (*Checker) IrreparableProcess

func (checker *Checker) IrreparableProcess(ctx context.Context) (err error)

IrreparableProcess iterates over all items in the irreparabledb. If an item can now be repaired then it is added to a worker queue.

func (*Checker) RefreshReliabilityCache

func (checker *Checker) RefreshReliabilityCache(ctx context.Context) error

RefreshReliabilityCache forces refreshing node online status cache.

func (*Checker) Run

func (checker *Checker) Run(ctx context.Context) (err error)

Run the checker loop.

type Config

type Config struct {
	Interval            time.Duration `help:"how frequently checker should check for bad segments" releaseDefault:"30s" devDefault:"0h0m10s"`
	IrreparableInterval time.Duration `help:"how frequently irrepairable checker should check for lost pieces" releaseDefault:"30m" devDefault:"0h0m5s"`

	ReliabilityCacheStaleness time.Duration   `help:"how stale reliable node cache can be" releaseDefault:"5m" devDefault:"5m"`
	RepairOverrides           RepairOverrides `` /* 173-byte string literal not displayed */
	// Node failure rate is an estimation based on a 6 hour checker run interval (4 checker iterations per day), a network of about 9200 nodes, and about 2 nodes churning per day.
	// This results in `2/9200/4 = 0.00005435` being the probability of any single node going down in the interval of one checker iteration.
	NodeFailureRate float64 `help:"the probability of a single node going down within the next checker iteration" default:"0.00005435"`
}

Config contains configurable values for checker.

type ReliabilityCache

type ReliabilityCache struct {
	// contains filtered or unexported fields
}

ReliabilityCache caches the reliable nodes for the specified staleness duration and updates automatically from overlay.

architecture: Service

func NewReliabilityCache

func NewReliabilityCache(overlay *overlay.Service, staleness time.Duration) *ReliabilityCache

NewReliabilityCache creates a new reliability checking cache.

func (*ReliabilityCache) LastUpdate

func (cache *ReliabilityCache) LastUpdate() time.Time

LastUpdate returns when the cache was last updated, or the zero value (time.Time{}) if it has never yet been updated. LastUpdate() does not trigger an update itself.

func (*ReliabilityCache) MissingPieces

func (cache *ReliabilityCache) MissingPieces(ctx context.Context, created time.Time, pieces []*pb.RemotePiece) (_ []int32, err error)

MissingPieces returns piece indices that are unreliable with the given staleness period.

func (*ReliabilityCache) NumNodes

func (cache *ReliabilityCache) NumNodes(ctx context.Context) (numNodes int, err error)

NumNodes returns the number of online active nodes (as determined by the reliability cache). This number is not guaranteed to be consistent with either the nodes database or the reliability cache after returning; it is just a best-effort count and should be treated as an estimate.

func (*ReliabilityCache) Refresh

func (cache *ReliabilityCache) Refresh(ctx context.Context) (err error)

Refresh refreshes the cache.

type RepairOverride

type RepairOverride struct {
	Min      int
	Success  int
	Total    int
	Override int32
}

RepairOverride is a configuration struct that contains an override repair value for a given RS k/o/n (min/success/total).

Can be used as a flag.

func (*RepairOverride) Set

func (ro *RepairOverride) Set(s string) error

Set sets the value from a string in the format k/o/n-override (min/optimal/total-repairOverride).

func (*RepairOverride) String

func (ro *RepairOverride) String() string

String is required for pflag.Value.

func (RepairOverride) Type

func (RepairOverride) Type() string

Type implements pflag.Value.

type RepairOverrides

type RepairOverrides struct {
	List []RepairOverride
}

RepairOverrides is a configuration struct that contains a list of override repair values for various given RS combinations of k/o/n (min/success/total).

Can be used as a flag.

func (*RepairOverrides) GetMap

func (ros *RepairOverrides) GetMap() RepairOverridesMap

GetMap creates a RepairOverridesMap from the config.

func (*RepairOverrides) Set

func (ros *RepairOverrides) Set(s string) error

Set sets the value from a string in the format "k/o/n-override,k/o/n-override,...".

func (*RepairOverrides) String

func (ros *RepairOverrides) String() string

String is required for pflag.Value. It is a comma separated list of RepairOverride configs.

func (RepairOverrides) Type

func (RepairOverrides) Type() string

Type implements pflag.Value.

type RepairOverridesMap

type RepairOverridesMap struct {
	// contains filtered or unexported fields
}

RepairOverridesMap is derived from the RepairOverrides config, and is used for quickly retrieving repair override values.

func (*RepairOverridesMap) GetOverrideValue

func (rom *RepairOverridesMap) GetOverrideValue(rs storj.RedundancyScheme) int32

GetOverrideValue returns the override value for an RS scheme if it exists, or 0 otherwise.

func (*RepairOverridesMap) GetOverrideValuePB

func (rom *RepairOverridesMap) GetOverrideValuePB(rs *pb.RedundancyScheme) int32

GetOverrideValuePB returns the override value for a pb RS scheme if it exists, or 0 otherwise.