checker

Documentation

Index

• Variables
• type Checker
  • func NewChecker(logger *zap.Logger, repairQueue queue.RepairQueue, irrdb irreparable.DB, ...) *Checker
  • func (checker *Checker) Close() error
  • func (checker *Checker) IdentifyInjuredSegments(ctx context.Context) (err error)
  • func (checker *Checker) IrreparableProcess(ctx context.Context) (err error)
  • func (checker *Checker) RefreshReliabilityCache(ctx context.Context) error
  • func (checker *Checker) Run(ctx context.Context) (err error)
• type Config
• type ReliabilityCache
  • func NewReliabilityCache(overlay *overlay.Service, staleness time.Duration) *ReliabilityCache
  • func (cache *ReliabilityCache) LastUpdate() time.Time
  • func (cache *ReliabilityCache) MissingPieces(ctx context.Context, created time.Time, pieces []*pb.RemotePiece) (_ []int32, err error)
  • func (cache *ReliabilityCache) Refresh(ctx context.Context) (err error)
• type RepairOverride
  • func (ro *RepairOverride) Set(s string) error
  • func (ro *RepairOverride) String() string
  • func (RepairOverride) Type() string
• type RepairOverrides
  • func (ros *RepairOverrides) GetMap() RepairOverridesMap
  • func (ros *RepairOverrides) Set(s string) error
  • func (ros *RepairOverrides) String() string
  • func (RepairOverrides) Type() string
• type RepairOverridesMap
  • func (rom *RepairOverridesMap) GetOverrideValue(rs storj.RedundancyScheme) int32
  • func (rom *RepairOverridesMap) GetOverrideValuePB(rs *pb.RedundancyScheme) int32

Variables

var (
	Error = errs.Class("checker error")
)

Error is a standard error class for this package.

Types

type Checker

type Checker struct {
	Loop            *sync2.Cycle
	IrreparableLoop *sync2.Cycle
	// contains filtered or unexported fields
}

Checker contains the information needed to do checks for missing pieces

architecture: Chore

func NewChecker

func NewChecker(logger *zap.Logger, repairQueue queue.RepairQueue, irrdb irreparable.DB, metainfo *metainfo.Service, metaLoop *metainfo.Loop, overlay *overlay.Service, config Config) *Checker

NewChecker creates a new instance of checker.

func (*Checker) Close

func (checker *Checker) Close() error

Close halts the Checker loop.

func (*Checker) IdentifyInjuredSegments

func (checker *Checker) IdentifyInjuredSegments(ctx context.Context) (err error)

IdentifyInjuredSegments checks for missing pieces off of the metainfo and overlay.

func (*Checker) IrreparableProcess

func (checker *Checker) IrreparableProcess(ctx context.Context) (err error)

IrreparableProcess iterates over all items in the irreparabledb. If an item can now be repaired then it is added to a worker queue.

func (*Checker) RefreshReliabilityCache

func (checker *Checker) RefreshReliabilityCache(ctx context.Context) error

RefreshReliabilityCache forces refreshing node online status cache.

func (*Checker) Run

func (checker *Checker) Run(ctx context.Context) (err error)

Run the checker loop.

type Config

type Config struct {
	Interval            time.Duration `help:"how frequently checker should check for bad segments" releaseDefault:"30s" devDefault:"0h0m10s"`
	IrreparableInterval time.Duration `help:"how frequently irrepairable checker should check for lost pieces" releaseDefault:"30m" devDefault:"0h0m5s"`

	ReliabilityCacheStaleness time.Duration   `help:"how stale reliable node cache can be" releaseDefault:"5m" devDefault:"5m"`
	RepairOverrides           RepairOverrides `` /* 173-byte string literal not displayed */
	// Node failure rate is an estimation based on a 6 hour checker run interval (4 checker iterations per day), a network of about 9200 nodes, and about 2 nodes churning per day.
	// This results in `2/9200/4 = 0.00005435` being the probability of any single node going down in the interval of one checker iteration.
	NodeFailureRate float64 `help:"the probability of a single node going down within the next checker iteration" default:"0.00005435"`
}

Config contains configurable values for checker.

type ReliabilityCache

type ReliabilityCache struct {
	// contains filtered or unexported fields
}

ReliabilityCache caches the reliable nodes for the specified staleness duration and updates automatically from overlay.

architecture: Service

func NewReliabilityCache

func NewReliabilityCache(overlay *overlay.Service, staleness time.Duration) *ReliabilityCache

NewReliabilityCache creates a new reliability checking cache.

func (*ReliabilityCache) LastUpdate

func (cache *ReliabilityCache) LastUpdate() time.Time

LastUpdate returns when the cache was last updated.

func (*ReliabilityCache) MissingPieces

func (cache *ReliabilityCache) MissingPieces(ctx context.Context, created time.Time, pieces []*pb.RemotePiece) (_ []int32, err error)

MissingPieces returns piece indices that are unreliable with the given staleness period.

func (*ReliabilityCache) Refresh

func (cache *ReliabilityCache) Refresh(ctx context.Context) (err error)

Refresh refreshes the cache.

type RepairOverride

type RepairOverride struct {
	Min      int
	Success  int
	Total    int
	Override int32
}

RepairOverride is a configuration struct that contains an override repair value for a given RS k/o/n (min/success/total).

Can be used as a flag.

func (*RepairOverride) Set

func (ro *RepairOverride) Set(s string) error

Set sets the value from a string in the format k/o/n-override (min/optimal/total-repairOverride).

func (*RepairOverride) String

func (ro *RepairOverride) String() string

String is required for pflag.Value.

func (RepairOverride) Type

func (RepairOverride) Type() string

Type implements pflag.Value.

type RepairOverrides

type RepairOverrides struct {
	List []RepairOverride
}

RepairOverrides is a configuration struct that contains a list of override repair values for various given RS combinations of k/o/n (min/success/total).

Can be used as a flag.

func (*RepairOverrides) GetMap

func (ros *RepairOverrides) GetMap() RepairOverridesMap

GetMap creates a RepairOverridesMap from the config.

func (*RepairOverrides) Set

func (ros *RepairOverrides) Set(s string) error

Set sets the value from a string in the format "k/o/n-override,k/o/n-override,...".

func (*RepairOverrides) String

func (ros *RepairOverrides) String() string

String is required for pflag.Value. It is a comma separated list of RepairOverride configs.

func (RepairOverrides) Type

func (RepairOverrides) Type() string

Type implements pflag.Value.

type RepairOverridesMap

type RepairOverridesMap struct {
	// contains filtered or unexported fields
}

RepairOverridesMap is derived from the RepairOverrides config, and is used for quickly retrieving repair override values.

func (*RepairOverridesMap) GetOverrideValue

func (rom *RepairOverridesMap) GetOverrideValue(rs storj.RedundancyScheme) int32

GetOverrideValue returns the override value for an RS scheme if it exists, or 0 otherwise.

func (*RepairOverridesMap) GetOverrideValuePB

func (rom *RepairOverridesMap) GetOverrideValuePB(rs *pb.RedundancyScheme) int32

GetOverrideValuePB returns the override value for a pb RS scheme if it exists, or 0 otherwise.