checker

package
v1.23.0-rc-multipart Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 12, 2021 License: AGPL-3.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	Error = errs.Class("checker error")
)

Error is a standard error class for this package.

Functions

This section is empty.

Types

type Checker

type Checker struct {
	Loop            *sync2.Cycle
	IrreparableLoop *sync2.Cycle
	// contains filtered or unexported fields
}

Checker contains the information needed to do checks for missing pieces.

architecture: Chore

func NewChecker

func NewChecker(logger *zap.Logger, repairQueue queue.RepairQueue, irrdb irreparable.DB, metabase metainfo.MetabaseDB, metaLoop *metainfo.Loop, overlay *overlay.Service, config Config) *Checker

NewChecker creates a new instance of checker.

func (*Checker) Close

func (checker *Checker) Close() error

Close halts the Checker loop.

func (*Checker) IdentifyInjuredSegments

func (checker *Checker) IdentifyInjuredSegments(ctx context.Context) (err error)

IdentifyInjuredSegments checks for missing pieces off of the metainfo and overlay.

func (*Checker) IrreparableProcess

func (checker *Checker) IrreparableProcess(ctx context.Context) (err error)

IrreparableProcess iterates over all items in the irreparabledb. If an item can now be repaired then it is added to a worker queue.

func (*Checker) RefreshReliabilityCache

func (checker *Checker) RefreshReliabilityCache(ctx context.Context) error

RefreshReliabilityCache forces refreshing node online status cache.

func (*Checker) Run

func (checker *Checker) Run(ctx context.Context) (err error)

Run the checker loop.

type Config

type Config struct {
	Interval            time.Duration `help:"how frequently checker should check for bad segments" releaseDefault:"30s" devDefault:"0h0m10s"`
	IrreparableInterval time.Duration `help:"how frequently irrepairable checker should check for lost pieces" releaseDefault:"30m" devDefault:"0h0m5s"`

	ReliabilityCacheStaleness time.Duration   `help:"how stale reliable node cache can be" releaseDefault:"5m" devDefault:"5m"`
	RepairOverrides           RepairOverrides `` /* 186-byte string literal not displayed */
	// Node failure rate is an estimation based on a 6 hour checker run interval (4 checker iterations per day), a network of about 9200 nodes, and about 2 nodes churning per day.
	// This results in `2/9200/4 = 0.00005435` being the probability of any single node going down in the interval of one checker iteration.
	NodeFailureRate float64 `help:"the probability of a single node going down within the next checker iteration" default:"0.00005435"`
}

Config contains configurable values for checker.

type ReliabilityCache

type ReliabilityCache struct {
	// contains filtered or unexported fields
}

ReliabilityCache caches the reliable nodes for the specified staleness duration and updates automatically from overlay.

architecture: Service

func NewReliabilityCache

func NewReliabilityCache(overlay *overlay.Service, staleness time.Duration) *ReliabilityCache

NewReliabilityCache creates a new reliability checking cache.

func (*ReliabilityCache) LastUpdate

func (cache *ReliabilityCache) LastUpdate() time.Time

LastUpdate returns when the cache was last updated, or the zero value (time.Time{}) if it has never yet been updated. LastUpdate() does not trigger an update itself.

func (*ReliabilityCache) MissingPieces

func (cache *ReliabilityCache) MissingPieces(ctx context.Context, created time.Time, pieces metabase.Pieces) (_ []int32, err error)

MissingPieces returns piece indices that are unreliable with the given staleness period.

func (*ReliabilityCache) NumNodes added in v1.20.1

func (cache *ReliabilityCache) NumNodes(ctx context.Context) (numNodes int, err error)

NumNodes returns the number of online active nodes (as determined by the reliability cache). This number is not guaranteed to be consistent with either the nodes database or the reliability cache after returning; it is just a best-effort count and should be treated as an estimate.

func (*ReliabilityCache) Refresh

func (cache *ReliabilityCache) Refresh(ctx context.Context) (err error)

Refresh refreshes the cache.

type RepairOverride added in v1.18.1

type RepairOverride struct {
	Min      int
	Success  int
	Total    int
	Override int32
}

RepairOverride is a configuration struct that contains an override repair value for a given RS k/o/n (min/success/total).

Can be used as a flag.

func (*RepairOverride) Set added in v1.18.1

func (ro *RepairOverride) Set(s string) error

Set sets the value from a string in the format k/o/n-override (min/optimal/total-repairOverride).

func (*RepairOverride) String added in v1.18.1

func (ro *RepairOverride) String() string

String is required for pflag.Value.

func (RepairOverride) Type added in v1.18.1

func (RepairOverride) Type() string

Type implements pflag.Value.

type RepairOverrides added in v1.18.1

type RepairOverrides struct {
	List []RepairOverride
}

RepairOverrides is a configuration struct that contains a list of override repair values for various given RS combinations of k/o/n (min/success/total).

Can be used as a flag.

func (*RepairOverrides) GetMap added in v1.18.1

func (ros *RepairOverrides) GetMap() RepairOverridesMap

GetMap creates a RepairOverridesMap from the config.

func (*RepairOverrides) Set added in v1.18.1

func (ros *RepairOverrides) Set(s string) error

Set sets the value from a string in the format "k/o/n-override,k/o/n-override,...".

func (*RepairOverrides) String added in v1.18.1

func (ros *RepairOverrides) String() string

String is required for pflag.Value. It is a comma separated list of RepairOverride configs.

func (RepairOverrides) Type added in v1.18.1

func (RepairOverrides) Type() string

Type implements pflag.Value.

type RepairOverridesMap added in v1.18.1

type RepairOverridesMap struct {
	// contains filtered or unexported fields
}

RepairOverridesMap is derived from the RepairOverrides config, and is used for quickly retrieving repair override values.

func (*RepairOverridesMap) GetOverrideValue added in v1.18.1

func (rom *RepairOverridesMap) GetOverrideValue(rs storj.RedundancyScheme) int32

GetOverrideValue returns the override value for an RS scheme if it exists, or 0 otherwise.

func (*RepairOverridesMap) GetOverrideValuePB added in v1.18.1

func (rom *RepairOverridesMap) GetOverrideValuePB(rs *pb.RedundancyScheme) int32

GetOverrideValuePB returns the override value for a pb RS scheme if it exists, or 0 otherwise.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL