verifier

Documentation

Index

• Constants
• func DuplicatedCompactionIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, ...) error
• func IndexIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, ...) error
• func OverlappedBlocksIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, _ objstore.Bucket, ...) error
• func SafeDelete(ctx context.Context, logger log.Logger, bkt objstore.Bucket, ...) error
• type Issue
• type Verifier
  • func New(logger log.Logger, bkt objstore.Bucket, issues []Issue) *Verifier
  • func NewWithRepair(logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, ...) *Verifier
  • func (v *Verifier) Verify(ctx context.Context, idMatcher func(ulid.ULID) bool) error

Constants

const DuplicatedCompactionIssueID = "duplicated_compaction"

const IndexIssueID = "index_issue"

const OverlappedBlocksIssueID = "overlapped_blocks"

Functions

func DuplicatedCompactionIssue

func DuplicatedCompactionIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool) error

DuplicatedCompactionIssue was a bug fixed in https://github.com/improbable-eng/thanos/commit/94e26c63e52ba45b713fd998638d0e7b2492664f. Bug resulted in source block not being removed immediately after compaction, so we were compacting again and again same sources until sync-delay passes. The expected print of this are same overlapped blocks with exactly the same sources, time ranges and stats. If repair is enabled, all but one duplicates are safely deleted.

func IndexIssue

func IndexIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool) error

IndexIssue verifies any known index issue. It rewrites the problematic blocks while fixing repairable inconsistencies. If the replacement was created successfully it is uploaded to the bucket and the input block is deleted. NOTE: This also verifies all indexes against chunks mismatches and duplicates.

func OverlappedBlocksIssue

func OverlappedBlocksIssue(ctx context.Context, logger log.Logger, bkt objstore.Bucket, _ objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool) error

OverlappedBlocksIssue checks bucket for blocks with overlapped time ranges. No repair is available for this issue.

func SafeDelete

func SafeDelete(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, id ulid.ULID) error

SafeDelete moves block to backup bucket and if succeeded, removes it from source bucket. It returns error if block dir already exists in backup bucket (blocks should be immutable) or any of the operation fails.

Types

type Issue

type Issue func(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool, idMatcher func(ulid.ULID) bool) error

Issue is an function that does verification and repair only if repair arg is true. It should log affected blocks using warn level logs. It should be safe for issue to run on healthy bucket.

type Verifier

type Verifier struct {
	// contains filtered or unexported fields
}

Verifier runs given issues to verify if bucket is healthy.

func New

func New(logger log.Logger, bkt objstore.Bucket, issues []Issue) *Verifier

New returns verifier that only logs affected blocks.

func NewWithRepair

func NewWithRepair(logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, issues []Issue) *Verifier

NewWithRepair returns verifier that logs affected blocks and attempts to repair them.

func (*Verifier) Verify

func (v *Verifier) Verify(ctx context.Context, idMatcher func(ulid.ULID) bool) error

Verify verifies registered issues.