verifier

Documentation

Index

• func BackupAndDelete(ctx Context, id ulid.ULID) error
• func BackupAndDeleteDownloaded(ctx Context, bdir string, id ulid.ULID) error
• func TSDBBlockExistsInBucket(ctx context.Context, bkt objstore.Bucket, id ulid.ULID) (bool, error)
• type Context
• type DuplicatedCompactionBlocks
  • func (DuplicatedCompactionBlocks) IssueID() string
  • func (DuplicatedCompactionBlocks) VerifyRepair(ctx Context, idMatcher func(ulid.ULID) bool, repair bool) error
• type IndexKnownIssues
  • func (IndexKnownIssues) IssueID() string
  • func (IndexKnownIssues) VerifyRepair(ctx Context, idMatcher func(ulid.ULID) bool, repair bool) error
• type Manager
  • func NewManager(reg prometheus.Registerer, logger log.Logger, bkt, backupBkt objstore.Bucket, ...) *Manager
  • func (m *Manager) Verify(ctx context.Context, idMatcher func(ulid.ULID) bool) error
  • func (m *Manager) VerifyAndRepair(ctx context.Context, idMatcher func(ulid.ULID) bool) error
• type OverlappedBlocksIssue
  • func (OverlappedBlocksIssue) IssueID() string
  • func (OverlappedBlocksIssue) Verify(ctx Context, idMatcher func(ulid.ULID) bool) error
• type Registry
  • func (r Registry) SubstractByIDs(ids []string, repair bool) (Registry, error)
  • func (r Registry) VerifierRepairersIDs() []string
  • func (r Registry) VerifiersIDs() []string
• type Verifier
• type VerifierRepairer

Functions

func BackupAndDelete

func BackupAndDelete(ctx Context, id ulid.ULID) error

BackupAndDelete moves a TSDB block to a backup bucket and on success removes it from the source bucket. If deleteDelay is zero, block is removed from source bucket. else the block is marked for deletion. It returns error if block dir already exists in the backup bucket (blocks should be immutable) or if any of the operations fail.

func BackupAndDeleteDownloaded

func BackupAndDeleteDownloaded(ctx Context, bdir string, id ulid.ULID) error

BackupAndDeleteDownloaded works much like BackupAndDelete in that it will move a TSDB block from a bucket to a backup bucket. If deleteDelay param is zero, block is removed from source bucket. else the block is marked for deletion. The bdir parameter points to the location on disk where the TSDB block was previously downloaded allowing this function to avoid downloading the TSDB block from the source bucket again. An error is returned if any operation fails.

func TSDBBlockExistsInBucket

func TSDBBlockExistsInBucket(ctx context.Context, bkt objstore.Bucket, id ulid.ULID) (bool, error)

TSDBBlockExistsInBucket checks to see if a given TSDB block ID exists in a bucket. If so, true is returned. An error is returned on failure and in such case the boolean result has no meaning.

Types

type Context

type Context struct {
	context.Context

	Logger      log.Logger
	Bkt         objstore.Bucket
	BackupBkt   objstore.Bucket
	Fetcher     block.MetadataFetcher
	DeleteDelay time.Duration
	// contains filtered or unexported fields
}

Context is an verifier config.

type DuplicatedCompactionBlocks

type DuplicatedCompactionBlocks struct{}

DuplicatedCompactionBlocks is for bug fixed in https://github.com/thanos-io/thanos/commit/94e26c63e52ba45b713fd998638d0e7b2492664f. Bug resulted in source block not being removed immediately after compaction, so we were compacting again and again same sources until sync-delay passes. The expected print of this are same overlapped blocks with exactly the same sources, time ranges and stats. If repair is enabled, all but one duplicates are safely deleted.

func (DuplicatedCompactionBlocks) IssueID

func (DuplicatedCompactionBlocks) IssueID() string

func (DuplicatedCompactionBlocks) VerifyRepair

func (DuplicatedCompactionBlocks) VerifyRepair(ctx Context, idMatcher func(ulid.ULID) bool, repair bool) error

type IndexKnownIssues

type IndexKnownIssues struct{}

IndexKnownIssues verifies any known index issue. It rewrites the problematic blocks while fixing repairable inconsistencies. If the replacement was created successfully it is uploaded to the bucket and the input block is deleted. NOTE: This also verifies all indexes against chunks mismatches and duplicates.

func (IndexKnownIssues) IssueID

func (IndexKnownIssues) IssueID() string

func (IndexKnownIssues) VerifyRepair

func (IndexKnownIssues) VerifyRepair(ctx Context, idMatcher func(ulid.ULID) bool, repair bool) error

type Manager

type Manager struct {
	Context
	// contains filtered or unexported fields
}

Manager runs given issues to verify if bucket is healthy.

func NewManager

func NewManager(reg prometheus.Registerer, logger log.Logger, bkt, backupBkt objstore.Bucket, fetcher block.MetadataFetcher, deleteDelay time.Duration, vs Registry) *Manager

New returns verifier's manager.

func (*Manager) Verify

func (m *Manager) Verify(ctx context.Context, idMatcher func(ulid.ULID) bool) error

Verify verifies matching blocks using registered list of Verifier and VerifierRepairer. TODO(blotka): Wrap bucket with BucketWithMetrics and print metrics after each issue (e.g how many blocks where touched).

func (*Manager) VerifyAndRepair

func (m *Manager) VerifyAndRepair(ctx context.Context, idMatcher func(ulid.ULID) bool) error

VerifyAndRepair verifies and repairs matching blocks using registered list of VerifierRepairer. TODO(blotka): Wrap bucket with BucketWithMetrics and print metrics after each issue (e.g how many blocks where touched).

type OverlappedBlocksIssue

type OverlappedBlocksIssue struct{}

OverlappedBlocksIssue checks bucket for blocks with overlapped time ranges. No repair is available for this issue.

func (OverlappedBlocksIssue) IssueID

func (OverlappedBlocksIssue) IssueID() string

func (OverlappedBlocksIssue) Verify

func (OverlappedBlocksIssue) Verify(ctx Context, idMatcher func(ulid.ULID) bool) error

type Registry

type Registry struct {
	Verifiers         []Verifier
	VerifierRepairers []VerifierRepairer
}

func (Registry) SubstractByIDs

func (r Registry) SubstractByIDs(ids []string, repair bool) (Registry, error)

func (Registry) VerifierRepairersIDs

func (r Registry) VerifierRepairersIDs() []string

func (Registry) VerifiersIDs

func (r Registry) VerifiersIDs() []string

type Verifier

type Verifier interface {
	IssueID() string
	Verify(ctx Context, idMatcher func(ulid.ULID) bool) error
}

type VerifierRepairer

type VerifierRepairer interface {
	IssueID() string
	VerifyRepair(ctx Context, idMatcher func(ulid.ULID) bool, repair bool) error
}