shipper

Documentation

Index

• Constants
• func WriteMetaFile(logger log.Logger, dir string, meta *Meta) error
• type BlockLister
• type Meta
  • func ReadMetaFile(dir string) (*Meta, error)
• type Shipper
  • func New(logger log.Logger, r prometheus.Registerer, blockLister BlockLister, ...) *Shipper
  • func (s *Shipper) Sync(ctx context.Context) (uploaded int, err error)
  • func (s *Shipper) Timestamps() (minTime, maxSyncTime model.Time, err error)

Constants

const (
	// MetaFilename is the known JSON filename for meta information.
	MetaFilename = "shipper.json"

	// MetaVersion1 represents 1 version of meta.
	MetaVersion1 = 1
)

Functions

func WriteMetaFile

func WriteMetaFile(logger log.Logger, dir string, meta *Meta) error

WriteMetaFile writes the given meta into <dir>/thanos.shipper.json.

Types

type BlockLister

type BlockLister interface {
	LocalDataPath() string
	BlockMetas(ctx context.Context) ([]*block.Meta, error)
}

type Meta

type Meta struct {
	Version  int         `json:"version"`
	Uploaded []ulid.ULID `json:"uploaded"`
}

Meta defines the format thanos.shipper.json file that the shipper places in the data directory.

func ReadMetaFile

func ReadMetaFile(dir string) (*Meta, error)

ReadMetaFile reads the given meta from <dir>/shipper.json.

type Shipper

type Shipper struct {
	// contains filtered or unexported fields
}

Shipper watches a directory for matching files and directories and uploads them to a remote data store.

func New

func New(
	logger log.Logger,
	r prometheus.Registerer,
	blockLister BlockLister,
	bucket objstore.Bucket,
	source block.SourceType,
	uploadCompacted bool,
	allowOutOfOrderUploads bool,
) *Shipper

New creates a new shipper that detects new TSDB blocks in dir and uploads them to remote if necessary. It attaches the Thanos metadata section in each meta JSON file. If uploadCompacted is enabled, it also uploads compacted blocks which are already in filesystem.

func (*Shipper) Sync

func (s *Shipper) Sync(ctx context.Context) (uploaded int, err error)

Sync performs a single synchronization, which ensures all non-compacted local blocks have been uploaded to the object bucket once.

If uploaded.

It is not concurrency-safe, however it is compactor-safe (running concurrently with compactor is ok).

func (*Shipper) Timestamps

func (s *Shipper) Timestamps() (minTime, maxSyncTime model.Time, err error)

Timestamps returns the minimum timestamp for which data is available and the highest timestamp of blocks that were successfully uploaded.