index

package
v2.25.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 27, 2021 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// MagicIndex 4 bytes at the head of an index file.
	MagicIndex = 0xBAAAD700
	// HeaderLen represents number of bytes reserved of index for header.
	HeaderLen = 5

	// FormatV1 represents 1 version of index.
	FormatV1 = 1
	// FormatV2 represents 2 version of index.
	FormatV2 = 2
)

Variables

This section is empty.

Functions

func AllPostingsKey 

func AllPostingsKey() (name, value string)

AllPostingsKey returns the label key that is used to store the postings list of all existing IDs.

func ExpandPostings 

func ExpandPostings(p Postings) (res []uint64, err error)

ExpandPostings returns the postings expanded as a slice.

func ReadOffsetTable 

func ReadOffsetTable(bs ByteSlice, off uint64, f func([]string, uint64, int) error) error

ReadOffsetTable reads an offset table and at the given position calls f for each found entry. If f returns an error it stops decoding and returns the received error.

Types

type ByteSlice 

type ByteSlice interface {
	Len() int
	Range(start, end int) []byte
}

ByteSlice abstracts a byte slice.

type Decoder 

type Decoder struct {
	LookupSymbol func(uint32) (string, error)
}

Decoder provides decoding methods for the v1 and v2 index file format.

It currently does not contain decoding methods for all entry types but can be extended by them if there's demand.

func (*Decoder) Postings 

func (dec *Decoder) Postings(b []byte) (int, Postings, error)

Postings returns a postings list for b and its number of elements.

func (*Decoder) Series 

func (dec *Decoder) Series(b []byte, lbls *labels.Labels, chks *[]chunks.Meta) error

Series decodes a series entry from the given byte slice into lset and chks.

type FileWriter added in v2.25.0 

type FileWriter struct {
	// contains filtered or unexported fields
}

func NewFileWriter added in v2.25.0 

func NewFileWriter(name string) (*FileWriter, error)

func (*FileWriter) AddPadding added in v2.25.0 

func (fw *FileWriter) AddPadding(size int) error

AddPadding adds zero byte padding until the file size is a multiple size.

func (*FileWriter) Close added in v2.25.0 

func (fw *FileWriter) Close() error

func (*FileWriter) Flush added in v2.25.0 

func (fw *FileWriter) Flush() error

func (*FileWriter) Pos added in v2.25.0 

func (fw *FileWriter) Pos() uint64

func (*FileWriter) Remove added in v2.25.0 

func (fw *FileWriter) Remove() error

func (*FileWriter) Write added in v2.25.0 

func (fw *FileWriter) Write(bufs ...[]byte) error

func (*FileWriter) WriteAt added in v2.25.0 

func (fw *FileWriter) WriteAt(buf []byte, pos uint64) error

type ListPostings 

type ListPostings struct {
	// contains filtered or unexported fields
}

ListPostings implements the Postings interface over a plain list.

func (*ListPostings) At 

func (it *ListPostings) At() uint64

func (*ListPostings) Err 

func (it *ListPostings) Err() error

func (*ListPostings) Next 

func (it *ListPostings) Next() bool

func (*ListPostings) Seek 

func (it *ListPostings) Seek(x uint64) bool

type MemPostings 

type MemPostings struct {
	// contains filtered or unexported fields
}

MemPostings holds postings list for series ID per label pair. They may be written to out of order. ensureOrder() must be called once before any reads are done. This allows for quick unordered batch fills on startup.

func NewMemPostings 

func NewMemPostings() *MemPostings

NewMemPostings returns a memPostings that's ready for reads and writes.

func NewUnorderedMemPostings 

func NewUnorderedMemPostings() *MemPostings

NewUnorderedMemPostings returns a memPostings that is not safe to be read from until ensureOrder was called once.

func (*MemPostings) Add 

func (p *MemPostings) Add(id uint64, lset labels.Labels)

Add a label set to the postings index.

func (*MemPostings) All 

func (p *MemPostings) All() Postings

All returns a postings list over all documents ever added.

func (*MemPostings) Delete 

func (p *MemPostings) Delete(deleted map[uint64]struct{})

Delete removes all ids in the given map from the postings lists.

func (*MemPostings) EnsureOrder 

func (p *MemPostings) EnsureOrder()

EnsureOrder ensures that all postings lists are sorted. After it returns all further calls to add and addFor will insert new IDs in a sorted manner.

func (*MemPostings) Get 

func (p *MemPostings) Get(name, value string) Postings

Get returns a postings list for the given label pair.

func (*MemPostings) Iter 

func (p *MemPostings) Iter(f func(labels.Label, Postings) error) error

Iter calls f for each postings list. It aborts if f returns an error and returns it.

func (*MemPostings) LabelNames added in v2.25.0 

func (p *MemPostings) LabelNames() []string

LabelNames returns all the unique label names.

func (*MemPostings) LabelValues added in v2.25.0 

func (p *MemPostings) LabelValues(name string) []string

LabelValues returns label values for the given name.

func (*MemPostings) SortedKeys 

func (p *MemPostings) SortedKeys() []labels.Label

SortedKeys returns a list of sorted label keys of the postings.

func (*MemPostings) Stats added in v2.25.0 

func (p *MemPostings) Stats(label string) *PostingsStats

Stats calculates the cardinality statistics from postings.

type Postings 

type Postings interface {
	// Next advances the iterator and returns true if another value was found.
	Next() bool

	// Seek advances the iterator to value v or greater and returns
	// true if a value was found.
	Seek(v uint64) bool

	// At returns the value at the current iterator position.
	At() uint64

	// Err returns the last error of the iterator.
	Err() error
}

Postings provides iterative access over a postings list.

func EmptyPostings 

func EmptyPostings() Postings

EmptyPostings returns a postings list that's always empty. NOTE: Returning EmptyPostings sentinel when index.Postings struct has no postings is recommended. It triggers optimized flow in other functions like Intersect, Without etc.

func ErrPostings 

func ErrPostings(err error) Postings

ErrPostings returns new postings that immediately error.

func Intersect 

func Intersect(its ...Postings) Postings

Intersect returns a new postings list over the intersection of the input postings.

func Merge 

func Merge(its ...Postings) Postings

Merge returns a new iterator over the union of the input iterators.

func NewListPostings 

func NewListPostings(list []uint64) Postings

func Without 

func Without(full, drop Postings) Postings

Without returns a new postings list that contains all elements from the full list that are not in the drop list.

type PostingsStats added in v2.25.0 

type PostingsStats struct {
	CardinalityMetricsStats []Stat
	CardinalityLabelStats   []Stat
	LabelValueStats         []Stat
	LabelValuePairsStats    []Stat
	NumLabelPairs           int
}

PostingsStats contains cardinality based statistics for postings.

type Range 

type Range struct {
	Start, End int64
}

Range marks a byte range.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

func NewFileReader 

func NewFileReader(path string) (*Reader, error)

NewFileReader returns a new index reader against the given index file.

func NewReader 

func NewReader(b ByteSlice) (*Reader, error)

NewReader returns a new index reader on the given byte slice. It automatically handles different format versions.

func (*Reader) Close 

func (r *Reader) Close() error

Close the reader and its underlying resources.

func (*Reader) LabelNames 

func (r *Reader) LabelNames() ([]string, error)

LabelNames returns all the unique label names present in the index.

func (*Reader) LabelValues 

func (r *Reader) LabelValues(name string) ([]string, error)

LabelValues returns value tuples that exist for the given label name. It is not safe to use the return value beyond the lifetime of the byte slice passed into the Reader.

func (*Reader) Postings 

func (r *Reader) Postings(name string, values ...string) (Postings, error)

func (*Reader) PostingsRanges 

func (r *Reader) PostingsRanges() (map[labels.Label]Range, error)

PostingsRanges returns a new map of byte range in the underlying index file for all postings lists.

func (*Reader) Series 

func (r *Reader) Series(id uint64, lbls *labels.Labels, chks *[]chunks.Meta) error

Series reads the series with the given ID and writes its labels and chunks into lbls and chks.

func (*Reader) Size 

func (r *Reader) Size() int64

Size returns the size of an index file.

func (*Reader) SortedLabelValues added in v2.25.0 

func (r *Reader) SortedLabelValues(name string) ([]string, error)

SortedLabelValues returns value tuples that exist for the given label name. It is not safe to use the return value beyond the lifetime of the byte slice passed into the Reader.

func (*Reader) SortedPostings 

func (r *Reader) SortedPostings(p Postings) Postings

SortedPostings returns the given postings list reordered so that the backing series are sorted.

func (*Reader) SymbolTableSize 

func (r *Reader) SymbolTableSize() uint64

SymbolTableSize returns the symbol table size in bytes.

func (*Reader) Symbols 

func (r *Reader) Symbols() StringIter

Symbols returns an iterator over the symbols that exist within the index.

func (*Reader) Version 

func (r *Reader) Version() int

Version returns the file format version of the underlying index.

type Stat added in v2.25.0 

type Stat struct {
	Name  string
	Count uint64
}

Stat holds values for a single cardinality statistic.

type StringIter added in v2.25.0 

type StringIter interface {
	// Next advances the iterator and returns true if another value was found.
	Next() bool

	// At returns the value at the current iterator position.
	At() string

	// Err returns the last error of the iterator.
	Err() error
}

StringIter iterates over a sorted list of strings.

func NewStringListIter added in v2.25.0 

func NewStringListIter(s []string) StringIter

NewStringListIter returns a StringIter for the given sorted list of strings.

type Symbols added in v2.25.0 

type Symbols struct {
	// contains filtered or unexported fields
}

func NewSymbols added in v2.25.0 

func NewSymbols(bs ByteSlice, version int, off int) (*Symbols, error)

NewSymbols returns a Symbols object for symbol lookups.

func (Symbols) Iter added in v2.25.0 

func (s Symbols) Iter() StringIter

func (Symbols) Lookup added in v2.25.0 

func (s Symbols) Lookup(o uint32) (string, error)

func (Symbols) ReverseLookup added in v2.25.0 

func (s Symbols) ReverseLookup(sym string) (uint32, error)

func (Symbols) Size added in v2.25.0 

func (s Symbols) Size() int

type TOC 

type TOC struct {
	Symbols           uint64
	Series            uint64
	LabelIndices      uint64
	LabelIndicesTable uint64
	Postings          uint64
	PostingsTable     uint64
}

TOC represents index Table Of Content that states where each section of index starts.

func NewTOCFromByteSlice 

func NewTOCFromByteSlice(bs ByteSlice) (*TOC, error)

NewTOCFromByteSlice return parsed TOC from given index byte slice.

type Writer 

type Writer struct {
	Version int
	// contains filtered or unexported fields
}

Writer implements the IndexWriter interface for the standard serialization format.

func NewWriter 

func NewWriter(ctx context.Context, fn string) (*Writer, error)

NewWriter returns a new Writer to the given filename. It serializes data in format version 2.

func (*Writer) AddSeries 

func (w *Writer) AddSeries(ref uint64, lset labels.Labels, chunks ...chunks.Meta) error

AddSeries adds the series one at a time along with its chunks.

func (*Writer) AddSymbol added in v2.25.0 

func (w *Writer) AddSymbol(sym string) error

func (*Writer) Close 

func (w *Writer) Close() error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.