stats

package
v1.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2020 License: Apache-2.0 Imports: 19 Imported by: 5

Documentation

Overview

Package stats provides primitives for recording metrics across the query path. Statistics are passed through the query context. To start a new query statistics context use: 

ctx := stats.NewContext(ctx)

Then you can update statistics by mutating data by using: 

stats.GetChunkData(ctx)
stats.GetIngesterData(ctx)
stats.GetStoreData

Finally to get a snapshot of the current query statistic use 

stats.Snapshot(ctx, time.Since(start))

Ingester statistics are sent across the GRPC stream using Trailers see https://github.com/grpc/grpc-go/blob/master/Documentation/grpc-metadata.md

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrInvalidLengthStats = fmt.Errorf("proto: negative length found during unmarshaling")
	ErrIntOverflowStats   = fmt.Errorf("proto: integer overflow")
)

Functions

func CollectTrailer 

func CollectTrailer(ctx context.Context) grpc.CallOption

CollectTrailer register a new trailer that can be collected by the engine.

func GetMutex added in v1.6.0 

func GetMutex(ctx context.Context) (*sync.Mutex, error)

GetChunkData returns the chunks statistics data from the current context.

func JoinResults added in v1.6.0 

func JoinResults(ctx context.Context, res Result) error

JoinResults merges a Result with the embedded Result in a context in a concurrency-safe manner.

func NewContext 

func NewContext(ctx context.Context) context.Context

NewContext creates a new statistics context

func SendAsTrailer 

func SendAsTrailer(ctx context.Context, stream grpc.ServerStream)

Types

type ChunkData 

type ChunkData struct {
	HeadChunkBytes    int64 `json:"headChunkBytes"`    // Total bytes processed but was already in memory. (found in the headchunk)
	HeadChunkLines    int64 `json:"headChunkLines"`    // Total lines processed but was already in memory. (found in the headchunk)
	DecompressedBytes int64 `json:"decompressedBytes"` // Total bytes decompressed and processed from chunks.
	DecompressedLines int64 `json:"decompressedLines"` // Total lines decompressed and processed from chunks.
	CompressedBytes   int64 `json:"compressedBytes"`   // Total bytes of compressed chunks (blocks) processed.
	TotalDuplicates   int64 `json:"totalDuplicates"`   // Total duplicates found while processing.
}

ChunkData contains chunks specific statistics.

func GetChunkData 

func GetChunkData(ctx context.Context) *ChunkData

GetChunkData returns the chunks statistics data from the current context.

type Ingester 

type Ingester struct {
	// Total ingester reached for this query.
	TotalReached int32 `protobuf:"varint,1,opt,name=totalReached,proto3" json:"totalReached"`
	// Total of chunks matched by the query from ingesters
	TotalChunksMatched int64 `protobuf:"varint,2,opt,name=totalChunksMatched,proto3" json:"totalChunksMatched"`
	// Total of batches sent from ingesters.
	TotalBatches int64 `protobuf:"varint,3,opt,name=totalBatches,proto3" json:"totalBatches"`
	// Total lines sent by ingesters.
	TotalLinesSent int64 `protobuf:"varint,4,opt,name=totalLinesSent,proto3" json:"totalLinesSent"`
	// Total bytes processed but was already in memory. (found in the headchunk)
	HeadChunkBytes int64 `protobuf:"varint,5,opt,name=headChunkBytes,proto3" json:"headChunkBytes"`
	// Total lines processed but was already in memory. (found in the headchunk)
	HeadChunkLines int64 `protobuf:"varint,6,opt,name=headChunkLines,proto3" json:"headChunkLines"`
	// Total bytes decompressed and processed from chunks.
	DecompressedBytes int64 `protobuf:"varint,7,opt,name=decompressedBytes,proto3" json:"decompressedBytes"`
	// Total lines decompressed and processed from chunks.
	DecompressedLines int64 `protobuf:"varint,8,opt,name=decompressedLines,proto3" json:"decompressedLines"`
	// Total bytes of compressed chunks (blocks) processed.
	CompressedBytes int64 `protobuf:"varint,9,opt,name=compressedBytes,proto3" json:"compressedBytes"`
	// Total duplicates found while processing.
	TotalDuplicates int64 `protobuf:"varint,10,opt,name=totalDuplicates,proto3" json:"totalDuplicates"`
}

func (*Ingester) Descriptor 

func (*Ingester) Descriptor() ([]byte, []int)

func (*Ingester) Equal 

func (this *Ingester) Equal(that interface{}) bool

func (*Ingester) GetCompressedBytes 

func (m *Ingester) GetCompressedBytes() int64

func (*Ingester) GetDecompressedBytes 

func (m *Ingester) GetDecompressedBytes() int64

func (*Ingester) GetDecompressedLines 

func (m *Ingester) GetDecompressedLines() int64

func (*Ingester) GetHeadChunkBytes 

func (m *Ingester) GetHeadChunkBytes() int64

func (*Ingester) GetHeadChunkLines 

func (m *Ingester) GetHeadChunkLines() int64

func (*Ingester) GetTotalBatches 

func (m *Ingester) GetTotalBatches() int64

func (*Ingester) GetTotalChunksMatched 

func (m *Ingester) GetTotalChunksMatched() int64

func (*Ingester) GetTotalDuplicates 

func (m *Ingester) GetTotalDuplicates() int64

func (*Ingester) GetTotalLinesSent 

func (m *Ingester) GetTotalLinesSent() int64

func (*Ingester) GetTotalReached 

func (m *Ingester) GetTotalReached() int32

func (*Ingester) GoString 

func (this *Ingester) GoString() string

func (*Ingester) Marshal 

func (m *Ingester) Marshal() (dAtA []byte, err error)

func (*Ingester) MarshalTo 

func (m *Ingester) MarshalTo(dAtA []byte) (int, error)

func (*Ingester) ProtoMessage 

func (*Ingester) ProtoMessage()

func (*Ingester) Reset 

func (m *Ingester) Reset()

func (*Ingester) Size 

func (m *Ingester) Size() (n int)

func (*Ingester) String 

func (this *Ingester) String() string

func (*Ingester) Unmarshal 

func (m *Ingester) Unmarshal(dAtA []byte) error

func (*Ingester) XXX_DiscardUnknown 

func (m *Ingester) XXX_DiscardUnknown()

func (*Ingester) XXX_Marshal 

func (m *Ingester) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Ingester) XXX_Merge 

func (m *Ingester) XXX_Merge(src proto.Message)

func (*Ingester) XXX_Size 

func (m *Ingester) XXX_Size() int

func (*Ingester) XXX_Unmarshal 

func (m *Ingester) XXX_Unmarshal(b []byte) error

type IngesterData 

type IngesterData struct {
	TotalChunksMatched int64 `json:"totalChunksMatched"` // Total of chunks matched by the query from ingesters
	TotalBatches       int64 `json:"totalBatches"`       // Total of batches sent from ingesters.
	TotalLinesSent     int64 `json:"totalLinesSent"`     // Total lines sent by ingesters.
}

IngesterData contains ingester specific statistics.

func GetIngesterData 

func GetIngesterData(ctx context.Context) *IngesterData

GetIngesterData returns the ingester statistics data from the current context.

type Result 

type Result struct {
	Summary  Summary  `protobuf:"bytes,1,opt,name=summary,proto3" json:"summary"`
	Store    Store    `protobuf:"bytes,2,opt,name=store,proto3" json:"store"`
	Ingester Ingester `protobuf:"bytes,3,opt,name=ingester,proto3" json:"ingester"`
}

Result contains LogQL query statistics.

func GetResult added in v1.6.0 

func GetResult(ctx context.Context) (*Result, error)

func Snapshot 

func Snapshot(ctx context.Context, execTime time.Duration) Result

Snapshot compute query statistics from a context using the total exec time.

func (*Result) ComputeSummary 

func (r *Result) ComputeSummary(execTime time.Duration)

ComputeSummary calculates the summary based on store and ingester data.

func (*Result) Descriptor 

func (*Result) Descriptor() ([]byte, []int)

func (*Result) Equal 

func (this *Result) Equal(that interface{}) bool

func (*Result) GetIngester 

func (m *Result) GetIngester() Ingester

func (*Result) GetStore 

func (m *Result) GetStore() Store

func (*Result) GetSummary 

func (m *Result) GetSummary() Summary

func (*Result) GoString 

func (this *Result) GoString() string

func (Result) Log 

func (r Result) Log(log log.Logger)

Log logs a query statistics result.

func (*Result) Marshal 

func (m *Result) Marshal() (dAtA []byte, err error)

func (*Result) MarshalTo 

func (m *Result) MarshalTo(dAtA []byte) (int, error)

func (*Result) Merge 

func (r *Result) Merge(m Result)

func (*Result) ProtoMessage 

func (*Result) ProtoMessage()

func (*Result) Reset 

func (m *Result) Reset()

func (*Result) Size 

func (m *Result) Size() (n int)

func (*Result) String 

func (this *Result) String() string

func (*Result) Unmarshal 

func (m *Result) Unmarshal(dAtA []byte) error

func (*Result) XXX_DiscardUnknown 

func (m *Result) XXX_DiscardUnknown()

func (*Result) XXX_Marshal 

func (m *Result) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Result) XXX_Merge 

func (m *Result) XXX_Merge(src proto.Message)

func (*Result) XXX_Size 

func (m *Result) XXX_Size() int

func (*Result) XXX_Unmarshal 

func (m *Result) XXX_Unmarshal(b []byte) error

type Store 

type Store struct {
	// The total of chunk reference fetched from index.
	TotalChunksRef int64 `protobuf:"varint,1,opt,name=totalChunksRef,proto3" json:"totalChunksRef"`
	// Total number of chunks fetched.
	TotalChunksDownloaded int64 `protobuf:"varint,2,opt,name=totalChunksDownloaded,proto3" json:"totalChunksDownloaded"`
	// Time spent fetching chunks in nanoseconds.
	ChunksDownloadTime float64 `protobuf:"fixed64,3,opt,name=chunksDownloadTime,proto3" json:"chunksDownloadTime"`
	// Total bytes processed but was already in memory. (found in the headchunk)
	HeadChunkBytes int64 `protobuf:"varint,4,opt,name=headChunkBytes,proto3" json:"headChunkBytes"`
	// Total lines processed but was already in memory. (found in the headchunk)
	HeadChunkLines int64 `protobuf:"varint,5,opt,name=headChunkLines,proto3" json:"headChunkLines"`
	// Total bytes decompressed and processed from chunks.
	DecompressedBytes int64 `protobuf:"varint,6,opt,name=decompressedBytes,proto3" json:"decompressedBytes"`
	// Total lines decompressed and processed from chunks.
	DecompressedLines int64 `protobuf:"varint,7,opt,name=decompressedLines,proto3" json:"decompressedLines"`
	// Total bytes of compressed chunks (blocks) processed.
	CompressedBytes int64 `protobuf:"varint,8,opt,name=compressedBytes,proto3" json:"compressedBytes"`
	// Total duplicates found while processing.
	TotalDuplicates int64 `protobuf:"varint,9,opt,name=totalDuplicates,proto3" json:"totalDuplicates"`
}

func (*Store) Descriptor 

func (*Store) Descriptor() ([]byte, []int)

func (*Store) Equal 

func (this *Store) Equal(that interface{}) bool

func (*Store) GetChunksDownloadTime 

func (m *Store) GetChunksDownloadTime() float64

func (*Store) GetCompressedBytes 

func (m *Store) GetCompressedBytes() int64

func (*Store) GetDecompressedBytes 

func (m *Store) GetDecompressedBytes() int64

func (*Store) GetDecompressedLines 

func (m *Store) GetDecompressedLines() int64

func (*Store) GetHeadChunkBytes 

func (m *Store) GetHeadChunkBytes() int64

func (*Store) GetHeadChunkLines 

func (m *Store) GetHeadChunkLines() int64

func (*Store) GetTotalChunksDownloaded 

func (m *Store) GetTotalChunksDownloaded() int64

func (*Store) GetTotalChunksRef 

func (m *Store) GetTotalChunksRef() int64

func (*Store) GetTotalDuplicates 

func (m *Store) GetTotalDuplicates() int64

func (*Store) GoString 

func (this *Store) GoString() string

func (*Store) Marshal 

func (m *Store) Marshal() (dAtA []byte, err error)

func (*Store) MarshalTo 

func (m *Store) MarshalTo(dAtA []byte) (int, error)

func (*Store) ProtoMessage 

func (*Store) ProtoMessage()

func (*Store) Reset 

func (m *Store) Reset()

func (*Store) Size 

func (m *Store) Size() (n int)

func (*Store) String 

func (this *Store) String() string

func (*Store) Unmarshal 

func (m *Store) Unmarshal(dAtA []byte) error

func (*Store) XXX_DiscardUnknown 

func (m *Store) XXX_DiscardUnknown()

func (*Store) XXX_Marshal 

func (m *Store) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Store) XXX_Merge 

func (m *Store) XXX_Merge(src proto.Message)

func (*Store) XXX_Size 

func (m *Store) XXX_Size() int

func (*Store) XXX_Unmarshal 

func (m *Store) XXX_Unmarshal(b []byte) error

type StoreData 

type StoreData struct {
	TotalChunksRef        int64         // The total of chunk reference fetched from index.
	TotalChunksDownloaded int64         // Total number of chunks fetched.
	ChunksDownloadTime    time.Duration // Time spent fetching chunks.
}

StoreData contains store specific statistics.

func GetStoreData 

func GetStoreData(ctx context.Context) *StoreData

GetStoreData returns the store statistics data from the current context.

type Summary 

type Summary struct {
	// Total bytes processed per second.
	BytesProcessedPerSecond int64 `protobuf:"varint,1,opt,name=bytesProcessedPerSecond,proto3" json:"bytesProcessedPerSecond"`
	// Total lines processed per second.
	LinesProcessedPerSecond int64 `protobuf:"varint,2,opt,name=linesProcessedPerSecond,proto3" json:"linesProcessedPerSecond"`
	// Total bytes processed.
	TotalBytesProcessed int64 `protobuf:"varint,3,opt,name=totalBytesProcessed,proto3" json:"totalBytesProcessed"`
	// Total lines processed.
	TotalLinesProcessed int64 `protobuf:"varint,4,opt,name=totalLinesProcessed,proto3" json:"totalLinesProcessed"`
	// Execution time in seconds.
	ExecTime float64 `protobuf:"fixed64,5,opt,name=execTime,proto3" json:"execTime"`
}

Summary is the summary of a query statistics.

func (*Summary) Descriptor 

func (*Summary) Descriptor() ([]byte, []int)

func (*Summary) Equal 

func (this *Summary) Equal(that interface{}) bool

func (*Summary) GetBytesProcessedPerSecond added in v1.5.0 

func (m *Summary) GetBytesProcessedPerSecond() int64

func (*Summary) GetExecTime 

func (m *Summary) GetExecTime() float64

func (*Summary) GetLinesProcessedPerSecond added in v1.5.0 

func (m *Summary) GetLinesProcessedPerSecond() int64

func (*Summary) GetTotalBytesProcessed 

func (m *Summary) GetTotalBytesProcessed() int64

func (*Summary) GetTotalLinesProcessed 

func (m *Summary) GetTotalLinesProcessed() int64

func (*Summary) GoString 

func (this *Summary) GoString() string

func (Summary) Log 

func (s Summary) Log(log log.Logger)

func (*Summary) Marshal 

func (m *Summary) Marshal() (dAtA []byte, err error)

func (*Summary) MarshalTo 

func (m *Summary) MarshalTo(dAtA []byte) (int, error)

func (*Summary) ProtoMessage 

func (*Summary) ProtoMessage()

func (*Summary) Reset 

func (m *Summary) Reset()

func (*Summary) Size 

func (m *Summary) Size() (n int)

func (*Summary) String 

func (this *Summary) String() string

func (*Summary) Unmarshal 

func (m *Summary) Unmarshal(dAtA []byte) error

func (*Summary) XXX_DiscardUnknown 

func (m *Summary) XXX_DiscardUnknown()

func (*Summary) XXX_Marshal 

func (m *Summary) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Summary) XXX_Merge 

func (m *Summary) XXX_Merge(src proto.Message)

func (*Summary) XXX_Size 

func (m *Summary) XXX_Size() int

func (*Summary) XXX_Unmarshal 

func (m *Summary) XXX_Unmarshal(b []byte) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.