stats

package
v1.3.25 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 6, 2024 License: MIT Imports: 29 Imported by: 2

Documentation

Overview

Package stats provides methods and functionality to register, track, log, and StatsD-notify statistics that, for the most part, include "counter" and "latency" kinds.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package stats provides methods and functionality to register, track, log, and StatsD-notify statistics that, for the most part, include "counter" and "latency" kinds.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package stats provides methods and functionality to register, track, log, and StatsD-notify statistics that, for the most part, include "counter" and "latency" kinds.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package stats provides methods and functionality to register, track, log, and StatsD-notify statistics that, for the most part, include "counter" and "latency" kinds.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package stats provides methods and functionality to register, track, log, and StatsD-notify statistics that, for the most part, include "counter" and "latency" kinds.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Index

Constants

View Source 
const (
	// lockless
	KindCounter            = "counter"
	KindTotal              = "total"
	KindSize               = "size"
	KindGauge              = "gauge"
	KindSpecial            = "special"
	KindComputedThroughput = "compbw" // disk read/write throughput
	// compound (+ semantics)
	KindLatency    = "latency"
	KindThroughput = "bw" // e.g. GetThroughput
)

enum: `statsValue` kinds

View Source 
const (
	// KindCounter:
	// all basic counters are accompanied by the corresponding (errPrefix + kind) error count:
	// e.g.: "get.n" => "err.get.n", "put.n" => "err.put.n", etc.
	GetCount    = "get.n" // GET(object) count = (cold + warm)
	PutCount    = "put.n" // ditto PUT
	HeadCount   = "head.n"
	AppendCount = "append.n"
	DeleteCount = "del.n"
	RenameCount = "ren.n"
	ListCount   = "lst.n" // list-objects

	// error counters
	// see also: `IncErr`, `regCommon`, `ioErrNames`
	ErrGetCount    = errPrefix + GetCount
	ErrPutCount    = errPrefix + PutCount
	ErrHeadCount   = errPrefix + HeadCount
	ErrAppendCount = errPrefix + AppendCount
	ErrDeleteCount = errPrefix + DeleteCount
	ErrRenameCount = errPrefix + RenameCount
	ErrListCount   = errPrefix + ListCount

	ErrKaliveCount = errPrefix + "kalive.n"

	// more errors
	// (for even more errors, see target_stats)
	ErrHTTPWriteCount = errPrefix + "http.write.n"
	ErrDownloadCount  = errPrefix + "dl.n"
	ErrPutMirrorCount = errPrefix + "put.mirror.n"

	// KindLatency
	// latency stats have numSamples used to compute average latency
	GetLatency         = "get.ns"
	GetLatencyTotal    = "get.ns.total"
	GetE2ELatencyTotal = "e2e.get.ns.total" // // e2e cold-GET latency
	ListLatency        = "lst.ns"
	KeepAliveLatency   = "kalive.ns"

	// KindSpecial
	Uptime = "up.ns.time"

	// KindGauge, cos.NodeStateFlags enum
	NodeAlerts = cos.NodeAlerts // "state.flags"
)

metrics

View Source 
const (
	LruEvictCount = "lru.evict.n"
	LruEvictSize  = "lru.evict.size"

	CleanupStoreCount = "cleanup.store.n"
	CleanupStoreSize  = "cleanup.store.size"

	VerChangeCount = "ver.change.n"
	VerChangeSize  = "ver.change.size"

	// errors
	ErrCksumCount = errPrefix + "cksum.n"
	ErrCksumSize  = errPrefix + "cksum.size"

	ErrFSHCCount = errPrefix + "fshc.n"

	// IO errors (must have ioErrPrefix)
	IOErrGetCount    = ioErrPrefix + "get.n"
	IOErrPutCount    = ioErrPrefix + "put.n"
	IOErrDeleteCount = ioErrPrefix + "del.n"

	// KindLatency
	PutLatency         = "put.ns"
	PutLatencyTotal    = "put.ns.total"
	PutE2ELatencyTotal = "e2e.put.ns.total" // e2e write-through PUT latency
	AppendLatency      = "append.ns"
	GetRedirLatency    = "get.redir.ns"
	PutRedirLatency    = "put.redir.ns"
	DownloadLatency    = "dl.ns"
	HeadLatency        = "head.ns"
	HeadLatencyTotal   = "head.ns.total"

	// Dsort
	DsortCreationReqCount    = "dsort.creation.req.n"
	DsortCreationRespCount   = "dsort.creation.resp.n"
	DsortCreationRespLatency = "dsort.creation.resp.ns"
	DsortExtractShardDskCnt  = "dsort.extract.shard.dsk.n"
	DsortExtractShardMemCnt  = "dsort.extract.shard.mem.n"
	DsortExtractShardSize    = "dsort.extract.shard.size" // uncompressed

	// Downloader
	DownloadSize = "dl.size"

	// KindThroughput
	GetThroughput = "get.bps" // bytes per second
	PutThroughput = "put.bps" // ditto

	// same as above via `.cumulative`
	GetSize = "get.size"
	PutSize = "put.size"

	// core
	RemoteDeletedDelCount = core.RemoteDeletedDelCount // compare w/ common `DeleteCount`

	LcacheCollisionCount = core.LcacheCollisionCount
	LcacheEvictedCount   = core.LcacheEvictedCount
	LcacheFlushColdCount = core.LcacheFlushColdCount
)

Naming Convention: 

-> "*.n" - counter
-> "*.ns" - latency (nanoseconds)
-> "*.size" - size (bytes)
-> "*.bps" - throughput (in byte/s)
-> "*.id" - ID

Variables

This section is empty.

Functions

func IsErrMetric added in v1.3.16 

func IsErrMetric(name string) bool

func IsIOErrMetric added in v1.3.24 

func IsIOErrMetric(name string) bool

func LatencyToCounter added in v1.3.24 

func LatencyToCounter(latency string) string

compare with base.init() at ais/backend/common

Types

type Cluster added in v1.3.16 

type Cluster struct {
	Proxy  *Node            `json:"proxy"`
	Target map[string]*Node `json:"target"`
}

type ClusterRaw added in v1.3.16 

type ClusterRaw struct {
	Proxy  *Node           `json:"proxy"`
	Target cos.JSONRawMsgs `json:"target"`
}

type Extra added in v1.3.24 

type Extra struct {
	StrName string
	Help    string
	Labels  cos.StrKVs
}

type Node added in v1.3.16 

type Node struct {
	Snode   *meta.Snode `json:"snode"`
	Tracker copyTracker `json:"tracker"`
	Tcdf    fs.Tcdf     `json:"capacity"`
}

REST API

type NodeStatus added in v1.3.16 

type NodeStatus struct {
	Node
	Cluster cos.NodeStateInfo
	RebSnap *core.Snap `json:"rebalance_snap,omitempty"`
	// assorted props
	Status         string         `json:"status"`
	DeploymentType string         `json:"deployment"`
	Version        string         `json:"ais_version"`  // major.minor.build
	BuildTime      string         `json:"build_time"`   // YYYY-MM-DD HH:MM:SS-TZ
	K8sPodName     string         `json:"k8s_pod_name"` // (via ais-k8s/operator `MY_POD` env var)
	Reserved1      string         `json:"reserved1,omitempty"`
	Reserved2      string         `json:"reserved2,omitempty"`
	MemCPUInfo     apc.MemCPUInfo `json:"sys_info"`
	SmapVersion    int64          `json:"smap_version,string"`
	Reserved3      int64          `json:"reserved3,omitempty"`
	Reserved4      int64          `json:"reserved4,omitempty"`
}

(includes stats.Node and more; NOTE: direct API call w/ no proxying)

type NodeStatusV322 added in v1.3.23 

type NodeStatusV322 struct {
	NodeV322
	RebSnap *core.Snap `json:"rebalance_snap,omitempty"`
	// assorted props
	Status         string         `json:"status"`
	DeploymentType string         `json:"deployment"`
	Version        string         `json:"ais_version"`  // major.minor.build
	BuildTime      string         `json:"build_time"`   // YYYY-MM-DD HH:MM:SS-TZ
	K8sPodName     string         `json:"k8s_pod_name"` // (via ais-k8s/operator `MY_POD` env var)
	MemCPUInfo     apc.MemCPUInfo `json:"sys_info"`
	SmapVersion    int64          `json:"smap_version,string"`
}

[backward compatibility]: includes v3.22 cdf* structures

type NodeV322 added in v1.3.23 

type NodeV322 struct {
	Snode   *meta.Snode      `json:"snode"`
	Tracker copyTracker      `json:"tracker"`
	Tcdf    fs.TargetCDFv322 `json:"capacity"`
}

[backward compatibility]: includes v3.22 cdf* structures

type Prunner 

type Prunner struct {
	// contains filtered or unexported fields
}

func (*Prunner) Add 

func (r *Prunner) Add(name string, val int64)

func (*Prunner) AddMany 

func (r *Prunner) AddMany(nvs ...cos.NamedVal64)

func (*Prunner) ClrFlag added in v1.3.24 

func (r *Prunner) ClrFlag(name string, clr cos.NodeStateFlags)

func (*Prunner) Collect 

func (r *Prunner) Collect(ch chan<- prometheus.Metric)

func (*Prunner) Describe 

func (r *Prunner) Describe(ch chan<- *prometheus.Desc)

func (*Prunner) Get 

func (r *Prunner) Get(name string) (val int64)

func (*Prunner) GetMetricNames added in v1.3.16 

func (r *Prunner) GetMetricNames() cos.StrKVs

func (*Prunner) GetStats added in v1.3.16 

func (r *Prunner) GetStats() *Node

func (*Prunner) GetStatsV322 added in v1.3.23 

func (r *Prunner) GetStatsV322() (out *NodeV322)

func (*Prunner) Inc added in v1.3.16 

func (r *Prunner) Inc(name string)

func (*Prunner) IncErr added in v1.3.16 

func (r *Prunner) IncErr(metric string)

same as above (readability)

func (*Prunner) Init 

func (r *Prunner) Init(p core.Node) *atomic.Bool

All stats that proxy currently has are CoreStats which are registered at startup

func (*Prunner) IsPrometheus 

func (*Prunner) IsPrometheus() bool

func (*Prunner) Name 

func (r *Prunner) Name() string

func (*Prunner) RegExtMetric added in v1.3.24 

func (r *Prunner) RegExtMetric(snode *meta.Snode, name, kind string, extra *Extra)

func (*Prunner) ResetStats added in v1.3.16 

func (r *Prunner) ResetStats(errorsOnly bool)

func (*Prunner) Run 

func (r *Prunner) Run() error

func (*Prunner) SetClrFlag added in v1.3.24 

func (r *Prunner) SetClrFlag(name string, set, clr cos.NodeStateFlags)

func (*Prunner) SetFlag added in v1.3.24 

func (r *Prunner) SetFlag(name string, set cos.NodeStateFlags)

func (*Prunner) StartedUp 

func (r *Prunner) StartedUp() bool

func (*Prunner) Stop 

func (r *Prunner) Stop(err error)

type Tracker 

type Tracker interface {
	cos.StatsUpdater

	StartedUp() bool

	IsPrometheus() bool

	IncErr(metric string)

	GetStats() *Node
	GetStatsV322() *NodeV322 // [backward compatibility]

	ResetStats(errorsOnly bool)
	GetMetricNames() cos.StrKVs // (name, kind) pairs

	// for aistore modules, to add their respective metrics
	RegExtMetric(node *meta.Snode, name, kind string, extra *Extra)
}

type Trunner 

type Trunner struct {
	Tcdf fs.Tcdf `json:"cdf"`
	// contains filtered or unexported fields
}

func NewTrunner added in v1.3.16 

func NewTrunner(t core.Target) *Trunner

func (*Trunner) Add 

func (r *Trunner) Add(name string, val int64)

func (*Trunner) AddMany 

func (r *Trunner) AddMany(nvs ...cos.NamedVal64)

func (*Trunner) ClrFlag added in v1.3.24 

func (r *Trunner) ClrFlag(name string, clr cos.NodeStateFlags)

func (*Trunner) Collect 

func (r *Trunner) Collect(ch chan<- prometheus.Metric)

func (*Trunner) Describe 

func (r *Trunner) Describe(ch chan<- *prometheus.Desc)

func (*Trunner) Get 

func (r *Trunner) Get(name string) (val int64)

func (*Trunner) GetMetricNames added in v1.3.16 

func (r *Trunner) GetMetricNames() cos.StrKVs

func (*Trunner) GetStats added in v1.3.16 

func (r *Trunner) GetStats() (ds *Node)

func (*Trunner) GetStatsV322 added in v1.3.23 

func (r *Trunner) GetStatsV322() (out *NodeV322)

[backward compatibility] v3.22 and prior

func (*Trunner) Inc added in v1.3.16 

func (r *Trunner) Inc(name string)

func (*Trunner) IncErr added in v1.3.16 

func (r *Trunner) IncErr(metric string)

same as above (readability)

func (*Trunner) Init 

func (r *Trunner) Init() *atomic.Bool

func (*Trunner) InitCDF added in v1.3.16 

func (r *Trunner) InitCDF(config *cmn.Config) error

func (*Trunner) IsPrometheus 

func (*Trunner) IsPrometheus() bool

func (*Trunner) Name 

func (r *Trunner) Name() string

func (*Trunner) RegDiskMetrics 

func (r *Trunner) RegDiskMetrics(snode *meta.Snode, disk string)

func (*Trunner) RegExtMetric added in v1.3.24 

func (r *Trunner) RegExtMetric(snode *meta.Snode, name, kind string, extra *Extra)

func (*Trunner) RegMetrics 

func (r *Trunner) RegMetrics(snode *meta.Snode)

target-specific metrics, in addition to common and already added via regCommon()

func (*Trunner) ResetStats added in v1.3.16 

func (r *Trunner) ResetStats(errorsOnly bool)

func (*Trunner) Run 

func (r *Trunner) Run() error

func (*Trunner) SetClrFlag added in v1.3.24 

func (r *Trunner) SetClrFlag(name string, set, clr cos.NodeStateFlags)

func (*Trunner) SetFlag added in v1.3.24 

func (r *Trunner) SetFlag(name string, set cos.NodeStateFlags)

func (*Trunner) Standby 

func (r *Trunner) Standby(v bool)

func (*Trunner) StartedUp 

func (r *Trunner) StartedUp() bool

func (*Trunner) Stop 

func (r *Trunner) Stop(err error)

Source Files

View all Source files

Directories

Path Synopsis
Package statsd provides a client to send basic statd metrics (timer, counter and gauge) to listening UDP StatsD server.
 Package statsd provides a client to send basic statd metrics (timer, counter and gauge) to listening UDP StatsD server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.