statistics

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 12, 2021 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// DefaultAotSize is default size of average over time.
	DefaultAotSize = 2
	// DefaultWriteMfSize is default size of write median filter
	DefaultWriteMfSize = 5
	// DefaultReadMfSize is default size of read median filter
	DefaultReadMfSize = 3
)
View Source 
const (
	// ContainerHeartBeatReportInterval is the heartbeat report interval of a container.
	ContainerHeartBeatReportInterval = 10
	// ResourceHeartBeatReportInterval is the heartbeat report interval of a resource.
	ResourceHeartBeatReportInterval = 60
)

Variables

View Source 
var Denoising = true

Denoising is an option to calculate flow base on the real heartbeats. Should only turned off by the simulator and the test.

Functions

func NewContainerStatisticsMap 

func NewContainerStatisticsMap(opt *config.PersistOptions) *containerStatisticsMap

NewContainerStatisticsMap create a container statistics map

Types

type ContainerHotPeersInfos 

type ContainerHotPeersInfos struct {
	AsPeer   ContainerHotPeersStat `json:"as_peer"`
	AsLeader ContainerHotPeersStat `json:"as_leader"`
}

ContainerHotPeersInfos is used to get human-readable description for hot resources.

func (*ContainerHotPeersInfos) GetContainerStatAsLeader 

func (info *ContainerHotPeersInfos) GetContainerStatAsLeader(containerID uint64) (string, *HotPeersStat)

GetContainerStatAsLeader returns stat stat as leader from the corresponding container.

func (*ContainerHotPeersInfos) GetContainerStatAsPeer 

func (info *ContainerHotPeersInfos) GetContainerStatAsPeer(containerID uint64) (string, *HotPeersStat)

GetContainerStatAsPeer returns stat as peer from the corresponding container.

type ContainerHotPeersStat 

type ContainerHotPeersStat map[uint64]*HotPeersStat

ContainerHotPeersStat is used to record the hot resource statistics group by container.

type ContainerStatInformer 

type ContainerStatInformer interface {
	GetContainersStats() *ContainersStats
}

ContainerStatInformer provides access to a shared informer of statistics.

type ContainersStats 

type ContainersStats struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

ContainersStats is a cache hold hot resources.

func NewContainersStats 

func NewContainersStats() *ContainersStats

NewContainersStats creates a new hot spot cache.

func (*ContainersStats) FilterUnhealthyContainer 

func (s *ContainersStats) FilterUnhealthyContainer(cluster core.ContainerSetInformer)

FilterUnhealthyContainer filter unhealthy container

func (*ContainersStats) GetContainerBytesRate 

func (s *ContainersStats) GetContainerBytesRate(containerID uint64) (writeRate float64, readRate float64)

GetContainerBytesRate returns the bytes write stat of the specified container.

func (*ContainersStats) GetContainerBytesReadRate 

func (s *ContainersStats) GetContainerBytesReadRate(containerID uint64) float64

GetContainerBytesReadRate returns the bytes read stat of the specified container.

func (*ContainersStats) GetContainerBytesWriteRate 

func (s *ContainersStats) GetContainerBytesWriteRate(containerID uint64) float64

GetContainerBytesWriteRate returns the bytes write stat of the specified container.

func (*ContainersStats) GetContainerCPUUsage 

func (s *ContainersStats) GetContainerCPUUsage(containerID uint64) float64

GetContainerCPUUsage returns the total cpu usages of threads of the specified container.

func (*ContainersStats) GetContainerDiskReadRate 

func (s *ContainersStats) GetContainerDiskReadRate(containerID uint64) float64

GetContainerDiskReadRate returns the total read disk io rate of threads of the specified container.

func (*ContainersStats) GetContainerDiskWriteRate 

func (s *ContainersStats) GetContainerDiskWriteRate(containerID uint64) float64

GetContainerDiskWriteRate returns the total write disk io rate of threads of the specified container.

func (*ContainersStats) GetContainersBytesReadStat 

func (s *ContainersStats) GetContainersBytesReadStat() map[uint64]float64

GetContainersBytesReadStat returns the bytes read stat of all CachedContainer.

func (*ContainersStats) GetContainersBytesWriteStat 

func (s *ContainersStats) GetContainersBytesWriteStat() map[uint64]float64

GetContainersBytesWriteStat returns the bytes write stat of all CachedContainer.

func (*ContainersStats) GetContainersCPUUsage 

func (s *ContainersStats) GetContainersCPUUsage(cluster core.ContainerSetInformer) map[uint64]float64

GetContainersCPUUsage returns the cpu usage stat of all CachedContainer.

func (*ContainersStats) GetContainersDiskReadRate 

func (s *ContainersStats) GetContainersDiskReadRate() map[uint64]float64

GetContainersDiskReadRate returns the disk read rate stat of all CachedContainer.

func (*ContainersStats) GetContainersDiskWriteRate 

func (s *ContainersStats) GetContainersDiskWriteRate() map[uint64]float64

GetContainersDiskWriteRate returns the disk write rate stat of all CachedContainer.

func (*ContainersStats) GetContainersKeysReadStat 

func (s *ContainersStats) GetContainersKeysReadStat() map[uint64]float64

GetContainersKeysReadStat returns the bytes read stat of all CachedContainer.

func (*ContainersStats) GetContainersKeysWriteStat 

func (s *ContainersStats) GetContainersKeysWriteStat() map[uint64]float64

GetContainersKeysWriteStat returns the keys write stat of all CachedContainer.

func (*ContainersStats) GetOrCreateRollingContainerStats 

func (s *ContainersStats) GetOrCreateRollingContainerStats(containerID uint64) *RollingContainerStats

GetOrCreateRollingContainerStats gets or creates RollingContainerStats with a given container ID.

func (*ContainersStats) GetRollingContainerStats 

func (s *ContainersStats) GetRollingContainerStats(containerID uint64) *RollingContainerStats

GetRollingContainerStats gets RollingContainerStats with a given container ID.

func (*ContainersStats) Observe 

func (s *ContainersStats) Observe(containerID uint64, stats *rpcpb.ContainerStats)

Observe records the current container status with a given container.

func (*ContainersStats) RemoveRollingContainerStats 

func (s *ContainersStats) RemoveRollingContainerStats(ContainerID uint64)

RemoveRollingContainerStats removes RollingContainerStats with a given container ID.

func (*ContainersStats) Set 

func (s *ContainersStats) Set(containerID uint64, stats *rpcpb.ContainerStats)

Set sets the container statistics (for test).

func (*ContainersStats) TotalBytesReadRate 

func (s *ContainersStats) TotalBytesReadRate() float64

TotalBytesReadRate returns the total read bytes rate of all CachedContainer.

func (*ContainersStats) TotalBytesWriteRate 

func (s *ContainersStats) TotalBytesWriteRate() float64

TotalBytesWriteRate returns the total written bytes rate of all CachedContainer.

func (*ContainersStats) TotalKeysReadRate 

func (s *ContainersStats) TotalKeysReadRate() float64

TotalKeysReadRate returns the total read keys rate of all CachedContainer.

func (*ContainersStats) TotalKeysWriteRate 

func (s *ContainersStats) TotalKeysWriteRate() float64

TotalKeysWriteRate returns the total written keys rate of all CachedContainer.

func (*ContainersStats) UpdateContainerHeartbeatMetrics 

func (s *ContainersStats) UpdateContainerHeartbeatMetrics(container *core.CachedContainer)

UpdateContainerHeartbeatMetrics is used to update container heartbeat interval metrics

func (*ContainersStats) UpdateTotalBytesRate 

func (s *ContainersStats) UpdateTotalBytesRate(f func() []*core.CachedContainer)

UpdateTotalBytesRate updates the total bytes write rate and read rate.

func (*ContainersStats) UpdateTotalKeysRate 

func (s *ContainersStats) UpdateTotalKeysRate(f func() []*core.CachedContainer)

UpdateTotalKeysRate updates the total keys write rate and read rate.

type FlowKind 

type FlowKind uint32

FlowKind is a identify Flow types. 

const (
	WriteFlow FlowKind = iota
	ReadFlow
)

Flags for flow.

func (FlowKind) String 

func (k FlowKind) String() string

type HotCache 

type HotCache struct {
	// contains filtered or unexported fields
}

HotCache is a cache hold hot resources.

func NewHotCache 

func NewHotCache() *HotCache

NewHotCache creates a new hot spot cache.

func (*HotCache) CheckRead 

func (w *HotCache) CheckRead(res *core.CachedResource) []*HotPeerStat

CheckRead checks the read status, returns update items.

func (*HotCache) CheckWrite 

func (w *HotCache) CheckWrite(res *core.CachedResource) []*HotPeerStat

CheckWrite checks the write status, returns update items.

func (*HotCache) CollectMetrics 

func (w *HotCache) CollectMetrics()

CollectMetrics collects the hot cache metrics.

func (*HotCache) GetFilledPeriod 

func (w *HotCache) GetFilledPeriod(kind FlowKind) int

GetFilledPeriod returns filled period.

func (*HotCache) IsResourceHot 

func (w *HotCache) IsResourceHot(res *core.CachedResource, hotDegree int) bool

IsResourceHot checks if the resource is hot.

func (*HotCache) RandHotResourceFromContainer 

func (w *HotCache) RandHotResourceFromContainer(containerID uint64, kind FlowKind, hotDegree int) *HotPeerStat

RandHotResourceFromContainer random picks a hot resource in specify container.

func (*HotCache) ResetMetrics 

func (w *HotCache) ResetMetrics()

ResetMetrics resets the hot cache metrics.

func (*HotCache) ResourceStats 

func (w *HotCache) ResourceStats(kind FlowKind) map[uint64][]*HotPeerStat

ResourceStats returns hot items according to kind

func (*HotCache) Update 

func (w *HotCache) Update(item *HotPeerStat)

Update updates the cache.

type HotPeerStat 

type HotPeerStat struct {
	ContainerID uint64 `json:"container_id"`
	ResourceID  uint64 `json:"resource_id"`

	// HotDegree records the times for the resource considered as hot spot during each HandleResourceHeartbeat
	HotDegree int `json:"hot_degree"`
	// AntiCount used to eliminate some noise when remove resource in cache
	AntiCount int `json:"anti_count"`

	Kind     FlowKind `json:"-"`
	ByteRate float64  `json:"flow_bytes"`
	KeyRate  float64  `json:"flow_keys"`

	// LastUpdateTime used to calculate average write
	LastUpdateTime time.Time `json:"last_update_time"`
	// contains filtered or unexported fields
}

HotPeerStat records each hot peer's statistics

func (*HotPeerStat) Clone 

func (stat *HotPeerStat) Clone() *HotPeerStat

Clone clones the HotPeerStat

func (*HotPeerStat) GetByteRate 

func (stat *HotPeerStat) GetByteRate() float64

GetByteRate returns denoised BytesRate if possible.

func (*HotPeerStat) GetKeyRate 

func (stat *HotPeerStat) GetKeyRate() float64

GetKeyRate returns denoised KeysRate if possible.

func (*HotPeerStat) ID 

func (stat *HotPeerStat) ID() uint64

ID returns resource ID. Implementing TopNItem.

func (*HotPeerStat) IsLeader 

func (stat *HotPeerStat) IsLeader() bool

IsLeader indicates the item belong to the leader.

func (*HotPeerStat) IsNeedDelete 

func (stat *HotPeerStat) IsNeedDelete() bool

IsNeedDelete to delete the item in cache.

func (*HotPeerStat) IsNew 

func (stat *HotPeerStat) IsNew() bool

IsNew indicates the item is first update in the cache of the resource.

func (*HotPeerStat) Less 

func (stat *HotPeerStat) Less(k int, than TopNItem) bool

Less compares two HotPeerStat.Implementing TopNItem.

type HotPeersStat 

type HotPeersStat struct {
	TotalBytesRate float64       `json:"total_flow_bytes"`
	TotalKeysRate  float64       `json:"total_flow_keys"`
	Count          int           `json:"resources_count"`
	Stats          []HotPeerStat `json:"statistics"`
}

HotPeersStat records all hot resources statistics

type HotStat 

type HotStat struct {
	*HotCache
	*ContainersStats
}

HotStat contains cluster's hotspot statistics.

func NewHotStat 

func NewHotStat() *HotStat

NewHotStat creates the container to hold cluster's hotspot statistics.

type LabelStatistics 

type LabelStatistics struct {
	// contains filtered or unexported fields
}

LabelStatistics is the statistics of the level of labels.

func NewLabelStatistics 

func NewLabelStatistics() *LabelStatistics

NewLabelStatistics creates a new LabelStatistics.

func (*LabelStatistics) ClearDefunctResource 

func (l *LabelStatistics) ClearDefunctResource(resID uint64, labels []string)

ClearDefunctResource is used to handle the overlap resource.

func (*LabelStatistics) Collect 

func (l *LabelStatistics) Collect()

Collect collects the metrics of the label status.

func (*LabelStatistics) Observe 

func (l *LabelStatistics) Observe(res *core.CachedResource, containers []*core.CachedContainer, labels []string)

Observe records the current label status.

func (*LabelStatistics) Reset 

func (l *LabelStatistics) Reset()

Reset resets the metrics of the label status.

type ResourceStatInformer 

type ResourceStatInformer interface {
	IsResourceHot(res *core.CachedResource) bool
	// ResourceWriteStats return the containerID -> write stat of peers on this container
	ResourceWriteStats() map[uint64][]*HotPeerStat
	// ResourceReadStats return the containerID -> read stat of peers on this container
	ResourceReadStats() map[uint64][]*HotPeerStat
	RandHotResourceFromContainer(container uint64, kind FlowKind) *core.CachedResource
}

ResourceStatInformer provides access to a shared informer of statistics.

type ResourceStatisticType 

type ResourceStatisticType uint32

ResourceStatisticType represents the type of the resource's status. 

const (
	MissPeer ResourceStatisticType = 1 << iota
	ExtraPeer
	DownPeer
	PendingPeer
	OfflinePeer
	LearnerPeer
	EmptyResource
)

resource status type

type ResourceStatistics 

type ResourceStatistics struct {
	// contains filtered or unexported fields
}

ResourceStatistics is used to record the status of resources.

func NewResourceStatistics 

func NewResourceStatistics(opt *config.PersistOptions, ruleManager *placement.RuleManager) *ResourceStatistics

NewResourceStatistics creates a new ResourceStatistics.

func (*ResourceStatistics) ClearDefunctResource 

func (r *ResourceStatistics) ClearDefunctResource(resID uint64)

ClearDefunctResource is used to handle the overlap resource.

func (*ResourceStatistics) Collect 

func (r *ResourceStatistics) Collect()

Collect collects the metrics of the resources' status.

func (*ResourceStatistics) GetResourceStatsByType 

func (r *ResourceStatistics) GetResourceStatsByType(typ ResourceStatisticType) []*core.CachedResource

GetResourceStatsByType gets the status of the resource by types.

func (*ResourceStatistics) Observe 

func (r *ResourceStatistics) Observe(res *core.CachedResource, containers []*core.CachedContainer)

Observe records the current resources' status.

func (*ResourceStatistics) Reset 

func (r *ResourceStatistics) Reset()

Reset resets the metrics of the resources' status.

type ResourceStats 

type ResourceStats struct {
	Count                int              `json:"count"`
	EmptyCount           int              `json:"empty_count"`
	StorageSize          int64            `json:"storage_size"`
	StorageKeys          int64            `json:"storage_keys"`
	ContainerLeaderCount map[uint64]int   `json:"container_leader_count"`
	ContainerPeerCount   map[uint64]int   `json:"container_peer_count"`
	ContainerLeaderSize  map[uint64]int64 `json:"container_leader_size"`
	ContainerLeaderKeys  map[uint64]int64 `json:"container_leader_keys"`
	ContainerPeerSize    map[uint64]int64 `json:"container_peer_size"`
	ContainerPeerKeys    map[uint64]int64 `json:"container_peer_keys"`
}

ResourceStats records a list of resources' statistics and distribution status.

func GetResourceStats 

func GetResourceStats(resources []*core.CachedResource) *ResourceStats

GetResourceStats sums resources' statistics.

func (*ResourceStats) Observe 

func (s *ResourceStats) Observe(r *core.CachedResource)

Observe adds a resource's statistics into ResourceStats.

type RollingContainerStats 

type RollingContainerStats struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

RollingContainerStats are multiple sets of recent historical records with specified windows size.

func (*RollingContainerStats) GetBytesRate 

func (r *RollingContainerStats) GetBytesRate() (writeRate float64, readRate float64)

GetBytesRate returns the bytes write rate and the bytes read rate.

func (*RollingContainerStats) GetBytesRateInstantaneous 

func (r *RollingContainerStats) GetBytesRateInstantaneous() (writeRate float64, readRate float64)

GetBytesRateInstantaneous returns the bytes write rate and the bytes read rate instantaneously.

func (*RollingContainerStats) GetBytesReadRate 

func (r *RollingContainerStats) GetBytesReadRate() float64

GetBytesReadRate returns the bytes read rate.

func (*RollingContainerStats) GetBytesWriteRate 

func (r *RollingContainerStats) GetBytesWriteRate() float64

GetBytesWriteRate returns the bytes write rate.

func (*RollingContainerStats) GetCPUUsage 

func (r *RollingContainerStats) GetCPUUsage() float64

GetCPUUsage returns the total cpu usages of threads in the container.

func (*RollingContainerStats) GetDiskReadRate 

func (r *RollingContainerStats) GetDiskReadRate() float64

GetDiskReadRate returns the total read disk io rate of threads in the container.

func (*RollingContainerStats) GetDiskWriteRate 

func (r *RollingContainerStats) GetDiskWriteRate() float64

GetDiskWriteRate returns the total write disk io rate of threads in the container.

func (*RollingContainerStats) GetKeysRate 

func (r *RollingContainerStats) GetKeysRate() (writeRate float64, readRate float64)

GetKeysRate returns the keys write rate and the keys read rate.

func (*RollingContainerStats) GetKeysRateInstantaneous 

func (r *RollingContainerStats) GetKeysRateInstantaneous() (writeRate float64, readRate float64)

GetKeysRateInstantaneous returns the keys write rate and the keys read rate instantaneously.

func (*RollingContainerStats) GetKeysReadRate 

func (r *RollingContainerStats) GetKeysReadRate() float64

GetKeysReadRate returns the keys read rate.

func (*RollingContainerStats) GetKeysWriteRate 

func (r *RollingContainerStats) GetKeysWriteRate() float64

GetKeysWriteRate returns the keys write rate.

func (*RollingContainerStats) Observe 

func (r *RollingContainerStats) Observe(stats *rpcpb.ContainerStats)

Observe records current statistics.

func (*RollingContainerStats) Set 

func (r *RollingContainerStats) Set(stats *rpcpb.ContainerStats)

Set sets the statistics (for test).

type TopN 

type TopN struct {
	// contains filtered or unexported fields
}

TopN maintains the N largest items of multiple dimensions.

func NewTopN 

func NewTopN(k, n int, ttl time.Duration) *TopN

NewTopN returns a k-dimensional TopN with given TTL. NOTE: panic if k <= 0 or n <= 0.

func (*TopN) Get 

func (tn *TopN) Get(id uint64) TopNItem

Get returns the item with given id, nil if there is no such item.

func (*TopN) GetAll 

func (tn *TopN) GetAll() []TopNItem

GetAll returns all items.

func (*TopN) GetAllTopN 

func (tn *TopN) GetAllTopN(k int) []TopNItem

GetAllTopN returns the top N items of the `k`th dimension.

func (*TopN) GetTopNMin 

func (tn *TopN) GetTopNMin(k int) TopNItem

GetTopNMin returns the min item in top N of the `k`th dimension.

func (*TopN) Len 

func (tn *TopN) Len() int

Len returns number of all items.

func (*TopN) Put 

func (tn *TopN) Put(item TopNItem) (isUpdate bool)

Put inserts item or updates the old item if it exists.

func (*TopN) Remove 

func (tn *TopN) Remove(id uint64) (item TopNItem)

Remove deletes the item by given ID and returns it.

func (*TopN) RemoveExpired 

func (tn *TopN) RemoveExpired()

RemoveExpired deletes all expired items.

type TopNItem 

type TopNItem interface {
	// ID is used to check identity.
	ID() uint64
	// Less tests whether the current item is less than the given argument in the `k`th dimension.
	Less(k int, than TopNItem) bool
}

TopNItem represents a single object in TopN.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.