statistics

package
v3.0.16 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 28, 2020 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// RegionHeartBeatReportInterval is the heartbeat report interval of a region.
	RegionHeartBeatReportInterval = 60
	// StoreHeartBeatReportInterval is the heartbeat report interval of a store.
	StoreHeartBeatReportInterval = 10
)

Variables

View Source 
var Simulating bool

Simulating is an option to overpass the impact of accelerated time. Should only turned on by the simulator.

Functions

func NewStoreStatisticsMap 

func NewStoreStatisticsMap(opt ScheduleOptions, classifier namespace.Classifier) *storeStatisticsMap

NewStoreStatisticsMap creates a new storeStatisticsMap.

Types

type FlowKind 

type FlowKind uint32

FlowKind is a identify Flow types. 

const (
	WriteFlow FlowKind = iota
	ReadFlow
)

Flags for flow.

type HotRegionsStat 

type HotRegionsStat struct {
	TotalFlowBytes uint64      `json:"total_flow_bytes"`
	RegionsCount   int         `json:"regions_count"`
	RegionsStat    RegionsStat `json:"statistics"`
}

HotRegionsStat records all hot regions statistics

type HotSpotCache 

type HotSpotCache struct {
	// contains filtered or unexported fields
}

HotSpotCache is a cache hold hot regions.

func NewHotSpotCache 

func NewHotSpotCache() *HotSpotCache

NewHotSpotCache creates a new hot spot cache.

func (*HotSpotCache) CheckRead 

func (w *HotSpotCache) CheckRead(region *core.RegionInfo, stats *StoresStats) (bool, *RegionStat)

CheckRead checks the read status, returns whether need update statistics and item.

func (*HotSpotCache) CheckWrite 

func (w *HotSpotCache) CheckWrite(region *core.RegionInfo, stats *StoresStats) (bool, *RegionStat)

CheckWrite checks the write status, returns whether need update statistics and item.

func (*HotSpotCache) CollectMetrics 

func (w *HotSpotCache) CollectMetrics(stats *StoresStats)

CollectMetrics collect the hot cache metrics

func (*HotSpotCache) IsRegionHot 

func (w *HotSpotCache) IsRegionHot(id uint64, hotThreshold int) bool

IsRegionHot checks if the region is hot.

func (*HotSpotCache) RandHotRegionFromStore 

func (w *HotSpotCache) RandHotRegionFromStore(storeID uint64, kind FlowKind, hotThreshold int) *RegionStat

RandHotRegionFromStore random picks a hot region in specify store.

func (*HotSpotCache) RegionStats 

func (w *HotSpotCache) RegionStats(kind FlowKind) []*RegionStat

RegionStats returns hot items according to kind

func (*HotSpotCache) ResetMetrics 

func (w *HotSpotCache) ResetMetrics()

ResetMetrics resets the hot cache metrics.

func (*HotSpotCache) Update 

func (w *HotSpotCache) Update(key uint64, item *RegionStat, kind FlowKind)

Update updates the cache.

type LabelLevelStatistics 

type LabelLevelStatistics struct {
	// contains filtered or unexported fields
}

LabelLevelStatistics is the statistics of the level of labels.

func NewLabelLevelStatistics 

func NewLabelLevelStatistics() *LabelLevelStatistics

NewLabelLevelStatistics creates a new LabelLevelStatistics.

func (*LabelLevelStatistics) ClearDefunctRegion 

func (l *LabelLevelStatistics) ClearDefunctRegion(regionID uint64, labels []string)

ClearDefunctRegion is used to handle the overlap region.

func (*LabelLevelStatistics) Collect 

func (l *LabelLevelStatistics) Collect()

Collect collects the metrics of the label status.

func (*LabelLevelStatistics) Observe 

func (l *LabelLevelStatistics) Observe(region *core.RegionInfo, stores []*core.StoreInfo, labels []string)

Observe records the current label status.

func (*LabelLevelStatistics) Reset 

func (l *LabelLevelStatistics) Reset()

Reset resets the metrics of the label status.

type RegionStat 

type RegionStat struct {
	RegionID  uint64 `json:"region_id"`
	FlowBytes uint64 `json:"flow_bytes"`
	// HotDegree records the hot region update times
	HotDegree int `json:"hot_degree"`
	// LastUpdateTime used to calculate average write
	LastUpdateTime time.Time `json:"last_update_time"`
	StoreID        uint64    `json:"-"`
	// AntiCount used to eliminate some noise when remove region in cache
	AntiCount int
	// Version used to check the region split times
	Version uint64
	// Stats is a rolling statistics, recording some recently added records.
	Stats *RollingStats
}

RegionStat records each hot region's statistics

func NewRegionStat 

func NewRegionStat(region *core.RegionInfo, flowBytes uint64, antiCount int) *RegionStat

NewRegionStat returns a RegionStat.

type RegionStatisticType 

type RegionStatisticType uint32

RegionStatisticType represents the type of the region's status. 

const (
	MissPeer RegionStatisticType = 1 << iota
	ExtraPeer
	DownPeer
	PendingPeer
	OfflinePeer
	IncorrectNamespace
	LearnerPeer
	EmptyRegion
)

region status type

type RegionStatistics 

type RegionStatistics struct {
	// contains filtered or unexported fields
}

RegionStatistics is used to record the status of regions.

func NewRegionStatistics 

func NewRegionStatistics(opt ScheduleOptions, classifier namespace.Classifier) *RegionStatistics

NewRegionStatistics creates a new RegionStatistics.

func (*RegionStatistics) ClearDefunctRegion 

func (r *RegionStatistics) ClearDefunctRegion(regionID uint64)

ClearDefunctRegion is used to handle the overlap region.

func (*RegionStatistics) Collect 

func (r *RegionStatistics) Collect()

Collect collects the metrics of the regions' status.

func (*RegionStatistics) GetRegionStatsByType 

func (r *RegionStatistics) GetRegionStatsByType(typ RegionStatisticType) []*core.RegionInfo

GetRegionStatsByType gets the status of the region by types.

func (*RegionStatistics) Observe 

func (r *RegionStatistics) Observe(region *core.RegionInfo, stores []*core.StoreInfo)

Observe records the current regions' status.

func (*RegionStatistics) Reset 

func (r *RegionStatistics) Reset()

Reset resets the metrics of the regions' status.

type RegionStats 

type RegionStats struct {
	Count            int              `json:"count"`
	EmptyCount       int              `json:"empty_count"`
	StorageSize      int64            `json:"storage_size"`
	StorageKeys      int64            `json:"storage_keys"`
	StoreLeaderCount map[uint64]int   `json:"store_leader_count"`
	StorePeerCount   map[uint64]int   `json:"store_peer_count"`
	StoreLeaderSize  map[uint64]int64 `json:"store_leader_size"`
	StoreLeaderKeys  map[uint64]int64 `json:"store_leader_keys"`
	StorePeerSize    map[uint64]int64 `json:"store_peer_size"`
	StorePeerKeys    map[uint64]int64 `json:"store_peer_keys"`
}

RegionStats records a list of regions' statistics and distribution status.

func GetRegionStats 

func GetRegionStats(r *core.RegionsInfo, startKey, endKey []byte) *RegionStats

GetRegionStats scans regions that intersect range [startKey, endKey) and sums up their statistics.

func (*RegionStats) Observe 

func (s *RegionStats) Observe(r *core.RegionInfo)

Observe adds a region's statistics into RegionStats.

type RegionsStat 

type RegionsStat []RegionStat

RegionsStat is a list of a group region state type

func (RegionsStat) Len 

func (m RegionsStat) Len() int

func (RegionsStat) Less 

func (m RegionsStat) Less(i, j int) bool

func (RegionsStat) Swap 

func (m RegionsStat) Swap(i, j int)

type RollingStats 

type RollingStats struct {
	// contains filtered or unexported fields
}

RollingStats provides rolling statistics with specified window size. There are window size records for calculating.

func NewRollingStats 

func NewRollingStats(size int) *RollingStats

NewRollingStats returns a RollingStats.

func (*RollingStats) Add 

func (r *RollingStats) Add(n float64)

Add adds an element.

func (*RollingStats) Median 

func (r *RollingStats) Median() float64

Median returns the median of the records. it can be used to filter noise. References: https://en.wikipedia.org/wiki/Median_filter.

type RollingStoreStats 

type RollingStoreStats struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

RollingStoreStats are multiple sets of recent historical records with specified windows size.

func (*RollingStoreStats) GetBytesRate 

func (r *RollingStoreStats) GetBytesRate() (writeRate float64, readRate float64)

GetBytesRate returns the bytes write rate and the bytes read rate.

func (*RollingStoreStats) GetKeysReadRate 

func (r *RollingStoreStats) GetKeysReadRate() float64

GetKeysReadRate returns the keys read rate.

func (*RollingStoreStats) GetKeysWriteRate 

func (r *RollingStoreStats) GetKeysWriteRate() float64

GetKeysWriteRate returns the keys write rate.

func (*RollingStoreStats) Observe 

func (r *RollingStoreStats) Observe(stats *pdpb.StoreStats)

Observe records current statistics.

type ScheduleOptions 

type ScheduleOptions interface {
	GetLocationLabels() []string

	GetLowSpaceRatio() float64
	GetHighSpaceRatio() float64
	GetTolerantSizeRatio() float64
	GetStoreBalanceRate() float64

	GetSchedulerMaxWaitingOperator() uint64
	GetLeaderScheduleLimit(name string) uint64
	GetRegionScheduleLimit(name string) uint64
	GetReplicaScheduleLimit(name string) uint64
	GetMergeScheduleLimit(name string) uint64
	GetHotRegionScheduleLimit(name string) uint64
	GetMaxReplicas(name string) int
	GetHotRegionCacheHitsThreshold() int
	GetMaxSnapshotCount() uint64
	GetMaxPendingPeerCount() uint64
	GetMaxMergeRegionSize() uint64
	GetMaxMergeRegionKeys() uint64
	GetLeaderSchedulePolicy() core.SchedulePolicy

	IsRaftLearnerEnabled() bool
	IsMakeUpReplicaEnabled() bool
	IsRemoveExtraReplicaEnabled() bool
	IsRemoveDownReplicaEnabled() bool
	IsReplaceOfflineReplicaEnabled() bool

	GetMaxStoreDownTime() time.Duration
}

ScheduleOptions is an interface to access configurations. TODO: merge the Options to schedule.Options

type StoreHotRegionInfos 

type StoreHotRegionInfos struct {
	AsPeer   StoreHotRegionsStat `json:"as_peer"`
	AsLeader StoreHotRegionsStat `json:"as_leader"`
}

StoreHotRegionInfos : used to get human readable description for hot regions.

type StoreHotRegionsStat 

type StoreHotRegionsStat map[uint64]*HotRegionsStat

StoreHotRegionsStat used to record the hot region statistics group by store

type StoresStats 

type StoresStats struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

StoresStats is a cache hold hot regions.

func NewStoresStats 

func NewStoresStats() *StoresStats

NewStoresStats creates a new hot spot cache.

func (*StoresStats) CreateRollingStoreStats 

func (s *StoresStats) CreateRollingStoreStats(storeID uint64)

CreateRollingStoreStats creates RollingStoreStats with a given store ID.

func (*StoresStats) GetRollingStoreStats 

func (s *StoresStats) GetRollingStoreStats(storeID uint64) *RollingStoreStats

GetRollingStoreStats gets RollingStoreStats with a given store ID.

func (*StoresStats) GetStoresBytesReadStat 

func (s *StoresStats) GetStoresBytesReadStat() map[uint64]uint64

GetStoresBytesReadStat returns the bytes read stat of all StoreInfo.

func (*StoresStats) GetStoresBytesWriteStat 

func (s *StoresStats) GetStoresBytesWriteStat() map[uint64]uint64

GetStoresBytesWriteStat returns the bytes write stat of all StoreInfo.

func (*StoresStats) GetStoresKeysReadStat 

func (s *StoresStats) GetStoresKeysReadStat() map[uint64]uint64

GetStoresKeysReadStat returns the bytes read stat of all StoreInfo.

func (*StoresStats) GetStoresKeysWriteStat 

func (s *StoresStats) GetStoresKeysWriteStat() map[uint64]uint64

GetStoresKeysWriteStat returns the keys write stat of all StoreInfo.

func (*StoresStats) Observe 

func (s *StoresStats) Observe(storeID uint64, stats *pdpb.StoreStats)

Observe records the current store status with a given store.

func (*StoresStats) RemoveRollingStoreStats 

func (s *StoresStats) RemoveRollingStoreStats(storeID uint64)

RemoveRollingStoreStats removes RollingStoreStats with a given store ID.

func (*StoresStats) TotalBytesReadRate 

func (s *StoresStats) TotalBytesReadRate() float64

TotalBytesReadRate returns the total read bytes rate of all StoreInfo.

func (*StoresStats) TotalBytesWriteRate 

func (s *StoresStats) TotalBytesWriteRate() float64

TotalBytesWriteRate returns the total written bytes rate of all StoreInfo.

func (*StoresStats) UpdateTotalBytesRate 

func (s *StoresStats) UpdateTotalBytesRate(stores *core.StoresInfo)

UpdateTotalBytesRate updates the total bytes write rate and read rate.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.