statistics

package
v3.0.12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 12, 2020 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// RegionHeartBeatReportInterval is the heartbeat report interval of a region.
	RegionHeartBeatReportInterval = 60
	// StoreHeartBeatReportInterval is the heartbeat report interval of a store.
	StoreHeartBeatReportInterval = 10
)

Variables

View Source
var Simulating bool

Simulating is an option to overpass the impact of accelerated time. Should only turned on by the simulator.

Functions

func NewStoreStatisticsMap

func NewStoreStatisticsMap(opt ScheduleOptions, classifier namespace.Classifier) *storeStatisticsMap

NewStoreStatisticsMap creates a new storeStatisticsMap.

Types

type FlowKind

type FlowKind uint32

FlowKind is a identify Flow types.

const (
	WriteFlow FlowKind = iota
	ReadFlow
)

Flags for flow.

type HotRegionsStat

type HotRegionsStat struct {
	TotalFlowBytes uint64      `json:"total_flow_bytes"`
	RegionsCount   int         `json:"regions_count"`
	RegionsStat    RegionsStat `json:"statistics"`
}

HotRegionsStat records all hot regions statistics

type HotSpotCache

type HotSpotCache struct {
	// contains filtered or unexported fields
}

HotSpotCache is a cache hold hot regions.

func NewHotSpotCache

func NewHotSpotCache() *HotSpotCache

NewHotSpotCache creates a new hot spot cache.

func (*HotSpotCache) CheckRead

func (w *HotSpotCache) CheckRead(region *core.RegionInfo, stats *StoresStats) (bool, *RegionStat)

CheckRead checks the read status, returns whether need update statistics and item.

func (*HotSpotCache) CheckWrite

func (w *HotSpotCache) CheckWrite(region *core.RegionInfo, stats *StoresStats) (bool, *RegionStat)

CheckWrite checks the write status, returns whether need update statistics and item.

func (*HotSpotCache) CollectMetrics

func (w *HotSpotCache) CollectMetrics(stats *StoresStats)

CollectMetrics collect the hot cache metrics

func (*HotSpotCache) IsRegionHot

func (w *HotSpotCache) IsRegionHot(id uint64, hotThreshold int) bool

IsRegionHot checks if the region is hot.

func (*HotSpotCache) RandHotRegionFromStore

func (w *HotSpotCache) RandHotRegionFromStore(storeID uint64, kind FlowKind, hotThreshold int) *RegionStat

RandHotRegionFromStore random picks a hot region in specify store.

func (*HotSpotCache) RegionStats

func (w *HotSpotCache) RegionStats(kind FlowKind) []*RegionStat

RegionStats returns hot items according to kind

func (*HotSpotCache) ResetMetrics

func (w *HotSpotCache) ResetMetrics()

ResetMetrics resets the hot cache metrics.

func (*HotSpotCache) Update

func (w *HotSpotCache) Update(key uint64, item *RegionStat, kind FlowKind)

Update updates the cache.

type LabelLevelStatistics

type LabelLevelStatistics struct {
	// contains filtered or unexported fields
}

LabelLevelStatistics is the statistics of the level of labels.

func NewLabelLevelStatistics

func NewLabelLevelStatistics() *LabelLevelStatistics

NewLabelLevelStatistics creates a new LabelLevelStatistics.

func (*LabelLevelStatistics) ClearDefunctRegion

func (l *LabelLevelStatistics) ClearDefunctRegion(regionID uint64, labels []string)

ClearDefunctRegion is used to handle the overlap region.

func (*LabelLevelStatistics) Collect

func (l *LabelLevelStatistics) Collect()

Collect collects the metrics of the label status.

func (*LabelLevelStatistics) Observe

func (l *LabelLevelStatistics) Observe(region *core.RegionInfo, stores []*core.StoreInfo, labels []string)

Observe records the current label status.

func (*LabelLevelStatistics) Reset

func (l *LabelLevelStatistics) Reset()

Reset resets the metrics of the label status.

type RegionStat

type RegionStat struct {
	RegionID  uint64 `json:"region_id"`
	FlowBytes uint64 `json:"flow_bytes"`
	// HotDegree records the hot region update times
	HotDegree int `json:"hot_degree"`
	// LastUpdateTime used to calculate average write
	LastUpdateTime time.Time `json:"last_update_time"`
	StoreID        uint64    `json:"-"`
	// AntiCount used to eliminate some noise when remove region in cache
	AntiCount int
	// Version used to check the region split times
	Version uint64
	// Stats is a rolling statistics, recording some recently added records.
	Stats *RollingStats
}

RegionStat records each hot region's statistics

func NewRegionStat

func NewRegionStat(region *core.RegionInfo, flowBytes uint64, antiCount int) *RegionStat

NewRegionStat returns a RegionStat.

type RegionStatisticType

type RegionStatisticType uint32

RegionStatisticType represents the type of the region's status.

const (
	MissPeer RegionStatisticType = 1 << iota
	ExtraPeer
	DownPeer
	PendingPeer
	OfflinePeer
	IncorrectNamespace
	LearnerPeer
	EmptyRegion
)

region status type

type RegionStatistics

type RegionStatistics struct {
	// contains filtered or unexported fields
}

RegionStatistics is used to record the status of regions.

func NewRegionStatistics

func NewRegionStatistics(opt ScheduleOptions, classifier namespace.Classifier) *RegionStatistics

NewRegionStatistics creates a new RegionStatistics.

func (*RegionStatistics) ClearDefunctRegion

func (r *RegionStatistics) ClearDefunctRegion(regionID uint64)

ClearDefunctRegion is used to handle the overlap region.

func (*RegionStatistics) Collect

func (r *RegionStatistics) Collect()

Collect collects the metrics of the regions' status.

func (*RegionStatistics) GetRegionStatsByType

func (r *RegionStatistics) GetRegionStatsByType(typ RegionStatisticType) []*core.RegionInfo

GetRegionStatsByType gets the status of the region by types.

func (*RegionStatistics) Observe

func (r *RegionStatistics) Observe(region *core.RegionInfo, stores []*core.StoreInfo)

Observe records the current regions' status.

func (*RegionStatistics) Reset

func (r *RegionStatistics) Reset()

Reset resets the metrics of the regions' status.

type RegionStats

type RegionStats struct {
	Count            int              `json:"count"`
	EmptyCount       int              `json:"empty_count"`
	StorageSize      int64            `json:"storage_size"`
	StorageKeys      int64            `json:"storage_keys"`
	StoreLeaderCount map[uint64]int   `json:"store_leader_count"`
	StorePeerCount   map[uint64]int   `json:"store_peer_count"`
	StoreLeaderSize  map[uint64]int64 `json:"store_leader_size"`
	StoreLeaderKeys  map[uint64]int64 `json:"store_leader_keys"`
	StorePeerSize    map[uint64]int64 `json:"store_peer_size"`
	StorePeerKeys    map[uint64]int64 `json:"store_peer_keys"`
}

RegionStats records a list of regions' statistics and distribution status.

func GetRegionStats

func GetRegionStats(r *core.RegionsInfo, startKey, endKey []byte) *RegionStats

GetRegionStats scans regions that intersect range [startKey, endKey) and sums up their statistics.

func (*RegionStats) Observe

func (s *RegionStats) Observe(r *core.RegionInfo)

Observe adds a region's statistics into RegionStats.

type RegionsStat

type RegionsStat []RegionStat

RegionsStat is a list of a group region state type

func (RegionsStat) Len

func (m RegionsStat) Len() int

func (RegionsStat) Less

func (m RegionsStat) Less(i, j int) bool

func (RegionsStat) Swap

func (m RegionsStat) Swap(i, j int)

type RollingStats

type RollingStats struct {
	// contains filtered or unexported fields
}

RollingStats provides rolling statistics with specified window size. There are window size records for calculating.

func NewRollingStats

func NewRollingStats(size int) *RollingStats

NewRollingStats returns a RollingStats.

func (*RollingStats) Add

func (r *RollingStats) Add(n float64)

Add adds an element.

func (*RollingStats) Median

func (r *RollingStats) Median() float64

Median returns the median of the records. it can be used to filter noise. References: https://en.wikipedia.org/wiki/Median_filter.

type RollingStoreStats

type RollingStoreStats struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

RollingStoreStats are multiple sets of recent historical records with specified windows size.

func (*RollingStoreStats) GetBytesRate

func (r *RollingStoreStats) GetBytesRate() (writeRate float64, readRate float64)

GetBytesRate returns the bytes write rate and the bytes read rate.

func (*RollingStoreStats) GetKeysReadRate

func (r *RollingStoreStats) GetKeysReadRate() float64

GetKeysReadRate returns the keys read rate.

func (*RollingStoreStats) GetKeysWriteRate

func (r *RollingStoreStats) GetKeysWriteRate() float64

GetKeysWriteRate returns the keys write rate.

func (*RollingStoreStats) Observe

func (r *RollingStoreStats) Observe(stats *pdpb.StoreStats)

Observe records current statistics.

type ScheduleOptions

type ScheduleOptions interface {
	GetLocationLabels() []string

	GetLowSpaceRatio() float64
	GetHighSpaceRatio() float64
	GetTolerantSizeRatio() float64
	GetStoreBalanceRate() float64

	GetSchedulerMaxWaitingOperator() uint64
	GetLeaderScheduleLimit(name string) uint64
	GetRegionScheduleLimit(name string) uint64
	GetReplicaScheduleLimit(name string) uint64
	GetMergeScheduleLimit(name string) uint64
	GetHotRegionScheduleLimit(name string) uint64
	GetMaxReplicas(name string) int
	GetHotRegionCacheHitsThreshold() int
	GetMaxSnapshotCount() uint64
	GetMaxPendingPeerCount() uint64
	GetMaxMergeRegionSize() uint64
	GetMaxMergeRegionKeys() uint64

	IsRaftLearnerEnabled() bool
	IsMakeUpReplicaEnabled() bool
	IsRemoveExtraReplicaEnabled() bool
	IsRemoveDownReplicaEnabled() bool
	IsReplaceOfflineReplicaEnabled() bool

	GetMaxStoreDownTime() time.Duration
}

ScheduleOptions is an interface to access configurations. TODO: merge the Options to schedule.Options

type StoreHotRegionInfos

type StoreHotRegionInfos struct {
	AsPeer   StoreHotRegionsStat `json:"as_peer"`
	AsLeader StoreHotRegionsStat `json:"as_leader"`
}

StoreHotRegionInfos : used to get human readable description for hot regions.

type StoreHotRegionsStat

type StoreHotRegionsStat map[uint64]*HotRegionsStat

StoreHotRegionsStat used to record the hot region statistics group by store

type StoresStats

type StoresStats struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

StoresStats is a cache hold hot regions.

func NewStoresStats

func NewStoresStats() *StoresStats

NewStoresStats creates a new hot spot cache.

func (*StoresStats) CreateRollingStoreStats

func (s *StoresStats) CreateRollingStoreStats(storeID uint64)

CreateRollingStoreStats creates RollingStoreStats with a given store ID.

func (*StoresStats) GetRollingStoreStats

func (s *StoresStats) GetRollingStoreStats(storeID uint64) *RollingStoreStats

GetRollingStoreStats gets RollingStoreStats with a given store ID.

func (*StoresStats) GetStoresBytesReadStat

func (s *StoresStats) GetStoresBytesReadStat() map[uint64]uint64

GetStoresBytesReadStat returns the bytes read stat of all StoreInfo.

func (*StoresStats) GetStoresBytesWriteStat

func (s *StoresStats) GetStoresBytesWriteStat() map[uint64]uint64

GetStoresBytesWriteStat returns the bytes write stat of all StoreInfo.

func (*StoresStats) GetStoresKeysReadStat

func (s *StoresStats) GetStoresKeysReadStat() map[uint64]uint64

GetStoresKeysReadStat returns the bytes read stat of all StoreInfo.

func (*StoresStats) GetStoresKeysWriteStat

func (s *StoresStats) GetStoresKeysWriteStat() map[uint64]uint64

GetStoresKeysWriteStat returns the keys write stat of all StoreInfo.

func (*StoresStats) Observe

func (s *StoresStats) Observe(storeID uint64, stats *pdpb.StoreStats)

Observe records the current store status with a given store.

func (*StoresStats) RemoveRollingStoreStats

func (s *StoresStats) RemoveRollingStoreStats(storeID uint64)

RemoveRollingStoreStats removes RollingStoreStats with a given store ID.

func (*StoresStats) TotalBytesReadRate

func (s *StoresStats) TotalBytesReadRate() float64

TotalBytesReadRate returns the total read bytes rate of all StoreInfo.

func (*StoresStats) TotalBytesWriteRate

func (s *StoresStats) TotalBytesWriteRate() float64

TotalBytesWriteRate returns the total written bytes rate of all StoreInfo.

func (*StoresStats) UpdateTotalBytesRate

func (s *StoresStats) UpdateTotalBytesRate(stores *core.StoresInfo)

UpdateTotalBytesRate updates the total bytes write rate and read rate.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL