workloadlearning

Documentation

Overview

Package workloadlearning implements the Workload-Based Learning Optimizer. The Workload-Based Learning Optimizer introduces a new module in TiDB that leverages captured workload history to enhance the database query optimizer. By learning from historical data, this module helps the optimizer make smarter decisions, such as identify hot and cold tables, analyze resource consumption, etc. The workload analysis results can be used to directly suggest a better path, or to indirectly influence the cost model and stats so that the optimizer can select the best plan more intelligently and adaptively.

Index

• type Handle
  • func NewWorkloadLearningHandle(pool util.SessionPool) *Handle
  • func (handle *Handle) HandleReadTableCost(infoSchema infoschema.InfoSchema)
  • func (handle *Handle) SaveReadTableCostMetrics(metrics map[ast.CIStr]*ReadTableCostMetrics, startTime, endTime time.Time, ...)
• type ReadTableCostMetrics

Types

type Handle

type Handle struct {
	// contains filtered or unexported fields
}

Handle The entry point for all workload-based learning related tasks

func NewWorkloadLearningHandle

func NewWorkloadLearningHandle(pool util.SessionPool) *Handle

NewWorkloadLearningHandle Create a new WorkloadLearningHandle WorkloadLearningHandle is Singleton pattern

func (*Handle) HandleReadTableCost

func (handle *Handle) HandleReadTableCost(infoSchema infoschema.InfoSchema)

HandleReadTableCost Start a new round of analysis of all historical read queries. According to abstracted table cost metrics, calculate the percentage of read scan time and memory usage for each table. The result will be saved to the table "mysql.tidb_workload_values". Dataflow

1. Abstract middle table cost metrics(scan time, memory usage, read frequency) from every record in statement_summary/statement_stats

2,3. Group by tablename, get the total scan time, total memory usage, and every table scan time, memory usage,

read frequency

4. Calculate table cost for each table, table cost = table scan time / total scan time + table mem usage / total mem usage 5. Save all table cost metrics[per table](scan time, table cost, etc) to table "mysql.tidb_workload_values"

func (*Handle) SaveReadTableCostMetrics

func (handle *Handle) SaveReadTableCostMetrics(metrics map[ast.CIStr]*ReadTableCostMetrics,
	startTime, endTime time.Time, infoSchema infoschema.InfoSchema)

SaveReadTableCostMetrics table cost metrics, workload-based start and end time, version,

type ReadTableCostMetrics

type ReadTableCostMetrics struct {
	DbName    ast.CIStr
	TableName ast.CIStr
	// TableScanTime[t] = sum(scan-time * readFrequency) of all records in statement_summary where table-name = t
	TableScanTime float64
	// TableMemUsage[t] = sum(mem-usage * readFrequency) of all records in statement_summary where table-name = t
	TableMemUsage float64
	// ReadFrequency[t] = sum(read-frequency) of all records in statement_summary where table-name = t
	ReadFrequency int64
	// TableCost[t] = TableScanTime[t] / totalScanTime  + TableMemUsage[t] / totalMemUsage
	// range between 0 ~ 2
	TableCost float64
}

ReadTableCostMetrics is used to indicate the intermediate status and results analyzed through read workload for function "HandleReadTableCost".