aggregation

package
v1.1.0-beta.0...-33f0727 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 23, 2024 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AggFuncToPBExpr 

func AggFuncToPBExpr(ctx expression.PushDownContext, aggFunc *AggFuncDesc, storeType kv.StoreType) (*tipb.Expr, error)

AggFuncToPBExpr converts aggregate function to pb.

func AggFunctionModeToPB 

func AggFunctionModeToPB(mode AggFunctionMode) (pbMode *tipb.AggFunctionMode)

AggFunctionModeToPB converts aggregate function mode to PB.

func CheckAggPushDown 

func CheckAggPushDown(ctx expression.EvalContext, aggFunc *AggFuncDesc, storeType kv.StoreType) bool

CheckAggPushDown checks whether an agg function can be pushed to storage.

func CheckAggPushFlash 

func CheckAggPushFlash(ctx expression.EvalContext, aggFunc *AggFuncDesc) bool

CheckAggPushFlash checks whether an agg function can be pushed to flash storage.

func ExplainAggFunc 

func ExplainAggFunc(ctx expression.EvalContext, agg *AggFuncDesc, normalized bool) string

ExplainAggFunc generates explain information for a aggregation function.

func IsAllFirstRow 

func IsAllFirstRow(aggFuncs []*AggFuncDesc) bool

IsAllFirstRow checks whether functions in `aggFuncs` are all FirstRow.

func NeedCount 

func NeedCount(name string) bool

NeedCount indicates whether the aggregate function should record count.

func NeedFrame 

func NeedFrame(name string) bool

NeedFrame checks if the function need frame specification.

func NeedValue 

func NeedValue(name string) bool

NeedValue indicates whether the aggregate function should record value.

func NewDistAggFunc 

func NewDistAggFunc(expr *tipb.Expr, fieldTps []*types.FieldType, ctx expression.BuildContext) (Aggregation, *AggFuncDesc, error)

NewDistAggFunc creates new Aggregate function for mock tikv.

func UseDefaultFrame 

func UseDefaultFrame(name string) (bool, ast.FrameClause)

UseDefaultFrame indicates if the window function has a provided frame that will override user's designation

func WindowFuncToPBExpr 

func WindowFuncToPBExpr(sctx expression.EvalContext, client kv.Client, desc *WindowFuncDesc) *tipb.Expr

WindowFuncToPBExpr converts aggregate function to pb.

Types

type AggEvaluateContext 

type AggEvaluateContext struct {
	Ctx             expression.EvalContext
	DistinctChecker *distinctChecker
	Count           int64
	Value           types.Datum
	Buffer          *bytes.Buffer // Buffer is used for group_concat.
	GotFirstRow     bool          // It will check if the agg has met the first row key.
}

AggEvaluateContext is used to store intermediate result when calculating aggregate functions.

type AggFuncDesc 

type AggFuncDesc struct {

	// Mode represents the execution mode of the aggregation function.
	Mode AggFunctionMode
	// HasDistinct represents whether the aggregation function contains distinct attribute.
	HasDistinct bool
	// OrderByItems represents the order by clause used in GROUP_CONCAT
	OrderByItems []*util.ByItems
	// GroupingID is used for distinguishing with not-set 0, starting from 1.
	GroupingID int
	// contains filtered or unexported fields
}

AggFuncDesc describes an aggregation function signature, only used in planner.

func NewAggFuncDesc 

func NewAggFuncDesc(ctx expression.BuildContext, name string, args []expression.Expression, hasDistinct bool) (*AggFuncDesc, error)

NewAggFuncDesc creates an aggregation function signature descriptor. this func cannot be called twice as the TypeInfer has changed the type of args in the first time.

func NewAggFuncDescForWindowFunc 

func NewAggFuncDescForWindowFunc(ctx expression.BuildContext, desc *WindowFuncDesc, hasDistinct bool) (*AggFuncDesc, error)

NewAggFuncDescForWindowFunc creates an aggregation function from window functions, where baseFuncDesc may be ready.

func PBExprToAggFuncDesc 

func PBExprToAggFuncDesc(ctx expression.BuildContext, aggFunc *tipb.Expr, fieldTps []*types.FieldType) (*AggFuncDesc, error)

PBExprToAggFuncDesc converts pb to aggregate function.

func (*AggFuncDesc) Clone 

func (a *AggFuncDesc) Clone() *AggFuncDesc

Clone copies an aggregation function signature totally.

func (*AggFuncDesc) Equal 

func (a *AggFuncDesc) Equal(ctx expression.EvalContext, other *AggFuncDesc) bool

Equal checks whether two aggregation function signatures are equal.

func (*AggFuncDesc) Equals 

func (a *AggFuncDesc) Equals(other any) bool

Equals checks whether two aggregation function signatures are equal.

func (*AggFuncDesc) EvalNullValueInOuterJoin 

func (a *AggFuncDesc) EvalNullValueInOuterJoin(ctx expression.BuildContext, schema *expression.Schema) (types.Datum, bool, error)

EvalNullValueInOuterJoin gets the null value when the aggregation is upon an outer join, and the aggregation function's input is null. If there is no matching row for the inner table of an outer join, an aggregation function only involves constant and/or columns belongs to the inner table will be set to the null value. The input stands for the schema of Aggregation's child. If the function can't produce a null value, the second return value will be false. e.g. Table t with only one row: +-------+---------+---------+ | Table | Field | Type | +-------+---------+---------+ | t | a | int(11) | +-------+---------+---------+ +------+ | a | +------+ | 1 | +------+

Table s which is empty: +-------+---------+---------+ | Table | Field | Type | +-------+---------+---------+ | s | a | int(11) | +-------+---------+---------+

Query: `select t.a as `t.a`, count(95), sum(95), avg(95), bit_or(95), bit_and(95), bit_or(95), max(95), min(95), s.a as `s.a`, avg(95) from t left join s on t.a = s.a;` +------+-----------+---------+---------+------------+-------------+------------+---------+---------+------+----------+ | t.a | count(95) | sum(95) | avg(95) | bit_or(95) | bit_and(95) | bit_or(95) | max(95) | min(95) | s.a | avg(s.a) | +------+-----------+---------+---------+------------+-------------+------------+---------+---------+------+----------+ | 1 | 1 | 95 | 95.0000 | 95 | 95 | 95 | 95 | 95 | NULL | NULL | +------+-----------+---------+---------+------------+-------------+------------+---------+---------+------+----------+

func (*AggFuncDesc) GetAggFunc 

func (a *AggFuncDesc) GetAggFunc(ctx expression.AggFuncBuildContext) Aggregation

GetAggFunc gets an evaluator according to the aggregation function signature.

func (*AggFuncDesc) GetDefaultValue 

func (a *AggFuncDesc) GetDefaultValue() (v types.Datum)

GetDefaultValue gets the default value when the function's input is null. According to MySQL, default values of the function are listed as follows: e.g. Table t which is empty: +-------+---------+---------+ | Table | Field | Type | +-------+---------+---------+ | t | a | int(11) | +-------+---------+---------+

Query: `select avg(a), sum(a), count(a), bit_xor(a), bit_or(a), bit_and(a), max(a), min(a), group_concat(a), approx_count_distinct(a), approx_percentile(a, 50) from test.t;` +--------+--------+----------+------------+-----------+----------------------+--------+--------+-----------------+--------------------------+--------------------------+ | avg(a) | sum(a) | count(a) | bit_xor(a) | bit_or(a) | bit_and(a) | max(a) | min(a) | group_concat(a) | approx_count_distinct(a) | approx_percentile(a, 50) | +--------+--------+----------+------------+-----------+----------------------+--------+--------+-----------------+--------------------------+--------------------------+ | NULL | NULL | 0 | 0 | 0 | 18446744073709551615 | NULL | NULL | NULL | 0 | NULL | +--------+--------+----------+------------+-----------+----------------------+--------+--------+-----------------+--------------------------+--------------------------+

func (*AggFuncDesc) GetTiPBExpr 

func (desc *AggFuncDesc) GetTiPBExpr(tryWindowDesc bool) (tp tipb.ExprType)

GetTiPBExpr return the TiPB ExprType of desc.

func (*AggFuncDesc) Hash64 

func (a *AggFuncDesc) Hash64(h base.Hasher)

Hash64 returns the hash64 for the aggregation function signature.

func (*AggFuncDesc) MemoryUsage 

func (a *AggFuncDesc) MemoryUsage() (sum int64)

MemoryUsage the memory usage of AggFuncDesc

func (*AggFuncDesc) Split 

func (a *AggFuncDesc) Split(ordinal []int) (partialAggDesc, finalAggDesc *AggFuncDesc)

Split splits `a` into two aggregate descriptors for partial phase and final phase individually. This function is only used when executing aggregate function parallelly. ordinal indicates the column ordinal of the intermediate result.

func (*AggFuncDesc) StringWithCtx 

func (a *AggFuncDesc) StringWithCtx(ctx expression.ParamValues, redact string) string

StringWithCtx returns the string representation within given ctx.

func (*AggFuncDesc) TypeInfer 

func (a *AggFuncDesc) TypeInfer(ctx expression.BuildContext) error

TypeInfer infers the arguments and return types of an function.

func (*AggFuncDesc) TypeInfer4AvgSum 

func (a *AggFuncDesc) TypeInfer4AvgSum(avgRetType *types.FieldType)

TypeInfer4AvgSum infers the type of sum from avg, which should extend the precision of decimal compatible with mysql.

func (*AggFuncDesc) TypeInfer4FinalCount 

func (a *AggFuncDesc) TypeInfer4FinalCount(finalCountRetType *types.FieldType)

TypeInfer4FinalCount infers the type of sum agg which is rewritten from final count agg run on MPP mode.

func (*AggFuncDesc) UpdateNotNullFlag4RetType 

func (a *AggFuncDesc) UpdateNotNullFlag4RetType(hasGroupBy, allAggsFirstRow bool) error

UpdateNotNullFlag4RetType checks if we should remove the NotNull flag for the return type of the agg.

func (*AggFuncDesc) WrapCastForAggArgs 

func (a *AggFuncDesc) WrapCastForAggArgs(ctx expression.BuildContext)

WrapCastForAggArgs wraps the args of an aggregate function with a cast function.

type AggFunctionMode 

type AggFunctionMode int

AggFunctionMode stands for the aggregation function's mode. 

const (
	CompleteMode AggFunctionMode = iota
	FinalMode
	Partial1Mode
	Partial2Mode
	DedupMode
)

|-----------------|--------------|--------------| | AggFunctionMode | input | output | |-----------------|--------------|--------------| | CompleteMode | origin data | final result | | FinalMode | partial data | final result | | Partial1Mode | origin data | partial data | | Partial2Mode | partial data | partial data | | DedupMode | origin data | origin data | |-----------------|--------------|--------------|

func PBAggFuncModeToAggFuncMode 

func PBAggFuncModeToAggFuncMode(pbMode *tipb.AggFunctionMode) (mode AggFunctionMode)

PBAggFuncModeToAggFuncMode converts pb to aggregate function mode.

func (AggFunctionMode) ToString 

func (a AggFunctionMode) ToString() string

ToString show the agg mode.

type Aggregation 

type Aggregation interface {
	// Update during executing.
	Update(evalCtx *AggEvaluateContext, sc *stmtctx.StatementContext, row chunk.Row) error

	// GetPartialResult will called by coprocessor to get partial results. For avg function, partial results will return
	// sum and count values at the same time.
	GetPartialResult(evalCtx *AggEvaluateContext) []types.Datum

	// GetResult will be called when all data have been processed.
	GetResult(evalCtx *AggEvaluateContext) types.Datum

	// CreateContext creates a new AggEvaluateContext for the aggregation function.
	CreateContext(ctx expression.EvalContext) *AggEvaluateContext

	// ResetContext resets the content of the evaluate context.
	ResetContext(ctx expression.EvalContext, evalCtx *AggEvaluateContext)
}

Aggregation stands for aggregate functions.

type WindowFuncDesc 

type WindowFuncDesc struct {
	// contains filtered or unexported fields
}

WindowFuncDesc describes a window function signature, only used in planner.

func NewWindowFuncDesc 

func NewWindowFuncDesc(ctx expression.BuildContext, name string, args []expression.Expression, skipCheckArgs bool) (*WindowFuncDesc, error)

NewWindowFuncDesc creates a window function signature descriptor.

func (*WindowFuncDesc) CanPushDownToTiFlash 

func (s *WindowFuncDesc) CanPushDownToTiFlash(ctx expression.PushDownContext) bool

CanPushDownToTiFlash control whether a window function desc can be push down to tiflash.

func (*WindowFuncDesc) Clone 

func (s *WindowFuncDesc) Clone() *WindowFuncDesc

Clone makes a copy of SortItem.

func (*WindowFuncDesc) Equals 

func (a *WindowFuncDesc) Equals(other any) bool

Equals implements the base.Equals interface.

func (*WindowFuncDesc) GetDefaultValue 

func (a *WindowFuncDesc) GetDefaultValue() (v types.Datum)

GetDefaultValue gets the default value when the function's input is null. According to MySQL, default values of the function are listed as follows: e.g. Table t which is empty: +-------+---------+---------+ | Table | Field | Type | +-------+---------+---------+ | t | a | int(11) | +-------+---------+---------+

Query: `select avg(a), sum(a), count(a), bit_xor(a), bit_or(a), bit_and(a), max(a), min(a), group_concat(a), approx_count_distinct(a), approx_percentile(a, 50) from test.t;` +--------+--------+----------+------------+-----------+----------------------+--------+--------+-----------------+--------------------------+--------------------------+ | avg(a) | sum(a) | count(a) | bit_xor(a) | bit_or(a) | bit_and(a) | max(a) | min(a) | group_concat(a) | approx_count_distinct(a) | approx_percentile(a, 50) | +--------+--------+----------+------------+-----------+----------------------+--------+--------+-----------------+--------------------------+--------------------------+ | NULL | NULL | 0 | 0 | 0 | 18446744073709551615 | NULL | NULL | NULL | 0 | NULL | +--------+--------+----------+------------+-----------+----------------------+--------+--------+-----------------+--------------------------+--------------------------+

func (*WindowFuncDesc) GetTiPBExpr 

func (desc *WindowFuncDesc) GetTiPBExpr(tryWindowDesc bool) (tp tipb.ExprType)

GetTiPBExpr return the TiPB ExprType of desc.

func (*WindowFuncDesc) Hash64 

func (a *WindowFuncDesc) Hash64(h base.Hasher)

Hash64 implements the base.Hasher interface.

func (*WindowFuncDesc) MemoryUsage 

func (a *WindowFuncDesc) MemoryUsage() (sum int64)

MemoryUsage return the memory usage of baseFuncDesc

func (*WindowFuncDesc) StringWithCtx 

func (a *WindowFuncDesc) StringWithCtx(ctx expression.ParamValues, redact string) string

StringWithCtx returns the string within given context.

func (*WindowFuncDesc) TypeInfer 

func (a *WindowFuncDesc) TypeInfer(ctx expression.BuildContext) error

TypeInfer infers the arguments and return types of an function.

func (*WindowFuncDesc) TypeInfer4AvgSum 

func (a *WindowFuncDesc) TypeInfer4AvgSum(avgRetType *types.FieldType)

TypeInfer4AvgSum infers the type of sum from avg, which should extend the precision of decimal compatible with mysql.

func (*WindowFuncDesc) TypeInfer4FinalCount 

func (a *WindowFuncDesc) TypeInfer4FinalCount(finalCountRetType *types.FieldType)

TypeInfer4FinalCount infers the type of sum agg which is rewritten from final count agg run on MPP mode.

func (*WindowFuncDesc) WrapCastForAggArgs 

func (a *WindowFuncDesc) WrapCastForAggArgs(ctx expression.BuildContext)

WrapCastForAggArgs wraps the args of an aggregate function with a cast function.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.