ranger

package
v1.1.0-beta.0...-5753106 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 29, 2024 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source 
const EmptyRangeSize = int64(unsafe.Sizeof(Range{}))

EmptyRangeSize is the size of empty range.

Variables

This section is empty.

Functions

func AddExpr4EqAndInCondition 

func AddExpr4EqAndInCondition(sctx *rangerctx.RangerContext, conditions []expression.Expression,
	cols []*expression.Column) ([]expression.Expression, error)

AddExpr4EqAndInCondition add the `tidb_shard(x) = xxx` prefix Add tidb_shard() for EQ and IN function. e.g. input condition is `WHERE a = 1`, output condition is `WHERE tidb_shard(a) = 214 AND a = 1`. e.g. input condition is `WHERE a IN (1, 2 ,3)`, output condition is `WHERE (tidb_shard(a) = 214 AND a = 1) OR (tidb_shard(a) = 143 AND a = 2) OR (tidb_shard(a) = 156 AND a = 3)` @param[in] conditions the original condition to be processed @param[in] cols the columns of shard index, such as [tidb_shard(a), a, ...] @param[in] lengths the length for every column of shard index @retval - the new condition after adding tidb_shard() prefix

func AddGcColumn4EqCond 

func AddGcColumn4EqCond(sctx *rangerctx.RangerContext,
	cols []*expression.Column,
	accessesCond []expression.Expression,
	columnValues []*valueInfo) ([]expression.Expression, error)

AddGcColumn4EqCond add the `tidb_shard(x) = xxx` prefix for equal condition For param explanation, please refer to the function `AddGcColumnCond`. @retval - []expression.Expression the new conditions after adding `tidb_shard() = xxx` prefix 

[]*valueInfo              the values of every columns in the returned new conditions
error                     if error gernerated, return error

func AddGcColumn4InCond 

func AddGcColumn4InCond(sctx *rangerctx.RangerContext,
	cols []*expression.Column,
	accessesCond []expression.Expression) ([]expression.Expression, error)

AddGcColumn4InCond add the `tidb_shard(x) = xxx` for `IN` condition For param explanation, please refer to the function `AddGcColumnCond`. @retval - []expression.Expression the new conditions after adding `tidb_shard() = xxx` prefix 

error                     if error gernerated, return error

func AddGcColumnCond 

func AddGcColumnCond(sctx *rangerctx.RangerContext,
	cols []*expression.Column,
	accessesCond []expression.Expression,
	columnValues []*valueInfo) ([]expression.Expression, error)

AddGcColumnCond add the `tidb_shard(x) = xxx` to the condition @param[in] cols the columns of shard index, such as [tidb_shard(a), a, ...] @param[in] accessCond the conditions relative to the index and arranged by the index column order. 

e.g. the index is uk(tidb_shard(a), a, b) and the where clause is
`WHERE b = 1 AND a = 2 AND c = 3`, the param accessCond is {a = 2, b = 1} that is
only relative to uk's columns.

@param[in] columnValues the values of index columns in param accessCond. if accessCond is {a = 2, b = 1}, 

columnValues is {2, 1}. if accessCond the "IN" function like `a IN (1, 2)`, columnValues
is empty.

@retval - []expression.Expression the new conditions after adding `tidb_shard() = xxx` prefix 

error                     if error gernerated, return error

func AppendConditionsIfNotExist 

func AppendConditionsIfNotExist(ectx expression.EvalContext, conditions, condsToAppend []expression.Expression) []expression.Expression

AppendConditionsIfNotExist appends conditions if they are absent.

func CutDatumByPrefixLen 

func CutDatumByPrefixLen(v *types.Datum, length int, tp *types.FieldType) bool

CutDatumByPrefixLen cuts the datum according to the prefix length. If it's binary or ascii encoded, we will cut it by bytes rather than characters.

func DetachCondsForColumn 

func DetachCondsForColumn(sctx *rangerctx.RangerContext, conds []expression.Expression, col *expression.Column) (accessConditions, otherConditions []expression.Expression)

DetachCondsForColumn detaches access conditions for specified column from other filter conditions.

func ExtractAccessConditionsForColumn 

func ExtractAccessConditionsForColumn(ctx *rangerctx.RangerContext, conds []expression.Expression, col *expression.Column) []expression.Expression

ExtractAccessConditionsForColumn extracts the access conditions used for range calculation. Since we don't need to return the remained filter conditions, it is much simpler than DetachCondsForColumn.

func ExtractColumnsFromExpr 

func ExtractColumnsFromExpr(virtaulExpr *expression.ScalarFunction) []*expression.Column

ExtractColumnsFromExpr get all fields from input expression virtaulExpr

func ExtractEqAndInCondition 

func ExtractEqAndInCondition(sctx *rangerctx.RangerContext, conditions []expression.Expression, cols []*expression.Column,
	lengths []int) ([]expression.Expression, []expression.Expression, []expression.Expression, []*valueInfo, bool)

ExtractEqAndInCondition will split the given condition into three parts by the information of index columns and their lengths. accesses: The condition will be used to build range. filters: filters is the part that some access conditions need to be evaluated again since it's only the prefix part of char column. newConditions: We'll simplify the given conditions if there're multiple in conditions or eq conditions on the same column. 

e.g. if there're a in (1, 2, 3) and a in (2, 3, 4). This two will be combined to a in (2, 3) and pushed to newConditions.

columnValues: the constant column values for all index columns. columnValues[i] is nil if cols[i] is not constant. bool: indicate whether there's nil range when merging eq and in conditions.

func HasFullRange 

func HasFullRange(ranges []*Range, unsignedIntHandle bool) bool

HasFullRange checks if any range in the slice is a full range.

func IsValidShardIndex 

func IsValidShardIndex(cols []*expression.Column) bool

IsValidShardIndex Check whether the definition of shard index is valid. The form of index should like `index(tidb_shard(a), a, ....)`. 1) the column count shoudle be >= 2 2) the first column should be tidb_shard(xxx) 3) the parameter of tidb_shard shoudle be a column that is the second column of index @param[in] cols the columns of shard index, such as [tidb_shard(a), a, ...] @retval - if the shard index is valid return true, otherwise return false

func MergeDNFItems4Col 

func MergeDNFItems4Col(ctx *rangerctx.RangerContext, dnfItems []expression.Expression) []expression.Expression

MergeDNFItems4Col receives a slice of DNF conditions, merges some of them which can be built into ranges on a single column, then returns. For example, [a > 5, b > 6, c > 7, a = 1, b > 3] will become [a > 5 or a = 1, b > 6 or b > 3, c > 7].

func NeedAddColumn4EqCond 

func NeedAddColumn4EqCond(cols []*expression.Column,
	accessCond []expression.Expression, columnValues []*valueInfo) bool

NeedAddColumn4EqCond `tidb_shard(x) = xxx` For param explanation, please refer to the function `NeedAddGcColumn4ShardIndex`. It checks whether EQ conditions need to be added tidb_shard() prefix. (1) columns in accessCond are all columns of the index except the first. (2) every column in accessCond has a constan value

func NeedAddColumn4InCond 

func NeedAddColumn4InCond(cols []*expression.Column, accessCond []expression.Expression, sf *expression.ScalarFunction) bool

NeedAddColumn4InCond `tidb_shard(x) = xxx` For param explanation, please refer to the function `NeedAddGcColumn4ShardIndex`. It checks whether "IN" conditions need to be added tidb_shard() prefix. (1) columns in accessCond are all columns of the index except the first. (2) the first param of "IN" function should be a column not a expression like `a + b` (3) the rest params of "IN" function all should be constant (4) the first param of "IN" function should be the column in the expression of first index field. 

e.g. uk(tidb_shard(a), a). If the conditions is `WHERE b in (1, 2, 3)`, the first param of "IN" function
is `b` that's not the column in `tidb_shard(a)`.

@param sf "IN" function, e.g. `a IN (1, 2, 3)`

func NeedAddGcColumn4ShardIndex 

func NeedAddGcColumn4ShardIndex(cols []*expression.Column, accessCond []expression.Expression, columnValues []*valueInfo) bool

NeedAddGcColumn4ShardIndex check whether to add `tidb_shard(x) = xxx` @param[in] cols the columns of shard index, such as [tidb_shard(a), a, ...] @param[in] accessCond the conditions relative to the index and arranged by the index column order. 

e.g. the index is uk(tidb_shard(a), a, b) and the where clause is
`WHERE b = 1 AND a = 2 AND c = 3`, the param accessCond is {a = 2, b = 1} that is
only relative to uk's columns.

@param[in] columnValues the values of index columns in param accessCond. if accessCond is {a = 2, b = 1}, 

columnValues is {2, 1}. if accessCond the "IN" function like `a IN (1, 2)`, columnValues
is empty.

@retval - return true if it needs to addr tidb_shard() prefix, ohterwise return false

func RangeSingleColToString 

func RangeSingleColToString(sc *stmtctx.StatementContext, lowVal, highVal types.Datum, lowExclude, highExclude bool, colName string, collator collate.Collator) (string, error)

RangeSingleColToString prints a single column of a Range into a string which can appear in an SQL as a condition.

func RangesToString 

func RangesToString(sc *stmtctx.StatementContext, rans Ranges, colNames []string) (string, error)

RangesToString print a list of Ranges into a string which can appear in an SQL as a condition.

func ReachPrefixLen 

func ReachPrefixLen(v *types.Datum, length int, tp *types.FieldType) bool

ReachPrefixLen checks whether the length of v is equal to the prefix length.

Types

type DetachRangeResult 

type DetachRangeResult struct {
	// Ranges is the ranges extracted and built from conditions.
	Ranges Ranges
	// AccessConds is the extracted conditions for access.
	AccessConds []expression.Expression
	// RemainedConds is the filter conditions which should be kept after access.
	RemainedConds []expression.Expression
	// ColumnValues records the constant column values for all index columns.
	// For the ith column, if it is evaluated as constant, ColumnValues[i] is its value. Otherwise ColumnValues[i] is nil.
	ColumnValues []*valueInfo
	// EqCondCount is the number of equal conditions extracted.
	EqCondCount int
	// EqOrInCount is the number of equal/in conditions extracted.
	EqOrInCount int
	// IsDNFCond indicates if the top layer of conditions are in DNF.
	// Please see comments of planner/util/AccessPath.MinAccessCondsForDNFCond for more details.
	IsDNFCond                bool
	MinAccessCondsForDNFCond int
}

DetachRangeResult wraps up results when detaching conditions and builing ranges.

func DetachCondAndBuildRangeForIndex 

func DetachCondAndBuildRangeForIndex(sctx *rangerctx.RangerContext, conditions []expression.Expression, cols []*expression.Column,
	lengths []int, rangeMaxSize int64) (*DetachRangeResult, error)

DetachCondAndBuildRangeForIndex will detach the index filters from table filters. rangeMaxSize is the max memory limit for ranges. O indicates no memory limit. If you ask that all conditions must be used for building ranges, set rangeMemQuota to 0 to avoid range fallback. The returned values are encapsulated into a struct DetachRangeResult, see its comments for explanation.

func DetachCondAndBuildRangeForPartition 

func DetachCondAndBuildRangeForPartition(sctx *rangerctx.RangerContext, conditions []expression.Expression, cols []*expression.Column,
	lengths []int, rangeMaxSize int64) (*DetachRangeResult, error)

DetachCondAndBuildRangeForPartition will detach the index filters from table filters. rangeMaxSize is the max memory limit for ranges. O indicates no memory limit. If you ask that all conditions must be used for building ranges, set rangeMemQuota to 0 to avoid range fallback. The returned values are encapsulated into a struct DetachRangeResult, see its comments for explanation.

type MutableRanges 

type MutableRanges interface {
	// Range returns the underlying range values.
	Range() Ranges
	// Rebuild rebuilds the underlying ranges again.
	Rebuild(sctx planctx.PlanContext) error
	// CloneForPlanCache clones the MutableRanges for plan cache.
	CloneForPlanCache() MutableRanges
}

MutableRanges represents a range may change after it is created. It's mainly designed for plan-cache, since some ranges in a cached plan have to be rebuild when reusing.

type Range 

type Range struct {
	LowVal      []types.Datum // Low value is exclusive.
	HighVal     []types.Datum // High value is exclusive.
	Collators   []collate.Collator
	LowExclude  bool
	HighExclude bool
}

Range represents a range generated in physical plan building phase.

func (*Range) Clone 

func (ran *Range) Clone() *Range

Clone clones a Range.

func (*Range) Encode 

func (ran *Range) Encode(ec errctx.Context, loc *time.Location, lowBuffer, highBuffer []byte) ([]byte, []byte, error)

Encode encodes the range to its encoded value.

func (*Range) IntersectRange 

func (ran *Range) IntersectRange(tc types.Context, otherRange *Range) (*Range, error)

IntersectRange computes intersection between two ranges. err is set of something went wrong during comparison.

func (*Range) IsFullRange 

func (ran *Range) IsFullRange(unsignedIntHandle bool) bool

IsFullRange check if the range is full scan range

func (*Range) IsOnlyNull 

func (ran *Range) IsOnlyNull() bool

IsOnlyNull checks if the range has [NULL, NULL] or [NULL NULL, NULL NULL] range.

func (*Range) IsPoint 

func (ran *Range) IsPoint(sctx *rangerctx.RangerContext) bool

IsPoint returns if the range is a point.

func (*Range) IsPointNonNullable 

func (ran *Range) IsPointNonNullable(tc types.Context) bool

IsPointNonNullable returns if the range is a point without NULL.

func (*Range) IsPointNullable 

func (ran *Range) IsPointNullable(tc types.Context) bool

IsPointNullable returns if the range is a point. TODO: unify the parameter type with IsPointNullable and IsPoint

func (*Range) MemUsage 

func (ran *Range) MemUsage() (sum int64)

MemUsage gets the memory usage of range.

func (*Range) PrefixEqualLen 

func (ran *Range) PrefixEqualLen(tc types.Context) (int, error)

PrefixEqualLen tells you how long the prefix of the range is a point. e.g. If this range is (1 2 3, 1 2 +inf), then the return value is 2.

func (*Range) Redact 

func (ran *Range) Redact(redact string) string

Redact is to print the range with redacting sensitive data.

func (*Range) String 

func (ran *Range) String() string

String implements the Stringer interface. don't use it in the product.

func (*Range) Subset 

func (ran *Range) Subset(tc types.Context, otherRange *Range) bool

Subset for Range type, check if range(ran) is a subset of another range(otherRange). This is done by:

  • Both ran and otherRange have the same collators. This is not needed for the current code path. But, it is used here for future use of the function.
  • Checking if the lower/upper bound of otherRange covers the corresponding lower/upper bound of ran. Thus include checking open/closed inetrvals.

func (*Range) Width 

func (ran *Range) Width() int

Width returns the width of this range.

type RangeType 

type RangeType int

RangeType is alias for int. 

const (
	IntRangeType RangeType = iota
	ColumnRangeType
	IndexRangeType
)

RangeType constants.

type Ranges 

type Ranges []*Range

Ranges implements the MutableRanges interface for range array.

func AppendRanges2PointRanges 

func AppendRanges2PointRanges(pointRanges Ranges, ranges Ranges, rangeMaxSize int64) (Ranges, bool)

AppendRanges2PointRanges appends additional ranges to point ranges. rangeMaxSize is the max memory limit for ranges. O indicates no memory limit. If the second return value is true, it means that the estimated memory after appending additional ranges to point ranges exceeds rangeMaxSize and the function rejects appending additional ranges to point ranges.

func BuildColumnRange 

func BuildColumnRange(conds []expression.Expression, sctx *rangerctx.RangerContext, tp *types.FieldType, colLen int,
	rangeMemQuota int64) (Ranges, []expression.Expression, []expression.Expression, error)

BuildColumnRange builds range from access conditions for general columns. rangeMaxSize is the max memory limit for ranges. O indicates no memory limit. If you ask that all conds must be used for building ranges, set rangeMemQuota to 0 to avoid range fallback. The second return value is the conditions used to build ranges and the third return value is the remained conditions. If you use the function to build ranges for some access path, you need to update the path's access conditions and filter conditions by the second and third return values respectively.

func BuildTableRange 

func BuildTableRange(accessConditions []expression.Expression, sctx *rangerctx.RangerContext, tp *types.FieldType,
	rangeMaxSize int64) (Ranges, []expression.Expression, []expression.Expression, error)

BuildTableRange builds range of PK column for PhysicalTableScan. rangeMaxSize is the max memory limit for ranges. O indicates no memory limit. If you ask that all conds must be used for building ranges, set rangeMemQuota to 0 to avoid range fallback. The second return value is the conditions used to build ranges and the third return value is the remained conditions. If you use the function to build ranges for some access path, you need to update the path's access conditions and filter conditions by the second and third return values respectively.

func DetachSimpleCondAndBuildRangeForIndex 

func DetachSimpleCondAndBuildRangeForIndex(sctx *rangerctx.RangerContext, conditions []expression.Expression,
	cols []*expression.Column, lengths []int, rangeMaxSize int64) (Ranges, []expression.Expression, error)

DetachSimpleCondAndBuildRangeForIndex will detach the index filters from table filters. It will find the point query column firstly and then extract the range query column. rangeMaxSize is the max memory limit for ranges. O indicates no memory limit. If you ask that all conditions must be used for building ranges, set rangeMemQuota to 0 to avoid range fallback.

func FullIntRange 

func FullIntRange(isUnsigned bool) Ranges

FullIntRange is used for table range. Since table range cannot accept MaxValueDatum as the max value. So we need to set it to MaxInt64.

func FullNotNullRange 

func FullNotNullRange() Ranges

FullNotNullRange is (-∞, +∞) for Range.

func FullRange 

func FullRange() Ranges

FullRange is [null, +∞) for Range.

func NullRange 

func NullRange() Ranges

NullRange is [null, null] for Range.

func UnionRanges 

func UnionRanges(sctx *rangerctx.RangerContext, ranges Ranges, mergeConsecutive bool) (Ranges, error)

UnionRanges sorts `ranges`, union adjacent ones if possible. For two intervals [a, b], [c, d], we have guaranteed that a <= c. If b >= c. Then two intervals are overlapped. And this two can be merged as [a, max(b, d)]. Otherwise they aren't overlapped.

func (Ranges) CloneForPlanCache 

func (rs Ranges) CloneForPlanCache() MutableRanges

CloneForPlanCache clones the MutableRanges for plan cache.

func (Ranges) IntersectRanges 

func (rs Ranges) IntersectRanges(tc types.Context, otherRanges Ranges) Ranges

IntersectRanges computes pairwise intersection between each element in rs and otherRangeList.

func (Ranges) MemUsage 

func (rs Ranges) MemUsage() (sum int64)

MemUsage gets the memory usage of ranges.

func (Ranges) Range 

func (rs Ranges) Range() Ranges

Range returns the range array.

func (Ranges) Rebuild 

func (Ranges) Rebuild(planctx.PlanContext) error

Rebuild rebuilds this range.

func (Ranges) Subset 

func (rs Ranges) Subset(tc types.Context, superRanges Ranges) bool

Subset checks if a list of ranges(rs) is a subset of another list of ranges(superRanges). This is true if every range in the first list is a subset of any range in the second list. Also, we check if all elements of superRanges are covered.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.