advisor

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2018 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package advisor contain heuristic rules, index rules and explain translator.

Index

Constants

This section is empty.

Variables

View Source 
var HeuristicRules map[string]Rule

HeuristicRules 启发式规则列表

Functions

func CompleteColumnsInfo 

func CompleteColumnsInfo(stmt sqlparser.Statement, cols []*common.Column, env *env.VirtualEnv) []*common.Column

CompleteColumnsInfo 补全索引可能会用到列的所属库名、表名等信息

func DigestExplainText 

func DigestExplainText(text string)

DigestExplainText 分析用户输入的EXPLAIN信息

func DuplicateKeyChecker 

func DuplicateKeyChecker(conn *database.Connector, databases ...string) map[string]Rule

DuplicateKeyChecker 对所有用到的库表检查是否存在重复索引

func ExplainAdvisor 

func ExplainAdvisor(exp *database.ExplainInfo) map[string]Rule

ExplainAdvisor 基于explain信息给出建议

func FormatSuggest 

func FormatSuggest(sql string, format string, suggests ...map[string]Rule) (map[string]Rule, string)

FormatSuggest 格式化输出优化建议 目前支持：json, text两种形式，其他形式会给结构体的pretty.Println

func InBlackList 

func InBlackList(sql string) bool

InBlackList 判断一条请求是否在黑名单列表中 如果在返回true，表示不需要评审 注意这里没有做指纹判断，是否用指纹在这个函数的外面处理

func IsIgnoreRule 

func IsIgnoreRule(item string) bool

IsIgnoreRule 判断是否是过滤规则 支持XXX*前缀匹配，OK规则不可设置过滤

func ListHeuristicRules 

func ListHeuristicRules(rules ...map[string]Rule)

ListHeuristicRules 打印支持的启发式规则，对应命令行参数-list-heuristic-rules

func ListTestSQLs 

func ListTestSQLs()

ListTestSQLs 打印测试用的SQL，方便测试，对应命令行参数-list-test-sqls

func MergeConflictHeuristicRules 

func MergeConflictHeuristicRules(rules map[string]Rule) map[string]Rule

MergeConflictHeuristicRules merge conflict rules

Types

type IndexAdvises 

type IndexAdvises []IndexInfo

IndexAdvises IndexAdvises列表

func (IndexAdvises) Format 

func (idxAdvs IndexAdvises) Format() map[string]Rule

Format 用于格式化输出索引建议

type IndexAdvisor 

type IndexAdvisor struct {
	Ast sqlparser.Statement // Vitess Parser生成的抽象语法树

	IndexMeta map[string]map[string]*database.TableIndexInfo
	// contains filtered or unexported fields
}

IndexAdvisor 索引建议需要使用到的所有信息

func NewAdvisor 

func NewAdvisor(env *env.VirtualEnv, rEnv database.Connector, q Query4Audit) (*IndexAdvisor, error)

NewAdvisor 构造一个 IndexAdvisor 的时候就会对其本身结构初始化 获取 condition 中的等值条件、非等值条件，以及group by 、 order by信息

func (*IndexAdvisor) HeuristicCheck 

func (idxAdv *IndexAdvisor) HeuristicCheck(q Query4Audit) map[string]Rule

HeuristicCheck 依赖数据字典的启发式检查 IndexAdvisor会构建测试环境和数据字典，所以放在这里实现

func (*IndexAdvisor) IndexAdvise 

func (idxAdv *IndexAdvisor) IndexAdvise() IndexAdvises

IndexAdvise 索引优化建议算法入口主函数 TODO 索引顺序该如何确定

func (*IndexAdvisor) RuleGroupByConst 

func (idxAdv *IndexAdvisor) RuleGroupByConst() Rule

RuleGroupByConst GRP.001

func (*IndexAdvisor) RuleImplicitConversion 

func (idxAdv *IndexAdvisor) RuleImplicitConversion() Rule

RuleImplicitConversion ARG.003 隐式类型转换检查：该项检查一定是在开启测试环境或线上环境情境下下进行的

func (*IndexAdvisor) RuleImpossibleOuterJoin 

func (idxAdv *IndexAdvisor) RuleImpossibleOuterJoin() Rule

RuleImpossibleOuterJoin JOI.003 TODO: 未实现完

func (*IndexAdvisor) RuleOrderByConst 

func (idxAdv *IndexAdvisor) RuleOrderByConst() Rule

RuleOrderByConst CLA.005 TODO: SELECT col FROM tbl WHERE col IN('NEWS') ORDER BY col;

func (*IndexAdvisor) RuleUpdatePrimaryKey 

func (idxAdv *IndexAdvisor) RuleUpdatePrimaryKey() Rule

RuleUpdatePrimaryKey CLA.016

type IndexInfo 

type IndexInfo struct {
	Name          string           `json:"name"`           // 索引名称
	Database      string           `json:"database"`       // 数据库名
	Table         string           `json:"table"`          // 表名
	DDL           string           `json:"ddl"`            // ALTER, CREATE等类型的DDL语句
	ColumnDetails []*common.Column `json:"column_details"` // 列详情
}

IndexInfo 创建一条索引需要的信息

type Query4Audit 

type Query4Audit struct {
	Query  string              // 查询语句
	Stmt   sqlparser.Statement // 通过Vitess解析出的抽象语法树
	TiStmt []tidb.StmtNode     // 通过TiDB解析出的抽象语法树
}

Query4Audit 待评审的SQL结构体，由原SQL和其对应的抽象语法树组成

func NewQuery4Audit 

func NewQuery4Audit(sql string, options ...string) (*Query4Audit, error)

NewQuery4Audit return a struct for Query4Audit

func (*Query4Audit) RuleAddDefaultValue 

func (q *Query4Audit) RuleAddDefaultValue() Rule

RuleAddDefaultValue COL.004

func (*Query4Audit) RuleAddDelimiter 

func (q *Query4Audit) RuleAddDelimiter() Rule

RuleAddDelimiter LIT.004

func (*Query4Audit) RuleAllowEngine 

func (q *Query4Audit) RuleAllowEngine() Rule

RuleAllowEngine TBL.002

func (*Query4Audit) RuleAlterCharset 

func (q *Query4Audit) RuleAlterCharset() Rule

RuleAlterCharset ALT.001

func (*Query4Audit) RuleAlterDropColumn 

func (q *Query4Audit) RuleAlterDropColumn() Rule

RuleAlterDropColumn ALT.003

func (*Query4Audit) RuleAlterDropKey 

func (q *Query4Audit) RuleAlterDropKey() Rule

RuleAlterDropKey ALT.004

func (*Query4Audit) RuleAutoIncUnsigned 

func (q *Query4Audit) RuleAutoIncUnsigned() Rule

RuleAutoIncUnsigned COL.003:

func (*Query4Audit) RuleAutoIncrementInitNotZero 

func (q *Query4Audit) RuleAutoIncrementInitNotZero() Rule

RuleAutoIncrementInitNotZero TBL.004

func (*Query4Audit) RuleBlobDefaultValue 

func (q *Query4Audit) RuleBlobDefaultValue() Rule

RuleBlobDefaultValue COL.015

func (*Query4Audit) RuleCantBeNull 

func (q *Query4Audit) RuleCantBeNull() Rule

RuleCantBeNull COL.012

func (*Query4Audit) RuleColCommentCheck 

func (q *Query4Audit) RuleColCommentCheck() Rule

RuleColCommentCheck COL.005

func (*Query4Audit) RuleColumnWithCharset 

func (q *Query4Audit) RuleColumnWithCharset() Rule

RuleColumnWithCharset COL.014

func (*Query4Audit) RuleCommaAnsiJoin 

func (q *Query4Audit) RuleCommaAnsiJoin() Rule

RuleCommaAnsiJoin JOI.001

func (*Query4Audit) RuleCompareWithFunction 

func (q *Query4Audit) RuleCompareWithFunction() Rule

RuleCompareWithFunction FUN.001

func (*Query4Audit) RuleCountConst 

func (q *Query4Audit) RuleCountConst() Rule

RuleCountConst FUN.005

func (*Query4Audit) RuleCountDistinctMultiCol 

func (q *Query4Audit) RuleCountDistinctMultiCol() Rule

RuleCountDistinctMultiCol DIS.002

func (*Query4Audit) RuleCountStar 

func (q *Query4Audit) RuleCountStar() Rule

RuleCountStar FUN.002

func (*Query4Audit) RuleCreateDualTable 

func (q *Query4Audit) RuleCreateDualTable() Rule

RuleCreateDualTable TBL.003

func (*Query4Audit) RuleDataDrop 

func (q *Query4Audit) RuleDataDrop() Rule

RuleDataDrop SEC.003

func (*Query4Audit) RuleDataNotQuote 

func (q *Query4Audit) RuleDataNotQuote() Rule

RuleDataNotQuote LIT.002

func (*Query4Audit) RuleDiffGroupByOrderBy 

func (q *Query4Audit) RuleDiffGroupByOrderBy() Rule

RuleDiffGroupByOrderBy CLA.006

func (*Query4Audit) RuleDistinctJoinUsage 

func (q *Query4Audit) RuleDistinctJoinUsage() Rule

RuleDistinctJoinUsage SUB.003

func (*Query4Audit) RuleDistinctStar 

func (q *Query4Audit) RuleDistinctStar() Rule

RuleDistinctStar DIS.003

func (*Query4Audit) RuleDistinctUsage 

func (q *Query4Audit) RuleDistinctUsage() Rule

RuleDistinctUsage DIS.001

func (*Query4Audit) RuleDupJoin 

func (q *Query4Audit) RuleDupJoin() Rule

RuleDupJoin JOI.002

func (*Query4Audit) RuleEqualLike 

func (q *Query4Audit) RuleEqualLike() Rule

RuleEqualLike ARG.002

func (*Query4Audit) RuleExplicitOrderBy 

func (q *Query4Audit) RuleExplicitOrderBy() Rule

RuleExplicitOrderBy CLA.008

func (*Query4Audit) RuleForbiddenSyntax 

func (q *Query4Audit) RuleForbiddenSyntax() Rule

RuleForbiddenSyntax CLA.017

func (*Query4Audit) RuleGroupByConst 

func (q *Query4Audit) RuleGroupByConst() Rule

RuleGroupByConst CLA.004

func (*Query4Audit) RuleGroupByExpr 

func (q *Query4Audit) RuleGroupByExpr() Rule

RuleGroupByExpr CLA.010

func (*Query4Audit) RuleHavingClause 

func (q *Query4Audit) RuleHavingClause() Rule

RuleHavingClause CLA.013

func (*Query4Audit) RuleHint 

func (q *Query4Audit) RuleHint() Rule

RuleHint ARG.010 TODO: sql_no_cache, straight join

func (*Query4Audit) RuleIPString 

func (q *Query4Audit) RuleIPString() Rule

RuleIPString LIT.001

func (*Query4Audit) RuleIdxPrefix 

func (q *Query4Audit) RuleIdxPrefix() Rule

RuleIdxPrefix STA.003

func (*Query4Audit) RuleImplicitAlias 

func (q *Query4Audit) RuleImplicitAlias() Rule

RuleImplicitAlias ALI.001

func (*Query4Audit) RuleImpossibleWhere 

func (q *Query4Audit) RuleImpossibleWhere() Rule

RuleImpossibleWhere RES.006

func (*Query4Audit) RuleImpreciseDataType 

func (q *Query4Audit) RuleImpreciseDataType() Rule

RuleImpreciseDataType COL.009

func (*Query4Audit) RuleIn 

func (q *Query4Audit) RuleIn() Rule

RuleIn ARG.005 && ARG.004

func (*Query4Audit) RuleInSubquery 

func (q *Query4Audit) RuleInSubquery() Rule

RuleInSubquery SUB.001

func (*Query4Audit) RuleIndexAttributeOrder 

func (q *Query4Audit) RuleIndexAttributeOrder() Rule

RuleIndexAttributeOrder KEY.004

func (*Query4Audit) RuleInsertColDef 

func (q *Query4Audit) RuleInsertColDef() Rule

RuleInsertColDef COL.002

func (*Query4Audit) RuleInsertOnDup 

func (q *Query4Audit) RuleInsertOnDup() Rule

RuleInsertOnDup LCK.002

func (*Query4Audit) RuleInsertSelect 

func (q *Query4Audit) RuleInsertSelect() Rule

RuleInsertSelect LCK.001

func (*Query4Audit) RuleIntPrecision 

func (q *Query4Audit) RuleIntPrecision() Rule

RuleIntPrecision COL.016

func (*Query4Audit) RuleIsNullIsNotNull 

func (q *Query4Audit) RuleIsNullIsNotNull() Rule

RuleIsNullIsNotNull ARG.006

func (*Query4Audit) RuleLoadFile 

func (q *Query4Audit) RuleLoadFile() Rule

RuleLoadFile RES.008

func (*Query4Audit) RuleMeaninglessWhere 

func (q *Query4Audit) RuleMeaninglessWhere() Rule

RuleMeaninglessWhere RES.007

func (*Query4Audit) RuleMixOrderBy 

func (q *Query4Audit) RuleMixOrderBy() Rule

RuleMixOrderBy CLA.007

func (*Query4Audit) RuleMultiDBJoin 

func (q *Query4Audit) RuleMultiDBJoin() Rule

RuleMultiDBJoin JOI.008

func (*Query4Audit) RuleMultiDeleteUpdate 

func (q *Query4Audit) RuleMultiDeleteUpdate() Rule

RuleMultiDeleteUpdate JOI.007

func (*Query4Audit) RuleMultiValueAttribute 

func (q *Query4Audit) RuleMultiValueAttribute() Rule

RuleMultiValueAttribute LIT.003

func (*Query4Audit) RuleNestedSubQueries 

func (q *Query4Audit) RuleNestedSubQueries() Rule

RuleNestedSubQueries JOI.006

func (*Query4Audit) RuleNoDeterministicGroupby 

func (q *Query4Audit) RuleNoDeterministicGroupby() Rule

RuleNoDeterministicGroupby RES.001

func (*Query4Audit) RuleNoDeterministicLimit 

func (q *Query4Audit) RuleNoDeterministicLimit() Rule

RuleNoDeterministicLimit RES.002

func (*Query4Audit) RuleNoOSCKey 

func (q *Query4Audit) RuleNoOSCKey() Rule

RuleNoOSCKey KEY.002

func (*Query4Audit) RuleNoWhere 

func (q *Query4Audit) RuleNoWhere() Rule

RuleNoWhere CLA.001 & CLA.014 & CLA.015

func (*Query4Audit) RuleNot 

func (q *Query4Audit) RuleNot() Rule

RuleNot ARG.011

func (*Query4Audit) RuleNullUsage 

func (q *Query4Audit) RuleNullUsage() Rule

RuleNullUsage COL.011

func (*Query4Audit) RuleOK 

func (q *Query4Audit) RuleOK() Rule

RuleOK OK

func (*Query4Audit) RuleORUsage 

func (q *Query4Audit) RuleORUsage() Rule

RuleORUsage ARG.008

func (*Query4Audit) RuleOffsetLimit 

func (q *Query4Audit) RuleOffsetLimit() Rule

RuleOffsetLimit CLA.003

func (*Query4Audit) RuleOrderByConst 

func (q *Query4Audit) RuleOrderByConst() Rule

RuleOrderByConst CLA.005

func (*Query4Audit) RuleOrderByExpr 

func (q *Query4Audit) RuleOrderByExpr() Rule

RuleOrderByExpr CLA.009

func (*Query4Audit) RuleOrderByMultiDirection 

func (q *Query4Audit) RuleOrderByMultiDirection() Rule

RuleOrderByMultiDirection KEY.008

func (*Query4Audit) RuleOrderByRand 

func (q *Query4Audit) RuleOrderByRand() Rule

RuleOrderByRand CLA.002

func (*Query4Audit) RulePKNotInt 

func (q *Query4Audit) RulePKNotInt() Rule

RulePKNotInt KEY.007 && KEY.001

func (*Query4Audit) RulePartitionNotAllowed 

func (q *Query4Audit) RulePartitionNotAllowed() Rule

RulePartitionNotAllowed TBL.001

func (*Query4Audit) RulePatternMatchingUsage 

func (q *Query4Audit) RulePatternMatchingUsage() Rule

RulePatternMatchingUsage ARG.007

func (*Query4Audit) RulePluralWord 

func (q *Query4Audit) RulePluralWord() Rule

RulePluralWord KWR.003 Reference: https://en.wikipedia.org/wiki/English_plurals

func (*Query4Audit) RulePrefixLike 

func (q *Query4Audit) RulePrefixLike() Rule

RulePrefixLike ARG.001

func (*Query4Audit) RuleReadablePasswords 

func (q *Query4Audit) RuleReadablePasswords() Rule

RuleReadablePasswords SEC.002

func (*Query4Audit) RuleRecursiveDependency 

func (q *Query4Audit) RuleRecursiveDependency() Rule

RuleRecursiveDependency KEY.003

func (*Query4Audit) RuleReduceNumberOfJoin 

func (q *Query4Audit) RuleReduceNumberOfJoin() Rule

RuleReduceNumberOfJoin JOI.005

func (*Query4Audit) RuleSQLCalcFoundRows 

func (q *Query4Audit) RuleSQLCalcFoundRows() Rule

RuleSQLCalcFoundRows KWR.001

func (*Query4Audit) RuleSameAlias 

func (q *Query4Audit) RuleSameAlias() Rule

RuleSameAlias ALI.003

func (*Query4Audit) RuleSelectStar 

func (q *Query4Audit) RuleSelectStar() Rule

RuleSelectStar COL.001

func (*Query4Audit) RuleSpaceAfterDot 

func (q *Query4Audit) RuleSpaceAfterDot() Rule

RuleSpaceAfterDot STA.002

func (*Query4Audit) RuleSpaceWithQuote 

func (q *Query4Audit) RuleSpaceWithQuote() Rule

RuleSpaceWithQuote ARG.009

func (*Query4Audit) RuleSpaghettiQueryAlert 

func (q *Query4Audit) RuleSpaghettiQueryAlert() Rule

RuleSpaghettiQueryAlert CLA.012

func (*Query4Audit) RuleStandardINEQ 

func (q *Query4Audit) RuleStandardINEQ() Rule

RuleStandardINEQ STA.001

func (*Query4Audit) RuleStandardName 

func (q *Query4Audit) RuleStandardName() Rule

RuleStandardName STA.004

func (*Query4Audit) RuleStarAlias 

func (q *Query4Audit) RuleStarAlias() Rule

RuleStarAlias ALI.002

func (*Query4Audit) RuleStringConcatenation 

func (q *Query4Audit) RuleStringConcatenation() Rule

RuleStringConcatenation FUN.003

func (*Query4Audit) RuleSubQueryFunctions 

func (q *Query4Audit) RuleSubQueryFunctions() Rule

RuleSubQueryFunctions SUB.006

func (*Query4Audit) RuleSubQueryLimit 

func (q *Query4Audit) RuleSubQueryLimit() Rule

RuleSubQueryLimit SUB.005 只有IN的SUBQUERY限制了LIMIT，FROM子句中的SUBQUERY并未限制LIMIT

func (*Query4Audit) RuleSubqueryDepth 

func (q *Query4Audit) RuleSubqueryDepth() Rule

RuleSubqueryDepth SUB.004

func (*Query4Audit) RuleSumNPE 

func (q *Query4Audit) RuleSumNPE() Rule

RuleSumNPE FUN.006

func (*Query4Audit) RuleSysdate 

func (q *Query4Audit) RuleSysdate() Rule

RuleSysdate FUN.004

func (*Query4Audit) RuleTableCharsetCheck 

func (q *Query4Audit) RuleTableCharsetCheck() Rule

RuleTableCharsetCheck TBL.005

func (*Query4Audit) RuleTblCommentCheck 

func (q *Query4Audit) RuleTblCommentCheck() Rule

RuleTblCommentCheck CLA.011

func (*Query4Audit) RuleTimestampDefault 

func (q *Query4Audit) RuleTimestampDefault() Rule

RuleTimestampDefault COL.013

func (*Query4Audit) RuleTooManyFields 

func (q *Query4Audit) RuleTooManyFields() Rule

RuleTooManyFields COL.006

func (*Query4Audit) RuleTooManyKeyParts 

func (q *Query4Audit) RuleTooManyKeyParts() Rule

RuleTooManyKeyParts KEY.006

func (*Query4Audit) RuleTooManyKeys 

func (q *Query4Audit) RuleTooManyKeys() Rule

RuleTooManyKeys KEY.005

func (*Query4Audit) RuleTruncateTable 

func (q *Query4Audit) RuleTruncateTable() Rule

RuleTruncateTable SEC.001

func (*Query4Audit) RuleUNIONUsage 

func (q *Query4Audit) RuleUNIONUsage() Rule

RuleUNIONUsage SUB.002

func (*Query4Audit) RuleUniqueKeyDup 

func (q *Query4Audit) RuleUniqueKeyDup() Rule

RuleUniqueKeyDup KEY.009 TODO: 目前只是给建议，期望能够实现自动检查

func (*Query4Audit) RuleUpdateDeleteWithLimit 

func (q *Query4Audit) RuleUpdateDeleteWithLimit() Rule

RuleUpdateDeleteWithLimit RES.003

func (*Query4Audit) RuleUpdateDeleteWithOrderby 

func (q *Query4Audit) RuleUpdateDeleteWithOrderby() Rule

RuleUpdateDeleteWithOrderby RES.004

func (*Query4Audit) RuleUpdateSetAnd 

func (q *Query4Audit) RuleUpdateSetAnd() Rule

RuleUpdateSetAnd RES.005

func (*Query4Audit) RuleUseKeyWord 

func (q *Query4Audit) RuleUseKeyWord() Rule

RuleUseKeyWord KWR.002

func (*Query4Audit) RuleValuesInDefinition 

func (q *Query4Audit) RuleValuesInDefinition() Rule

RuleValuesInDefinition COL.010

func (*Query4Audit) RuleVarcharLength 

func (q *Query4Audit) RuleVarcharLength() Rule

RuleVarcharLength COL.017

func (*Query4Audit) RuleVarcharVSChar 

func (q *Query4Audit) RuleVarcharVSChar() Rule

RuleVarcharVSChar COL.008

type Rule 

type Rule struct {
	Item     string                  `json:"Item"`     // 规则代号
	Severity string                  `json:"Severity"` // 危险等级：L[0-8], 数字越大表示级别越高
	Summary  string                  `json:"Summary"`  // 规则摘要
	Content  string                  `json:"Content"`  // 规则解释
	Case     string                  `json:"Case"`     // SQL示例
	Position int                     `json:"Position"` // 建议所处SQL字符位置，默认0表示全局建议
	Func     func(*Query4Audit) Rule `json:"-"`        // 函数名
}

Rule 评审规则元数据结构

func RuleMySQLError 

func RuleMySQLError(item string, err error) Rule

RuleMySQLError ERR.XXX

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.