compile

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 13, 2023 License: Apache-2.0 Imports: 119 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DistributedThreshold   uint64 = 10 * mpool.MB
	SingleLineSizeEstimate uint64 = 300 * mpool.B
)

Note: Now the cost going from stat is actually the number of rows, so we can only estimate a number for the size of each row. The current insertion of around 200,000 rows triggers cn to write s3 directly

View Source 
const (
	Merge magicType = iota
	Normal
	Remote
	Parallel
	Pushdown
	CreateDatabase
	CreateTable
	CreateIndex
	DropDatabase
	DropTable
	DropIndex
	Deletion
	Insert
	InsertValues
	TruncateTable
	AlterView
	AlterTable
	MergeInsert
	MergeDelete
	CreateSequence
	DropSequence
	AlterSequence
	MagicDelete
	Replace
)

type of scope

View Source 
const (
	INDEX_TYPE_PRIMARY  = "PRIMARY"
	INDEX_TYPE_UNIQUE   = "UNIQUE"
	INDEX_TYPE_MULTIPLE = "MULTIPLE"
)
View Source 
const (
	INDEX_VISIBLE_YES = 1
	INDEX_VISIBLE_NO  = 0
)
View Source 
const (
	INDEX_HIDDEN_YES = 1
	INDEX_HIDDEN_NO  = 0
)
View Source 
const (
	NULL_VALUE   = "null"
	EMPTY_STRING = ""
)
View Source 
const (
	ALLOCID_INDEX_KEY = "index_key"
)
View Source 
const (

	// need to fix this in the future. for now, increase it to make tpch1T can run on 3 CN
	HandleNotifyTimeout = 300 * time.Second
)
View Source 
const StreamMaxInterval = 8192

Variables

This section is empty.

Functions

func ApplyRuntimeFilters added in v1.0.0 

func ApplyRuntimeFilters(
	ctx context.Context,
	proc *process.Process,
	tableDef *plan.TableDef,
	blockInfos [][]byte,
	exprs []*plan.Expr,
	runtimeFilters []*pipeline.RuntimeFilter,
) ([][]byte, error)

func CnServerMessageHandler added in v0.6.0 

func CnServerMessageHandler(
	ctx context.Context,
	cnAddr string,
	message morpc.Message,
	cs morpc.ClientSession,
	storeEngine engine.Engine,
	fileService fileservice.FileService,
	lockService lockservice.LockService,
	queryService queryservice.QueryService,
	cli client.TxnClient,
	aicm *defines.AutoIncrCacheManager,
	messageAcquirer func() morpc.Message) error

CnServerMessageHandler is responsible for processing the cn-client message received at cn-server. the message is always *pipeline.Message here. It's a byte array which encoded by method encodeScope.

func DebugShowScopes added in v0.6.0 

func DebugShowScopes(ss []*Scope) string

DebugShowScopes show information of a scope structure.

func NewSQLExecutor added in v0.8.0 

func NewSQLExecutor(
	addr string,
	eng engine.Engine,
	mp *mpool.MPool,
	txnClient client.TxnClient,
	fs fileservice.FileService,
	qs queryservice.QueryService,
	aicm *defines.AutoIncrCacheManager) executor.SQLExecutor

NewSQLExecutor returns a internal used sql service. It can execute sql in current CN.

Types

type Col 

type Col struct {
	Typ  types.T
	Name string
}

Col is the information of attribute

type Compile added in v0.5.0 

type Compile struct {
	// contains filtered or unexported fields
}

Compile contains all the information needed for compilation.

func New 

func New(
	addr, db, sql, tenant, uid string,
	ctx context.Context,
	e engine.Engine,
	proc *process.Process,
	stmt tree.Statement,
	isInternal bool,
	cnLabel map[string]string,
	startAt time.Time) *Compile

New is used to new an object of compile

func (*Compile) Compile added in v0.5.0 

func (c *Compile) Compile(ctx context.Context, pn *plan.Plan, u any, fill func(any, *batch.Batch) error) (err error)

Compile is the entrance of the compute-execute-layer. It generates a scope (logic pipeline) for a query plan.

func (*Compile) NeedInitTempEngine added in v0.7.0 

func (c *Compile) NeedInitTempEngine(InitTempEngine bool) bool

helper function to judge if init temporary engine is needed

func (*Compile) Run added in v0.5.0 

func (c *Compile) Run(_ uint64) (*util2.RunResult, error)

Run is an important function of the compute-layer, it executes a single sql according to its scope

func (*Compile) SetBuildPlanFunc added in v1.0.0 

func (c *Compile) SetBuildPlanFunc(buildPlanFunc func() (*plan2.Plan, error))

func (*Compile) SetTempEngine added in v0.7.0 

func (c *Compile) SetTempEngine(ctx context.Context, te engine.Engine)

type RemoteReceivRegInfo added in v0.7.0 

type RemoteReceivRegInfo struct {
	Idx      int
	Uuid     uuid.UUID
	FromAddr string
}

type RuntimeFilterEvaluator added in v1.0.0 

type RuntimeFilterEvaluator interface {
	Evaluate(objectio.ZoneMap) bool
}

type RuntimeInFilter added in v1.0.0 

type RuntimeInFilter struct {
	InList *vector.Vector
}

func (*RuntimeInFilter) Evaluate added in v1.0.0 

func (f *RuntimeInFilter) Evaluate(zm objectio.ZoneMap) bool

type RuntimeZonemapFilter added in v1.0.0 

type RuntimeZonemapFilter struct {
	Zm objectio.ZoneMap
}

func (*RuntimeZonemapFilter) Evaluate added in v1.0.0 

func (f *RuntimeZonemapFilter) Evaluate(zm objectio.ZoneMap) bool

type Scope 

type Scope struct {
	// Magic specifies the type of Scope.
	// 0 -  execution unit for reading data.
	// 1 -  execution unit for processing intermediate results.
	// 2 -  execution unit that requires remote call.
	Magic magicType

	// IsJoin means the pipeline is join
	IsJoin bool

	// IsEnd means the pipeline is end
	IsEnd bool

	// IsRemote means the pipeline is remote
	IsRemote bool

	// IsLoad means the pipeline is load
	IsLoad bool

	Plan *plan.Plan
	// DataSource stores information about data source.
	DataSource *Source
	// PreScopes contains children of this scope will inherit and execute.
	PreScopes []*Scope
	// NodeInfo contains the information about the remote node.
	NodeInfo engine.Node
	// Instructions contains command list of this scope.
	Instructions vm.Instructions
	// Proc contains the execution context.
	Proc *process.Process

	Reg *process.WaitRegister

	RemoteReceivRegInfos []RemoteReceivRegInfo

	BuildIdx       int
	ShuffleCnt     int
	PartialResults []any
}

Scope is the output of the compile process. Each sql will be compiled to one or more execution unit scopes.

func (*Scope) AlterSequence added in v1.0.0 

func (s *Scope) AlterSequence(c *Compile) error

func (*Scope) AlterTable added in v0.8.0 

func (s *Scope) AlterTable(c *Compile) error

func (*Scope) AlterTableCopy added in v1.0.0 

func (s *Scope) AlterTableCopy(c *Compile) error

func (*Scope) AlterTableInplace added in v1.0.0 

func (s *Scope) AlterTableInplace(c *Compile) error

func (*Scope) AlterView added in v0.7.0 

func (s *Scope) AlterView(c *Compile) error

Drop the old view, and create the new view.

func (*Scope) CreateDatabase 

func (s *Scope) CreateDatabase(c *Compile) error

func (*Scope) CreateIndex 

func (s *Scope) CreateIndex(c *Compile) error

func (*Scope) CreateSequence added in v0.8.0 

func (s *Scope) CreateSequence(c *Compile) error

func (*Scope) CreateTable 

func (s *Scope) CreateTable(c *Compile) error

func (*Scope) CreateTempTable added in v0.7.0 

func (s *Scope) CreateTempTable(c *Compile) error

func (*Scope) Delete 

func (s *Scope) Delete(c *Compile) (uint64, error)

func (*Scope) DropDatabase 

func (s *Scope) DropDatabase(c *Compile) error

func (*Scope) DropIndex 

func (s *Scope) DropIndex(c *Compile) error

func (*Scope) DropSequence added in v0.8.0 

func (s *Scope) DropSequence(c *Compile) error

func (*Scope) DropTable 

func (s *Scope) DropTable(c *Compile) error

func (*Scope) Insert 

func (s *Scope) Insert(c *Compile) (uint64, error)

func (*Scope) JoinRun added in v0.6.0 

func (s *Scope) JoinRun(c *Compile) error

func (*Scope) LoadRun added in v0.8.0 

func (s *Scope) LoadRun(c *Compile) error

func (*Scope) MergeRun 

func (s *Scope) MergeRun(c *Compile) error

MergeRun range and run the scope's pre-scopes by go-routine, and finally run itself to do merge work.

func (*Scope) ParallelRun 

func (s *Scope) ParallelRun(c *Compile, remote bool) error

ParallelRun try to execute the scope in parallel way.

func (*Scope) PushdownRun added in v0.6.0 

func (s *Scope) PushdownRun() error

func (*Scope) RemoteRun 

func (s *Scope) RemoteRun(c *Compile) error

RemoteRun send the scope to a remote node for execution. if no target node information, just execute it at local.

func (*Scope) Run 

func (s *Scope) Run(c *Compile) (err error)

Run read data from storage engine and run the instructions of scope.

func (*Scope) SetContextRecursively added in v0.8.0 

func (s *Scope) SetContextRecursively(ctx context.Context)

func (*Scope) TruncateTable added in v0.6.0 

func (s *Scope) TruncateTable(c *Compile) error

Truncation operations cannot be performed if the session holds an active table lock.

type Source 

type Source struct {
	PushdownId             uint64
	PushdownAddr           string
	SchemaName             string
	RelationName           string
	PartitionRelationNames []string
	Attributes             []string
	R                      engine.Reader
	Bat                    *batch.Batch
	Expr                   *plan.Expr
	TableDef               *plan.TableDef
	Timestamp              timestamp.Timestamp
	AccountId              *plan.PubInfo

	RuntimeFilterSpecs []*plan.RuntimeFilterSpec
}

Source contains information of a relation which will be used in execution,

type TxnOperator added in v0.6.0 

type TxnOperator = client.TxnOperator

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.