tabletenv

package
v0.21.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 29, 2024 License: Apache-2.0 Imports: 34 Imported by: 16

Documentation

Overview

Package tabletenv maintains environment variables and types that are common for all packages of tabletserver.

Index

Constants

View Source 
const (
	Enable       = "enable"
	Disable      = "disable"
	Dryrun       = "dryRun"
	NotOnPrimary = "notOnPrimary"
	Polling      = "polling"
	Heartbeat    = "heartbeat"
)

These constants represent values for various config parameters.

View Source 
const (
	// QuerySourceConsolidator means query result is found in consolidator.
	QuerySourceConsolidator = 1 << iota
	// QuerySourceMySQL means query result is returned from MySQL.
	QuerySourceMySQL
)

Variables

View Source 
var (

	// TxLogger can be used to enable logging of transactions.
	// Call TxLogger.ServeLogs in your main program to enable logging.
	// The log format can be inferred by looking at TxConnection.Format.
	TxLogger = streamlog.New[any]("TxLog", 10)

	// StatsLogger is the main stream logger object
	StatsLogger = streamlog.New[*LogStats]("TabletServer", 50)
)

Functions

func Init 

func Init()

Init must be called after flag.Parse, and before doing any other operations.

func IsLocalContext 

func IsLocalContext(ctx context.Context) bool

IsLocalContext returns true if the context is based on LocalContext.

func LocalContext 

func LocalContext() context.Context

LocalContext returns a context that's local to the process.

func RegisterTabletEnvFlags added in v0.15.0 

func RegisterTabletEnvFlags(fs *pflag.FlagSet)

RegisterTabletEnvFlags is a public API to register tabletenv flags for use by test cases that expect some flags to be set with default values

func SecondsVar 

func SecondsVar(fs *pflag.FlagSet, p *Seconds, name string, value Seconds, usage string)

SecondsVar is like a flag.Float64Var, but it works for Seconds.

Types

type ConnPoolConfig 

type ConnPoolConfig struct {
	Size               int           `json:"size,omitempty"`
	Timeout            time.Duration `json:"timeoutSeconds,omitempty"`
	IdleTimeout        time.Duration `json:"idleTimeoutSeconds,omitempty"`
	MaxLifetime        time.Duration `json:"maxLifetimeSeconds,omitempty"`
	PrefillParallelism int           `json:"prefillParallelism,omitempty"`
}

ConnPoolConfig contains the config for a conn pool.

func (*ConnPoolConfig) MarshalJSON added in v0.17.0 

func (cfg *ConnPoolConfig) MarshalJSON() ([]byte, error)

func (*ConnPoolConfig) UnmarshalJSON added in v0.19.0 

func (cfg *ConnPoolConfig) UnmarshalJSON(data []byte) (err error)

type Env 

type Env interface {
	CheckMySQL()
	Config() *TabletConfig
	Exporter() *servenv.Exporter
	Stats() *Stats
	LogError()
	Environment() *vtenv.Environment
}

Env defines the functions supported by TabletServer that the sub-components need to access.

func NewEnv 

func NewEnv(env *vtenv.Environment, config *TabletConfig, exporterName string) Env

NewEnv creates an Env that can be used for tabletserver subcomponents without an actual TabletServer.

type GracePeriodsConfig 

type GracePeriodsConfig struct {
	Shutdown   time.Duration
	Transition time.Duration
}

GracePeriodsConfig contains various grace periods. TODO(sougou): move lameduck here?

func (*GracePeriodsConfig) MarshalJSON added in v0.17.0 

func (cfg *GracePeriodsConfig) MarshalJSON() ([]byte, error)

func (*GracePeriodsConfig) UnmarshalJSON added in v0.19.0 

func (cfg *GracePeriodsConfig) UnmarshalJSON(data []byte) (err error)

type HealthcheckConfig 

type HealthcheckConfig struct {
	Interval           time.Duration
	DegradedThreshold  time.Duration
	UnhealthyThreshold time.Duration
}

HealthcheckConfig contains the config for healthcheck.

func (*HealthcheckConfig) MarshalJSON added in v0.17.0 

func (cfg *HealthcheckConfig) MarshalJSON() ([]byte, error)

func (*HealthcheckConfig) UnmarshalJSON added in v0.19.0 

func (cfg *HealthcheckConfig) UnmarshalJSON(data []byte) (err error)

type HotRowProtectionConfig 

type HotRowProtectionConfig struct {
	// Mode can be disable, dryRun or enable. Default is disable.
	Mode               string `json:"mode,omitempty"`
	MaxQueueSize       int    `json:"maxQueueSize,omitempty"`
	MaxGlobalQueueSize int    `json:"maxGlobalQueueSize,omitempty"`
	MaxConcurrency     int    `json:"maxConcurrency,omitempty"`
}

HotRowProtectionConfig contains the config for hot row protection.

type LogStats 

type LogStats struct {
	Ctx           context.Context
	Method        string
	Target        *querypb.Target
	PlanType      string
	OriginalSQL   string
	BindVariables map[string]*querypb.BindVariable

	RowsAffected         int
	NumberOfQueries      int
	StartTime            time.Time
	EndTime              time.Time
	MysqlResponseTime    time.Duration
	WaitingForConnection time.Duration
	QuerySources         byte
	Rows                 [][]sqltypes.Value
	TransactionID        int64
	ReservedID           int64
	Error                error
	CachedPlan           bool
	// contains filtered or unexported fields
}

LogStats records the stats for a single query

func NewLogStats 

func NewLogStats(ctx context.Context, methodName string) *LogStats

NewLogStats constructs a new LogStats with supplied Method and ctx field values, and the StartTime field set to the present time.

func (*LogStats) AddRewrittenSQL 

func (stats *LogStats) AddRewrittenSQL(sql string, start time.Time)

AddRewrittenSQL adds a single sql statement to the rewritten list

func (*LogStats) CallInfo 

func (stats *LogStats) CallInfo() (string, string)

CallInfo returns some parts of CallInfo if set

func (*LogStats) ContextHTML 

func (stats *LogStats) ContextHTML() safehtml.HTML

ContextHTML returns the HTML version of the context that was used, or "". This is a method on LogStats instead of a field so that it doesn't need to be passed by value everywhere.

func (*LogStats) EffectiveCaller 

func (stats *LogStats) EffectiveCaller() string

EffectiveCaller returns the effective caller stored in LogStats.Ctx

func (*LogStats) ErrorStr 

func (stats *LogStats) ErrorStr() string

ErrorStr returns the error string or ""

func (*LogStats) EventTime 

func (stats *LogStats) EventTime() time.Time

EventTime returns the time the event was created.

func (*LogStats) FmtQuerySources 

func (stats *LogStats) FmtQuerySources() string

FmtQuerySources returns a comma separated list of query sources. If there were no query sources, it returns the string "none".

func (*LogStats) ImmediateCaller 

func (stats *LogStats) ImmediateCaller() string

ImmediateCaller returns the immediate caller stored in LogStats.Ctx

func (*LogStats) Logf 

func (stats *LogStats) Logf(w io.Writer, params url.Values) error

Logf formats the log record to the given writer, either as tab-separated list of logged fields or as JSON.

func (*LogStats) RewrittenSQL 

func (stats *LogStats) RewrittenSQL() string

RewrittenSQL returns a semicolon separated list of SQL statements that were executed.

func (*LogStats) Send 

func (stats *LogStats) Send()

Send finalizes a record and sends it

func (*LogStats) SizeOfResponse 

func (stats *LogStats) SizeOfResponse() int

SizeOfResponse returns the approximate size of the response in bytes (this does not take in account protocol encoding). It will return 0 for streaming requests.

func (*LogStats) TotalTime 

func (stats *LogStats) TotalTime() time.Duration

TotalTime returns how long this query has been running

type OlapConfig added in v0.15.0 

type OlapConfig struct {
	TxTimeout time.Duration `json:"txTimeoutSeconds,omitempty"`
}

OlapConfig contains the config for olap settings.

func (*OlapConfig) MarshalJSON added in v0.17.0 

func (cfg *OlapConfig) MarshalJSON() ([]byte, error)

func (*OlapConfig) UnmarshalJSON added in v0.19.0 

func (cfg *OlapConfig) UnmarshalJSON(data []byte) (err error)

type OltpConfig 

type OltpConfig struct {
	QueryTimeout time.Duration `json:"queryTimeoutSeconds,omitempty"`
	TxTimeout    time.Duration `json:"txTimeoutSeconds,omitempty"`
	MaxRows      int           `json:"maxRows,omitempty"`
	WarnRows     int           `json:"warnRows,omitempty"`
}

OltpConfig contains the config for oltp settings.

func (*OltpConfig) MarshalJSON added in v0.17.0 

func (cfg *OltpConfig) MarshalJSON() ([]byte, error)

func (*OltpConfig) UnmarshalJSON added in v0.19.0 

func (cfg *OltpConfig) UnmarshalJSON(data []byte) (err error)

type ReplicationTrackerConfig 

type ReplicationTrackerConfig struct {
	// Mode can be disable, polling or heartbeat. Default is disable.
	Mode              string `json:"mode,omitempty"`
	HeartbeatInterval time.Duration
	HeartbeatOnDemand time.Duration
}

ReplicationTrackerConfig contains the config for the replication tracker.

func (*ReplicationTrackerConfig) MarshalJSON added in v0.17.0 

func (cfg *ReplicationTrackerConfig) MarshalJSON() ([]byte, error)

func (*ReplicationTrackerConfig) UnmarshalJSON added in v0.19.0 

func (cfg *ReplicationTrackerConfig) UnmarshalJSON(data []byte) (err error)

type RowStreamerConfig added in v0.14.0 

type RowStreamerConfig struct {
	MaxInnoDBTrxHistLen int64 `json:"maxInnoDBTrxHistLen,omitempty"`
	MaxMySQLReplLagSecs int64 `json:"maxMySQLReplLagSecs,omitempty"`
}

RowStreamerConfig contains configuration parameters for a vstreamer (source) that is copying the contents of a table to a target

type Seconds 

type Seconds float64

Seconds provides convenience functions for extracting duration from float64 seconds values.

func (Seconds) Get 

func (s Seconds) Get() time.Duration

Get converts Seconds to time.Duration

func (*Seconds) Set 

func (s *Seconds) Set(d time.Duration)

Set sets the value from time.Duration

type Stats 

type Stats struct {
	MySQLTimings           *servenv.TimingsWrapper        // Time spent executing MySQL commands
	QueryTimings           *servenv.TimingsWrapper        // Query timings
	QPSRates               *stats.Rates                   // Human readable QPS rates
	WaitTimings            *servenv.TimingsWrapper        // waits like Consolidations etc
	KillCounters           *stats.CountersWithSingleLabel // Connection and transaction kills
	ErrorCounters          *stats.CountersWithSingleLabel
	InternalErrors         *stats.CountersWithSingleLabel
	Warnings               *stats.CountersWithSingleLabel
	Unresolved             *stats.GaugesWithSingleLabel   // For now, only Prepares are tracked
	UserTableQueryCount    *stats.CountersWithMultiLabels // Per CallerID/table counts
	UserTableQueryTimesNs  *stats.CountersWithMultiLabels // Per CallerID/table latencies
	UserTransactionCount   *stats.CountersWithMultiLabels // Per CallerID transaction counts
	UserTransactionTimesNs *stats.CountersWithMultiLabels // Per CallerID transaction latencies
	ResultHistogram        *stats.Histogram               // Row count histograms
	TableaclAllowed        *stats.CountersWithMultiLabels // Number of allows
	TableaclDenied         *stats.CountersWithMultiLabels // Number of denials
	TableaclPseudoDenied   *stats.CountersWithMultiLabels // Number of pseudo denials

	UserActiveReservedCount *stats.CountersWithSingleLabel // Per CallerID active reserved connection counts
	UserReservedCount       *stats.CountersWithSingleLabel // Per CallerID reserved connection counts
	UserReservedTimesNs     *stats.CountersWithSingleLabel // Per CallerID reserved connection duration

	QueryTimingsByTabletType *servenv.TimingsWrapper // Query timings split by current tablet type
}

Stats contains tracked by various parts of TabletServer.

func NewStats 

func NewStats(exporter *servenv.Exporter) *Stats

NewStats instantiates a new set of stats scoped by exporter.

func (*Stats) Stop added in v0.18.0 

func (st *Stats) Stop()

type TabletConfig 

type TabletConfig struct {
	DB *dbconfigs.DBConfigs `json:"db,omitempty"`

	Unmanaged bool `json:"unmanaged,omitempty"`

	OltpReadPool ConnPoolConfig `json:"oltpReadPool,omitempty"`
	OlapReadPool ConnPoolConfig `json:"olapReadPool,omitempty"`
	TxPool       ConnPoolConfig `json:"txPool,omitempty"`

	Olap             OlapConfig             `json:"olap,omitempty"`
	Oltp             OltpConfig             `json:"oltp,omitempty"`
	HotRowProtection HotRowProtectionConfig `json:"hotRowProtection,omitempty"`

	Healthcheck  HealthcheckConfig  `json:"healthcheck,omitempty"`
	GracePeriods GracePeriodsConfig `json:"gracePeriods,omitempty"`

	ReplicationTracker ReplicationTrackerConfig `json:"replicationTracker,omitempty"`

	// Consolidator can be enable, disable, or notOnPrimary. Default is enable.
	Consolidator                string        `json:"consolidator,omitempty"`
	PassthroughDML              bool          `json:"passthroughDML,omitempty"`
	StreamBufferSize            int           `json:"streamBufferSize,omitempty"`
	ConsolidatorStreamTotalSize int64         `json:"consolidatorStreamTotalSize,omitempty"`
	ConsolidatorStreamQuerySize int64         `json:"consolidatorStreamQuerySize,omitempty"`
	QueryCacheMemory            int64         `json:"queryCacheMemory,omitempty"`
	QueryCacheDoorkeeper        bool          `json:"queryCacheDoorkeeper,omitempty"`
	SchemaReloadInterval        time.Duration `json:"schemaReloadIntervalSeconds,omitempty"`
	SchemaChangeReloadTimeout   time.Duration `json:"schemaChangeReloadTimeout,omitempty"`
	WatchReplication            bool          `json:"watchReplication,omitempty"`
	TrackSchemaVersions         bool          `json:"trackSchemaVersions,omitempty"`
	SchemaVersionMaxAgeSeconds  int64         `json:"schemaVersionMaxAgeSeconds,omitempty"`
	TerseErrors                 bool          `json:"terseErrors,omitempty"`
	TruncateErrorLen            int           `json:"truncateErrorLen,omitempty"`
	AnnotateQueries             bool          `json:"annotateQueries,omitempty"`
	MessagePostponeParallelism  int           `json:"messagePostponeParallelism,omitempty"`
	SignalWhenSchemaChange      bool          `json:"signalWhenSchemaChange,omitempty"`

	ExternalConnections map[string]*dbconfigs.DBConfigs `json:"externalConnections,omitempty"`

	SanitizeLogMessages  bool    `json:"-"`
	StrictTableACL       bool    `json:"-"`
	EnableTableACLDryRun bool    `json:"-"`
	TableACLExemptACL    string  `json:"-"`
	TwoPCEnable          bool    `json:"-"`
	TwoPCAbandonAge      Seconds `json:"-"`

	EnableTxThrottler              bool                          `json:"-"`
	TxThrottlerConfig              *TxThrottlerConfigFlag        `json:"-"`
	TxThrottlerHealthCheckCells    []string                      `json:"-"`
	TxThrottlerDefaultPriority     int                           `json:"-"`
	TxThrottlerTabletTypes         *topoproto.TabletTypeListFlag `json:"-"`
	TxThrottlerTopoRefreshInterval time.Duration                 `json:"-"`
	TxThrottlerDryRun              bool                          `json:"-"`

	EnableTableGC bool `json:"-"` // can be turned off programmatically by tests

	TransactionLimitConfig `json:"-"`

	EnforceStrictTransTables bool `json:"-"`
	EnableOnlineDDL          bool `json:"-"`

	RowStreamer RowStreamerConfig `json:"rowStreamer,omitempty"`

	EnableViews bool `json:"-"`

	EnablePerWorkloadTableMetrics bool `json:"-"`
}

TabletConfig contains all the configuration for query service

func NewCurrentConfig 

func NewCurrentConfig() *TabletConfig

NewCurrentConfig returns a copy of the current config.

func NewDefaultConfig 

func NewDefaultConfig() *TabletConfig

NewDefaultConfig returns a new TabletConfig with pre-initialized defaults.

func (*TabletConfig) Clone 

func (c *TabletConfig) Clone() *TabletConfig

Clone creates a clone of TabletConfig.

func (*TabletConfig) MarshalJSON added in v0.17.0 

func (cfg *TabletConfig) MarshalJSON() ([]byte, error)

func (*TabletConfig) SetTxTimeoutForWorkload added in v0.15.0 

func (c *TabletConfig) SetTxTimeoutForWorkload(val time.Duration, workload querypb.ExecuteOptions_Workload)

SetTxTimeoutForWorkload updates workload transaction timeouts. Used in tests only.

func (*TabletConfig) TxTimeoutForWorkload added in v0.15.0 

func (c *TabletConfig) TxTimeoutForWorkload(workload querypb.ExecuteOptions_Workload) time.Duration

TxTimeoutForWorkload returns the transaction timeout for the given workload type. Defaults to returning OLTP timeout.

func (*TabletConfig) UnmarshalJSON added in v0.19.0 

func (cfg *TabletConfig) UnmarshalJSON(data []byte) (err error)

func (*TabletConfig) Verify 

func (c *TabletConfig) Verify() error

Verify checks for contradicting flags.

type TransactionLimitConfig 

type TransactionLimitConfig struct {
	EnableTransactionLimit         bool
	EnableTransactionLimitDryRun   bool
	TransactionLimitPerUser        float64
	TransactionLimitByUsername     bool
	TransactionLimitByPrincipal    bool
	TransactionLimitByComponent    bool
	TransactionLimitBySubcomponent bool
}

TransactionLimitConfig captures configuration of transaction pool slots limiter configuration.

type TxThrottlerConfigFlag added in v0.18.0 

type TxThrottlerConfigFlag struct {
	*throttlerdatapb.Configuration
}

func NewTxThrottlerConfigFlag added in v0.18.0 

func NewTxThrottlerConfigFlag() *TxThrottlerConfigFlag

func (*TxThrottlerConfigFlag) Get added in v0.18.0 

func (t *TxThrottlerConfigFlag) Get() *throttlerdatapb.Configuration

func (*TxThrottlerConfigFlag) Set added in v0.18.0 

func (t *TxThrottlerConfigFlag) Set(arg string) error

func (*TxThrottlerConfigFlag) Type added in v0.18.0 

func (t *TxThrottlerConfigFlag) Type() string

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.