pgserver

package
v0.0.0-...-be912e8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 20, 2024 License: Apache-2.0 Imports: 49 Imported by: 0

README

The code in this directory was copied and modified from the DoltgreSQL project (as of 2024-10-25, https://github.com/dolthub/doltgresql/blob/main/server). The original code is licensed under the Apache License, Version 2.0. The modifications are also licensed under the Apache License, Version 2.0.

Documentation

Overview

Copyright 2024 Dolthub, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

View Source
const (
	SASLMechanism_SCRAM_SHA_256      = "SCRAM-SHA-256"
	SASLMechanism_SCRAM_SHA_256_PLUS = "SCRAM-SHA-256-PLUS"
)

These are mechanisms that are used for SASL authentication.

View Source
const (
	CopyFormatParquet = tree.CopyFormatCSV + 1
	CopyFormatJSON    = tree.CopyFormatCSV + 2
	CopyFormatArrow   = tree.CopyFormatCSV + 3
)
View Source
const PrintErrorStackTracesEnvKey = "MYDUCK_PRINT_ERROR_STACK_TRACES"

Variables

View Source
var EnableAuthentication = true

EnableAuthentication handles whether authentication is enabled. If enabled, it verifies that the given user exists, and checks that the encrypted password is derivable from the stored encrypted password.

View Source
var ErrCopyAborted = fmt.Errorf("COPY operation aborted")
View Source
var HandlePanics = true

HandlePanics determines whether panics should be handled in the connection handler. See |disablePanicHandlingEnvVar|.

Functions

func ConvertToSys

func ConvertToSys(sql string) string

func ExecuteRestore

func ExecuteRestore(dbName, localDir, localFile, remoteUri, endpoint, accessKeyId, secretAccessKey string) (string, error)

func GetSqlDatabaseFromContext

func GetSqlDatabaseFromContext(ctx *sql.Context, database string) (sql.Database, error)

GetSqlDatabaseFromContext returns the database from the context. Uses the context's current database if an empty string is provided. Returns nil if the database was not found.

func GetSqlTableFromContext

func GetSqlTableFromContext(ctx *sql.Context, databaseName string, tableName string) (sql.Table, error)

GetSqlTableFromContext returns the table from the context. Uses the context's current database if an empty database name is provided. Returns nil if no table was found. TODO(fan): Support search_path.

func GetStatementTag

func GetStatementTag(stmt *duckdb.Stmt) string

func GuessStatementTag

func GuessStatementTag(query string) string

func InitSuperuser

func InitSuperuser(password string)

func IsWellKnownStatementTag

func IsWellKnownStatementTag(tag string) bool

func ParseCopyFrom

func ParseCopyFrom(stmt string) (target string, format tree.CopyFormat, options string, ok bool)

ParseCopyFrom parses a COPY FROM statement and returns the query, format, and options.

func ParseCopyOptions

func ParseCopyOptions(options string, allowed map[string]OptionValueType) (result map[string]any, err error)

ParseCopyOptions parses the options string and returns the CopyOptions. The options string is a comma-separated list of key-value pairs: `OPT1 1, OPT2, OPT3 'v3', OPT4 E'v4', ...`. The allowed map specifies the allowed options and their types. Its keys are the option names in uppercase.

func ParseCopyTo

func ParseCopyTo(stmt string) (query string, format tree.CopyFormat, options string, ok bool)

ParseCopyTo parses a COPY TO statement and returns the query, format, and options.

func ParseFormat

func ParseFormat(s string) (format tree.CopyFormat, ok bool)

func RemoveComments

func RemoveComments(query string) string

RemoveComments removes comments from a query string. It supports line comments (--), block comments (/* ... */), and quoted strings. Author: Claude Sonnet 3.5

func RemoveLeadingComments

func RemoveLeadingComments(query string) string

func SearchPath

func SearchPath(ctx *sql.Context) ([]string, error)

SearchPath returns all the schemas in the search_path setting, with elements like "$user" expanded

func ValidateCopyFrom

func ValidateCopyFrom(cf *tree.CopyFrom, ctx *sql.Context) (sql.InsertableTable, error)

ValidateCopyFrom returns an error if the CopyFrom node is invalid.

func ValidateCopyTo

func ValidateCopyTo(ct *tree.CopyTo, ctx *sql.Context) (sql.Table, error)

ValidateCopyTo returns an error if the CopyTo node is invalid, for example if it contains columns that are not in the table schema.

func VitessTypeToObjectID

func VitessTypeToObjectID(typ query.Type) (uint32, error)

VitessTypeToObjectID returns a type, as defined by Vitess, into a type as defined by Postgres. OIDs can be obtained with the following query: `SELECT oid, typname FROM pg_type ORDER BY 1;`

Types

type Action

type Action string

Action represents the type of SQL action.

const (
	Create       Action = "CREATE"
	Drop         Action = "DROP"
	AlterDisable Action = "DISABLE"
	AlterEnable  Action = "ENABLE"
)

type ArrowDataLoader

type ArrowDataLoader struct {
	PipeDataLoader
	// contains filtered or unexported fields
}

type ArrowWriter

type ArrowWriter struct {
	// contains filtered or unexported fields
}

func NewArrowWriter

func NewArrowWriter(
	ctx *sql.Context,
	handler *DuckHandler,
	schema string, table sql.Table, columns tree.NameList,
	query string,
	rawOptions string,
) (*ArrowWriter, error)

func (*ArrowWriter) Close

func (dw *ArrowWriter) Close()

func (*ArrowWriter) Start

func (dw *ArrowWriter) Start() (string, chan CopyToResult, error)

type BackupConfig

type BackupConfig struct {
	DbName        string
	RemotePath    string
	StorageConfig *storage.ObjectStorageConfig
}

func NewBackupConfig

func NewBackupConfig(dbName, remotePath string, storageConfig *storage.ObjectStorageConfig) *BackupConfig

type ConnectionDetails

type ConnectionDetails struct {
	DBName   string
	Host     string
	Port     string
	User     string
	Password string
}

ConnectionDetails holds parsed connection string components.

type ConnectionHandler

type ConnectionHandler struct {
	// contains filtered or unexported fields
}

ConnectionHandler is responsible for the entire lifecycle of a user connection: receiving messages they send, executing queries, sending the correct messages in return, and terminating the connection when appropriate.

func NewConnectionHandler

func NewConnectionHandler(conn net.Conn, handler mysql.Handler, engine *gms.Engine, sm *server.SessionManager, connID uint32, server *Server) *ConnectionHandler

NewConnectionHandler returns a new ConnectionHandler for the connection provided

func (*ConnectionHandler) Conn

func (h *ConnectionHandler) Conn() net.Conn

Conn returns the underlying net.Conn for this connection.

func (*ConnectionHandler) HandleConnection

func (h *ConnectionHandler) HandleConnection()

HandleConnection handles a connection's session, reading messages, executing queries, and sending responses. Expected to run in a goroutine per connection.

type ConvertedStatement

type ConvertedStatement struct {
	String             string
	AST                tree.Statement
	Tag                string
	PgParsable         bool
	SubscriptionConfig *SubscriptionConfig
	BackupConfig       *BackupConfig
}

ConvertedStatement represents a statement that has been converted from the Postgres representation to the Vitess representation. String may contain the string version of the converted statement. AST will contain the tree version of the converted statement, and is the recommended form to use. If AST is nil, then use the String version, otherwise always prefer to AST.

type CopyToResult

type CopyToResult struct {
	RowCount int64
	Err      error
}

type CsvDataLoader

type CsvDataLoader struct {
	PipeDataLoader
	// contains filtered or unexported fields
}

type DataLoader

type DataLoader interface {
	// Start prepares the DataLoader for loading data. This may involve creating goroutines, opening files, etc.
	// Start must be called before any calls to LoadChunk.
	Start() <-chan error

	// LoadChunk reads the records from |data| and inserts them into the previously configured table. Data records
	// are not guaranteed to stard and end cleanly on chunk boundaries, so implementations must recognize incomplete
	// records and save them to prepend on the next processed chunk.
	LoadChunk(ctx *sql.Context, data []byte) error

	// Abort aborts the current load operation and releases all used resources.
	Abort(ctx *sql.Context) error

	// Finish finalizes the current load operation and commits the inserted rows so that the data becomes visibile
	// to clients. Implementations should check that the last call to LoadChunk did not end with an incomplete
	// record and return an error to the caller if so. The returned LoadDataResults describe the load operation,
	// including how many rows were inserted.
	Finish(ctx *sql.Context) (*LoadDataResults, error)
}

DataLoader allows callers to insert rows from multiple chunks into a table. Rows encoded in each chunk will not necessarily end cleanly on a chunk boundary, so DataLoader implementations must handle recognizing partial, or incomplete records, and saving that partial record until the next call to LoadChunk, so that it may be prefixed with the incomplete record.

func NewArrowDataLoader

func NewArrowDataLoader(ctx *sql.Context, handler *DuckHandler, schema string, table sql.InsertableTable, columns tree.NameList, options string) (DataLoader, error)

func NewCsvDataLoader

func NewCsvDataLoader(
	ctx *sql.Context, handler *DuckHandler,
	schema string, table sql.InsertableTable, columns tree.NameList, options *tree.CopyOptions,
	rawOptions string,
) (DataLoader, error)

type DataWriter

type DataWriter interface {
	Start() (string, chan CopyToResult, error)
	Close()
}

type DriverRowIter

type DriverRowIter struct {
	// contains filtered or unexported fields
}

DriverRowIter wraps a standard driver.Rows as a RowIter.

func NewDriverRowIter

func NewDriverRowIter(rows driver.Rows, schema sql.Schema) (*DriverRowIter, error)

func (*DriverRowIter) Close

func (iter *DriverRowIter) Close(ctx *sql.Context) error

Close closes the underlying driver.Rows.

func (*DriverRowIter) Next

func (iter *DriverRowIter) Next(ctx *sql.Context) (sql.Row, error)

Next retrieves the next row. It will return io.EOF if it's the last row.

func (*DriverRowIter) String

func (iter *DriverRowIter) String() string

type DuckDataWriter

type DuckDataWriter struct {
	// contains filtered or unexported fields
}

func NewDuckDataWriter

func NewDuckDataWriter(
	ctx *sql.Context,
	handler *DuckHandler,
	schema string, table sql.Table, columns tree.NameList,
	query string,
	options *tree.CopyOptions, rawOptions string,
) (*DuckDataWriter, error)

func (*DuckDataWriter) Close

func (dw *DuckDataWriter) Close()

func (*DuckDataWriter) Start

func (dw *DuckDataWriter) Start() (string, chan CopyToResult, error)

type DuckHandler

type DuckHandler struct {
	// contains filtered or unexported fields
}

DuckHandler is a handler uses DuckDB and the SQLe engine directly running Postgres specific queries.

func (*DuckHandler) ComBind

func (h *DuckHandler) ComBind(ctx context.Context, c *mysql.Conn, prepared PreparedStatementData, bindVars []any) ([]pgproto3.FieldDescription, error)

ComBind implements the Handler interface.

func (*DuckHandler) ComExecuteBound

func (h *DuckHandler) ComExecuteBound(ctx context.Context, conn *mysql.Conn, portal PortalData, callback func(*Result) error) error

ComExecuteBound implements the Handler interface.

func (*DuckHandler) ComPrepareParsed

func (h *DuckHandler) ComPrepareParsed(ctx context.Context, c *mysql.Conn, query string, parsed tree.Statement) (*duckdb.Stmt, []uint32, []pgproto3.FieldDescription, error)

ComPrepareParsed implements the Handler interface.

func (*DuckHandler) ComQuery

func (h *DuckHandler) ComQuery(ctx context.Context, c *mysql.Conn, query string, parsed tree.Statement, callback func(*Result) error) error

ComQuery implements the Handler interface.

func (*DuckHandler) ComResetConnection

func (h *DuckHandler) ComResetConnection(c *mysql.Conn) error

ComResetConnection implements the Handler interface.

func (*DuckHandler) ConnectionClosed

func (h *DuckHandler) ConnectionClosed(c *mysql.Conn)

ConnectionClosed implements the Handler interface.

func (*DuckHandler) NewConnection

func (h *DuckHandler) NewConnection(c *mysql.Conn)

NewConnection implements the Handler interface.

func (*DuckHandler) NewContext

func (h *DuckHandler) NewContext(ctx context.Context, c *mysql.Conn, query string) (*sql.Context, error)

NewContext implements the Handler interface.

type ErrorResponseSeverity

type ErrorResponseSeverity string

ErrorResponseSeverity represents the severity of an ErrorResponse message.

const (
	ErrorResponseSeverity_Error   ErrorResponseSeverity = "ERROR"
	ErrorResponseSeverity_Fatal   ErrorResponseSeverity = "FATAL"
	ErrorResponseSeverity_Panic   ErrorResponseSeverity = "PANIC"
	ErrorResponseSeverity_Warning ErrorResponseSeverity = "WARNING"
	ErrorResponseSeverity_Notice  ErrorResponseSeverity = "NOTICE"
	ErrorResponseSeverity_Debug   ErrorResponseSeverity = "DEBUG"
	ErrorResponseSeverity_Info    ErrorResponseSeverity = "INFO"
	ErrorResponseSeverity_Log     ErrorResponseSeverity = "LOG"
)

type Handler

type Handler interface {
	// ComBind is called when a connection receives a request to bind a prepared statement to a set of values.
	ComBind(ctx context.Context, c *mysql.Conn, prepared PreparedStatementData, bindVars []any) ([]pgproto3.FieldDescription, error)
	// ComExecuteBound is called when a connection receives a request to execute a prepared statement that has already bound to a set of values.
	ComExecuteBound(ctx context.Context, conn *mysql.Conn, portal PortalData, callback func(*Result) error) error
	// ComPrepareParsed is called when a connection receives a prepared statement query that has already been parsed.
	ComPrepareParsed(ctx context.Context, c *mysql.Conn, query string, parsed tree.Statement) (*duckdb.Stmt, []uint32, []pgproto3.FieldDescription, error)
	// ComQuery is called when a connection receives a query. Note the contents of the query slice may change
	// after the first call to callback. So the DoltgresHandler should not hang on to the byte slice.
	ComQuery(ctx context.Context, c *mysql.Conn, query string, parsed tree.Statement, callback func(*Result) error) error
	// ComResetConnection resets the connection's session, clearing out any cached prepared statements, locks, user and
	// session variables. The currently selected database is preserved.
	ComResetConnection(c *mysql.Conn) error
	// ConnectionClosed reports that a connection has been closed.
	ConnectionClosed(c *mysql.Conn)
	// NewConnection reports that a new connection has been established.
	NewConnection(c *mysql.Conn)
	// NewContext creates a new sql.Context instance for the connection |c|. The
	// optional |query| can be specified to populate the sql.Context's query field.
	NewContext(ctx context.Context, c *mysql.Conn, query string) (*sql.Context, error)
}

type Listener

type Listener struct {
	// contains filtered or unexported fields
}

Listener listens for connections to process PostgreSQL requests into Dolt requests.

func NewListener

func NewListener(listenerCfg mysql.ListenerConfig) (*Listener, error)

NewListener creates a new Listener.

func NewListenerWithOpts

func NewListenerWithOpts(listenerCfg mysql.ListenerConfig, opts ...ListenerOpt) (*Listener, error)

func (*Listener) Accept

func (l *Listener) Accept(server *Server)

Accept handles incoming connections.

func (*Listener) Addr

func (l *Listener) Addr() net.Addr

Addr returns the address that the listener is listening on.

func (*Listener) Close

func (l *Listener) Close()

Close stops the handling of incoming connections.

func (*Listener) Engine

func (l *Listener) Engine() *gms.Engine

Engine returns the engine that the listener is using.

func (*Listener) SessionManager

func (l *Listener) SessionManager() *server.SessionManager

SessionManager returns the session manager that the listener is using.

type ListenerOpt

type ListenerOpt func(*Listener)

func WithCertificate

func WithCertificate(cert tls.Certificate) ListenerOpt

func WithConnID

func WithConnID(connID *atomic.Uint32) ListenerOpt

func WithEngine

func WithEngine(engine *gms.Engine) ListenerOpt

func WithSessionManager

func WithSessionManager(sm *server.SessionManager) ListenerOpt

type LoadDataResults

type LoadDataResults struct {
	// RowsLoaded contains the total number of rows inserted during a load data operation.
	RowsLoaded int32
}

LoadDataResults contains the results of a load data operation, including the number of rows loaded.

type OptionValueType

type OptionValueType uint8
const (
	OptionValueTypeBool   OptionValueType = iota // bool
	OptionValueTypeInt                           // int
	OptionValueTypeFloat                         // float64
	OptionValueTypeString                        // string
)

type PGCatalogHandler

type PGCatalogHandler struct {
	// HandledInPlace is a function that determines if the query should be handled in place and not passed to the engine.
	HandledInPlace func(ConvertedStatement) (bool, error)
	Handler        func(*ConnectionHandler, ConvertedStatement) (bool, error)
}

type PipeDataLoader

type PipeDataLoader struct {
	// contains filtered or unexported fields
}

func (*PipeDataLoader) Abort

func (loader *PipeDataLoader) Abort(ctx *sql.Context) error

func (*PipeDataLoader) Finish

func (loader *PipeDataLoader) Finish(ctx *sql.Context) (*LoadDataResults, error)

func (*PipeDataLoader) LoadChunk

func (loader *PipeDataLoader) LoadChunk(ctx *sql.Context, data []byte) error

func (*PipeDataLoader) Start

func (loader *PipeDataLoader) Start() <-chan error

type PortalData

type PortalData struct {
	Statement         ConvertedStatement
	IsEmptyQuery      bool
	Fields            []pgproto3.FieldDescription
	ResultFormatCodes []int16
	Stmt              *duckdb.Stmt
	Vars              []any
	Closed            *atomic.Bool
}

type PreparedStatementData

type PreparedStatementData struct {
	Statement    ConvertedStatement
	ReturnFields []pgproto3.FieldDescription
	BindVarTypes []uint32
	Stmt         *duckdb.Stmt
	Closed       *atomic.Bool
}

type QueryExecutor

type QueryExecutor func(ctx *sql.Context, query string, parsed tree.Statement, stmt *duckdb.Stmt, vars []any) (sql.Schema, sql.RowIter, *sql.QueryFlags, error)

QueryExecutor is a function that executes a query and returns the result as a schema and iterator. Either of |parsed| or |analyzed| can be nil depending on the use case

type QueryMode

type QueryMode bool
const (
	SimpleQueryMode   QueryMode = false
	ExtendedQueryMode QueryMode = true
)

type QueryModifier

type QueryModifier func(string) string

func NewQueryRemover

func NewQueryRemover(regex string) QueryModifier

func NewQueryReplacer

func NewQueryReplacer(regex string, replacement string) QueryModifier

type ReadyForQueryTransactionIndicator

type ReadyForQueryTransactionIndicator byte

ReadyForQueryTransactionIndicator indicates the state of the transaction related to the query.

const (
	ReadyForQueryTransactionIndicator_Idle                   ReadyForQueryTransactionIndicator = 'I'
	ReadyForQueryTransactionIndicator_TransactionBlock       ReadyForQueryTransactionIndicator = 'T'
	ReadyForQueryTransactionIndicator_FailedTransactionBlock ReadyForQueryTransactionIndicator = 'E'
)

type RestoreConfig

type RestoreConfig struct {
	DbName        string
	RemoteFile    string
	StorageConfig *storage.ObjectStorageConfig
}

func NewRestoreConfig

func NewRestoreConfig(dbName, remoteUri, endpoint, accessKeyId, secretAccessKey string) (*RestoreConfig, error)

type Result

type Result struct {
	Fields       []pgproto3.FieldDescription `json:"fields"`
	Rows         []Row                       `json:"rows"`
	RowsAffected uint64                      `json:"rows_affected"`
}

Result represents a query result.

type Row

type Row struct {
	// contains filtered or unexported fields
}

Row represents a single row value in bytes format. |val| represents array of a single row elements, which each element value is in byte array format.

type SASLBindingFlag

type SASLBindingFlag string

SASLBindingFlag are the flags for gs2-cbind-flag, used in SASL authentication.

const (
	SASLBindingFlag_NoClientSupport        SASLBindingFlag = "n"
	SASLBindingFlag_AssumedNoServerSupport SASLBindingFlag = "y"
	SASLBindingFlag_Used                   SASLBindingFlag = "p"
)

type SASLContinue

type SASLContinue struct {
	Nonce      string
	Salt       string // Base64 encoded salt
	Iterations uint32
}

SASLContinue is the structured form of the output for *pgproto3.SASLInitialResponse.

func (SASLContinue) Encode

Encode returns the struct as an AuthenticationSASLContinue message.

type SASLInitial

type SASLInitial struct {
	Flag     SASLBindingFlag
	BindName string // Only set when Flag is SASLBindingFlag_Used
	Binding  string // Base64 encoding of cbind-input
	Authzid  string // Authorization ID, currently ignored in favor of the startup message's username
	Username string // Prepared using SASLprep, currently ignored in favor of the startup message's username
	Nonce    string
	RawData  []byte // The bytes that were received in the message
}

SASLInitial is the structured form of the input given by *pgproto3.SASLInitialResponse.

func (SASLInitial) Base64Header

func (si SASLInitial) Base64Header() string

Base64Header returns the base64-encoded GS2 header and channel binding data.

func (SASLInitial) MessageBare

func (si SASLInitial) MessageBare() []byte

MessageBare returns the message without the GS2 header.

type SASLResponse

type SASLResponse struct {
	GS2Header   string
	Nonce       string
	ClientProof string // Base64 encoded
	RawData     []byte // The bytes that were received in the message
}

SASLResponse is the structured form of the input given by *pgproto3.SASLResponse.

func (SASLResponse) MessageWithoutProof

func (sr SASLResponse) MessageWithoutProof() []byte

MessageWithoutProof returns the client-final-message-without-proof.

type Server

type Server struct {
	Listener       *Listener
	Provider       *catalog.DatabaseProvider
	ConnPool       *backend.ConnectionPool
	NewInternalCtx func() *sql.Context
}

func NewServer

func NewServer(provider *catalog.DatabaseProvider, connPool *backend.ConnectionPool, host string, port int, password string, newCtx func() *sql.Context, options ...ListenerOpt) (*Server, error)

func (*Server) Close

func (s *Server) Close()

func (*Server) Start

func (s *Server) Start()

type SqlRowIter

type SqlRowIter struct {
	// contains filtered or unexported fields
}

SqlRowIter wraps a standard sql.Rows as a RowIter.

func NewSqlRowIter

func NewSqlRowIter(rows *stdsql.Rows, schema sql.Schema) (*SqlRowIter, error)

func (*SqlRowIter) Close

func (iter *SqlRowIter) Close(ctx *sql.Context) error

Close closes the underlying sql.Rows.

func (*SqlRowIter) Next

func (iter *SqlRowIter) Next(ctx *sql.Context) (sql.Row, error)

Next retrieves the next row. It will return io.EOF if it's the last row.

func (*SqlRowIter) String

func (iter *SqlRowIter) String() string

type SubscriptionConfig

type SubscriptionConfig struct {
	SubscriptionName string
	PublicationName  string
	Connection       *ConnectionDetails // Embedded pointer to ConnectionDetails
	Action           Action
}

SubscriptionConfig represents the configuration of a subscription.

func (*SubscriptionConfig) ToConnectionInfo

func (config *SubscriptionConfig) ToConnectionInfo() string

ToConnectionInfo Format SubscriptionConfig into a ConnectionInfo

func (*SubscriptionConfig) ToDNS

func (config *SubscriptionConfig) ToDNS() string

ToDNS Format SubscriptionConfig into a DNS

Directories

Path Synopsis
Copied from github.com/dolthub/doltgresql/server/config/parameters.go
Copied from github.com/dolthub/doltgresql/server/config/parameters.go

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL