postgres

package
v1.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 7, 2022 License: Apache-2.0 Imports: 55 Imported by: 0

Documentation

Overview

Package postgres contains the function about starting up, shutting down and managing a PostgreSQL instance. This functions are primarily used by PGK

Index

Constants

View Source 
const (
	// ShutdownModeSmart waits for all active clients to disconnect and any online backup to finish.
	// If the server is in hot standby, recovery and streaming replication will be terminated once
	// all clients have disconnected.
	ShutdownModeSmart = "smart"

	// ShutdownModeFast does not wait for clients to disconnect and will terminate an online
	// backup in progress.
	ShutdownModeFast = "fast"

	// ShutdownModeImmediate aborts all server processes immediately, without a clean shutdown.
	ShutdownModeImmediate = "immediate"
)
View Source 
const PostgresqlPidFile = "postmaster.pid" //wokeignore:rule=master

PostgresqlPidFile is the name of the file which contains the PostgreSQL PID file

Variables

View Source 
var (
	// ErrPgRejectingConnection postgres is alive, but rejecting connections
	ErrPgRejectingConnection = fmt.Errorf("server is alive but rejecting connections")

	// ErrNoConnectionEstablished postgres is alive, but rejecting connections
	ErrNoConnectionEstablished = fmt.Errorf("could not establish connection")
)
View Source 
var (
	// ErrInstanceInRecovery is raised while PostgreSQL is still in recovery mode
	ErrInstanceInRecovery = fmt.Errorf("instance in recovery")

	// RetryUntilRecoveryDone is the default retry configuration that is used
	// to wait for a restored cluster to promote itself
	RetryUntilRecoveryDone = wait.Backoff{
		Duration: 5 * time.Second,

		Steps: math.MaxInt32,
	}
)
View Source 
var DefaultShutdownOptions = ShutdownOptions{
	Mode:    ShutdownModeFast,
	Wait:    true,
	Timeout: nil,
}

DefaultShutdownOptions are the default shutdown options. That is: 1. use the "fast" mode 2. wait for the operation to succeed 3. without setting any explicit timeout (defaulted by PostgreSQL to 60 seconds)

View Source 
var RetryUntilServerAvailable = wait.Backoff{
	Duration: 5 * time.Second,

	Steps: math.MaxInt32,
}

RetryUntilServerAvailable is the default retry configuration that is used to wait for a successful connection to a certain server

Functions

func ClonePgData 

func ClonePgData(connectionString, targetPgData string) error

ClonePgData clones an existing server, given its connection string, to a certain data directory

func GetEnforcedParametersThroughPgControldata 

func GetEnforcedParametersThroughPgControldata(pgData string) (map[string]string, error)

GetEnforcedParametersThroughPgControldata will parse the output of pg_controldata in order to get the values of all the hot standby sensible parameters

func GetReadyWALFiles 

func GetReadyWALFiles() (fileNames []string, err error)

GetReadyWALFiles returns an array containing the list of all the WAL files that are marked as ready to be archived.

func GetServerPort 

func GetServerPort() int

GetServerPort gets the port where the postmaster will be listening using the environment variable or, when empty, the default one

func GetSocketDir 

func GetSocketDir() string

GetSocketDir gets the name of the directory that will contain the Unix socket for the PostgreSQL server. This is detected using the PGHOST environment variable or using a default

func GetWALArchiveCounters 

func GetWALArchiveCounters() (ready, done int, err error)

GetWALArchiveCounters returns the number of WAL files with status ready, and the number of those in status done.

func InstallPgDataFileContent 

func InstallPgDataFileContent(pgdata, contents, destinationFile string) (bool, error)

InstallPgDataFileContent installs a file in PgData, returning true/false if the file has been changed and an error state

func RemoveArchiveModeFromPostgresAutoConf 

func RemoveArchiveModeFromPostgresAutoConf(pgData string) (changed bool, err error)

RemoveArchiveModeFromPostgresAutoConf removes the "archive_mode" option from "postgresql.auto.conf"

func UpdateBackupStatusAndRetry 

func UpdateBackupStatusAndRetry(
	ctx context.Context,
	cli client.Client,
	backup *apiv1.Backup,
) error

UpdateBackupStatusAndRetry updates a certain backup's status in the k8s database, retries when error occurs

func UpdateReplicaConfiguration 

func UpdateReplicaConfiguration(pgData string, clusterName string, podName string) (changed bool, err error)

UpdateReplicaConfiguration updates the postgresql.auto.conf or recovery.conf file for the proper version of PostgreSQL

func UpdateReplicaConfigurationForPrimary 

func UpdateReplicaConfigurationForPrimary(pgData string, primaryConnInfo string) (changed bool, err error)

UpdateReplicaConfigurationForPrimary updates the postgresql.auto.conf or recovery.conf file for the proper version of PostgreSQL, using the specified connection string to connect to the primary server

func WritePostgresUserMaps 

func WritePostgresUserMaps(pgData string) error

WritePostgresUserMaps creates a pg_ident.conf file containing only one map called "local" that maps the current user to "postgres" user.

Types

type BackupCommand 

type BackupCommand struct {
	Cluster  *apiv1.Cluster
	Backup   *apiv1.Backup
	Client   client.Client
	Recorder record.EventRecorder
	Env      []string
	Log      log.Logger
	Instance *Instance
}

BackupCommand represent a backup command that is being executed

func NewBackupCommand 

func NewBackupCommand(
	cluster *apiv1.Cluster,
	backup *apiv1.Backup,
	client client.Client,
	recorder record.EventRecorder,
	instance *Instance,
	log log.Logger,
) *BackupCommand

NewBackupCommand initializes a BackupCommand object

func (*BackupCommand) Start 

func (b *BackupCommand) Start(ctx context.Context) error

Start initiates a backup for this instance using barman-cloud-backup

type InitInfo 

type InitInfo struct {
	// The data directory where to generate the new cluster
	PgData string

	// The name of the database to be generated for the applications
	ApplicationDatabase string

	// The name of the role to be generated for the applications
	ApplicationUser string

	// The parent node, used to fill primary_conninfo
	ParentNode string

	// The current node, used to fill application_name
	PodName string

	// The cluster name to assign to
	ClusterName string

	// The namespace where the cluster will be installed
	Namespace string

	// The list options that should be passed to initdb to
	// create the cluster
	InitDBOptions []string

	// The list of queries to be executed just after having
	// configured a new instance
	PostInitSQL []string

	// The list of queries to be executed just after having
	// the application database created
	PostInitApplicationSQL []string

	// The list of queries to be executed inside the template1
	// database just after having configured a new instance
	PostInitTemplateSQL []string

	// Whether it is a temporary instance that will never contain real data.
	Temporary bool
}

InitInfo contains all the info needed to bootstrap a new PostgreSQL instance

func (InitInfo) Bootstrap 

func (info InitInfo) Bootstrap(ctx context.Context) error

Bootstrap creates and configures this new PostgreSQL instance

func (InitInfo) ConfigureInstanceAfterRestore 

func (info InitInfo) ConfigureInstanceAfterRestore(env []string) error

ConfigureInstanceAfterRestore changes the superuser password of the instance to be coherent with the one specified in the cluster. This function also ensures that we can really connect to this cluster using the password in the secrets

func (InitInfo) ConfigureNewInstance 

func (info InitInfo) ConfigureNewInstance(instance *Instance) error

ConfigureNewInstance creates the expected users and databases in a new PostgreSQL instance. If any error occurs, we return it

func (InitInfo) CreateDataDirectory 

func (info InitInfo) CreateDataDirectory() error

CreateDataDirectory creates a new data directory given the configuration

func (InitInfo) GetInstance 

func (info InitInfo) GetInstance() *Instance

GetInstance gets the PostgreSQL instance which correspond to these init information

func (InitInfo) Join 

func (info InitInfo) Join() error

Join creates a new instance joined to an existing PostgreSQL cluster

func (InitInfo) Restore 

func (info InitInfo) Restore(ctx context.Context) error

Restore restores a PostgreSQL cluster from a backup into the object storage

func (InitInfo) VerifyPGData 

func (info InitInfo) VerifyPGData() error

VerifyPGData verifies if the passed configuration is OK, otherwise it returns an error

func (InitInfo) WriteInitialPostgresqlConf 

func (info InitInfo) WriteInitialPostgresqlConf(cluster *apiv1.Cluster) error

WriteInitialPostgresqlConf resets the postgresql.conf that there is in the instance using a new bootstrapped instance as reference

func (InitInfo) WriteRestoreHbaConf 

func (info InitInfo) WriteRestoreHbaConf() error

WriteRestoreHbaConf writes a pg_hba.conf allowing access without password from localhost. this is needed to set the PostgreSQL password after the postgres server is started and active

type Instance 

type Instance struct {
	// The data directory
	PgData string

	// The socket directory
	SocketDirectory string

	// The environment variables that will be used to start the instance
	Env []string

	// Command line options to pass to the postgres process, see the
	// '-c' option of pg_ctl for an useful example
	StartupOptions []string

	// The namespace of the k8s object representing this cluster
	Namespace string

	// The name of the Pod where the controller is executing
	PodName string

	// The name of the cluster of which this Pod is belonging
	ClusterName string

	// The sha256 of the config. It is computed on the config string, before
	// adding the PostgreSQL CNPGConfigSha256 parameter
	ConfigSha256 string

	// PgCtlTimeoutForPromotion specifies the maximum number of seconds to wait when waiting for promotion to complete
	PgCtlTimeoutForPromotion int32

	// specifies the maximum number of seconds to wait when shutting down for a switchover
	MaxSwitchoverDelay int32

	// InstanceManagerIsUpgrading tells if there is an instance manager upgrade in process
	InstanceManagerIsUpgrading bool

	// PgRewindIsRunning tells if there is a `pg_rewind` process running
	PgRewindIsRunning bool

	// MaxStopDelay is the current MaxStopDelay of the cluster
	MaxStopDelay int32
	// contains filtered or unexported fields
}

Instance represent a PostgreSQL instance to be executed in the current environment

func NewInstance 

func NewInstance() *Instance

NewInstance creates a new Instance object setting the defaults

func (*Instance) CanCheckReadiness 

func (instance *Instance) CanCheckReadiness() bool

CanCheckReadiness checks whether the instance should be checked for readiness

func (*Instance) CheckForExistingPostmaster 

func (instance *Instance) CheckForExistingPostmaster(postgresExecutables ...string) (*os.Process, error)

CheckForExistingPostmaster checks if a postmaster process is running on the PGDATA volume. If it is, it returns its process entry.

To do that, this function will read the PID file from the data directory and check the existence of the relative process. If the process exists, then that process entry is returned. If it doesn't exist then the PID file is stale and is removed.

We are requiring a list of PostgreSQL executables that is used to check if the process with the ID read from the PID file is really a postmaster. This isn't really needed per se, but allows this function to be easier to test.

func (*Instance) CleanUpStalePid 

func (instance *Instance) CleanUpStalePid() error

CleanUpStalePid cleans up the files left around by a crashed PostgreSQL instance. It removes the default PostgreSQL pid file and the content of the socket directory.

func (*Instance) CompleteCrashRecovery 

func (instance *Instance) CompleteCrashRecovery() error

CompleteCrashRecovery temporary starts up the server and wait for it to be fully available for queries. This will ensure that the crash recovery is fully done. Important: this function must be called only when the instance isn't started

func (*Instance) ConnectionPool 

func (instance *Instance) ConnectionPool() *pool.ConnectionPool

ConnectionPool gets or initializes the connection pool for this instance

func (*Instance) Demote 

func (instance *Instance) Demote() error

Demote demote an existing PostgreSQL instance

func (*Instance) GeneratePostgresqlHBA 

func (instance *Instance) GeneratePostgresqlHBA(cluster *apiv1.Cluster, ldapBindPassword string) (string, error)

GeneratePostgresqlHBA generates the pg_hba.conf content with the LDAP configuration if configured.

func (*Instance) GetDecreasedSensibleSettings 

func (instance *Instance) GetDecreasedSensibleSettings(superUserDB *sql.DB) (map[string]string, error)

GetDecreasedSensibleSettings tries to get all decreased hot standby sensible parameters from the instance. Returns a map containing all the decreased hot standby sensible parameters with their new value. See https://www.postgresql.org/docs/current/hot-standby.html#HOT-STANDBY-ADMIN for more details.

func (*Instance) GetInstanceCommandChan 

func (instance *Instance) GetInstanceCommandChan() <-chan InstanceCommand

GetInstanceCommandChan is the channel where the lifecycle manager will wait for the operations requested on the instance

func (*Instance) GetPgVersion 

func (instance *Instance) GetPgVersion() (semver.Version, error)

GetPgVersion queries the postgres instance to know the current version, parses it and memoize it for future uses

func (*Instance) GetPostmasterPidFromFile added in v1.15.1 

func (instance *Instance) GetPostmasterPidFromFile(pidFile string) ([]byte, int, error)

GetPostmasterPidFromFile reads the given postmaster pid file, parse it and return its content and the actual pid

func (*Instance) GetStatus 

func (instance *Instance) GetStatus() (result *postgres.PostgresqlStatus, err error)

GetStatus Extract the status of this PostgreSQL database

func (*Instance) GetSuperUserDB 

func (instance *Instance) GetSuperUserDB() (*sql.DB, error)

GetSuperUserDB gets a connection to the "postgres" database on this instance

func (*Instance) GetTemplateDB 

func (instance *Instance) GetTemplateDB() (*sql.DB, error)

GetTemplateDB gets a connection to the "template1" database on this instance

func (*Instance) IsFenced 

func (instance *Instance) IsFenced() bool

IsFenced checks whether the instance is marked as fenced

func (*Instance) IsPrimary 

func (instance *Instance) IsPrimary() (bool, error)

IsPrimary check if the data directory belongs to a primary server or to a secondary one by looking for a "standby.signal" file inside the data directory. IMPORTANT: this method also works when the instance is not started up

func (*Instance) IsServerHealthy 

func (instance *Instance) IsServerHealthy() error

IsServerHealthy check if the instance is healthy

func (*Instance) IsServerReady 

func (instance *Instance) IsServerReady() error

IsServerReady check if the instance is healthy and can really accept connections

func (*Instance) IsWALReceiverActive 

func (instance *Instance) IsWALReceiverActive() (bool, error)

IsWALReceiverActive check if the WAL receiver process is active by looking at the number of records in the `pg_stat_wal_receiver` table

func (*Instance) LogPgControldata 

func (instance *Instance) LogPgControldata(reason string)

LogPgControldata logs the content of PostgreSQL control data, for debugging and tracing

func (*Instance) MightBeUnavailable 

func (instance *Instance) MightBeUnavailable() bool

MightBeUnavailable checks whether we expect the instance to be down

func (*Instance) PgIsReady 

func (instance *Instance) PgIsReady() error

PgIsReady gets the status from the pg_isready command

func (*Instance) PromoteAndWait 

func (instance *Instance) PromoteAndWait() error

PromoteAndWait promotes this instance, and wait DefaultPgCtlTimeoutForPromotion seconds for it to happen

func (*Instance) RefreshConfigurationFilesFromCluster 

func (instance *Instance) RefreshConfigurationFilesFromCluster(
	cluster *apiv1.Cluster,
) (bool, error)

RefreshConfigurationFilesFromCluster receives a cluster object, then generates the PostgreSQL configuration and rewrites the file in the PGDATA if needed. This function will return "true" if the configuration has been really changed.

func (*Instance) RefreshPGHBA 

func (instance *Instance) RefreshPGHBA(cluster *apiv1.Cluster, ldapBindPassword string) (
	postgresHBAChanged bool,
	err error,
)

RefreshPGHBA generates and writes down the pg_hba.conf file

func (*Instance) Reload 

func (instance *Instance) Reload() error

Reload makes a certain active instance reload the configuration

func (*Instance) RequestAndWaitFencingOff 

func (instance *Instance) RequestAndWaitFencingOff() error

RequestAndWaitFencingOff will request to remove the fencing and wait for the instance to be restarted

func (*Instance) RequestAndWaitRestartSmartFast 

func (instance *Instance) RequestAndWaitRestartSmartFast() error

RequestAndWaitRestartSmartFast requests the lifecycle manager to restart the postmaster, and wait for the postmaster to be restarted

func (*Instance) RequestFastImmediateShutdown 

func (instance *Instance) RequestFastImmediateShutdown()

RequestFastImmediateShutdown request the lifecycle manager to shut down PostegreSQL using the fast strategy and then the immediate strategy.

func (*Instance) RequestFencingOn 

func (instance *Instance) RequestFencingOn()

RequestFencingOn request the lifecycle manager to shut down postgres and enable fencing

func (*Instance) Rewind 

func (instance *Instance) Rewind(postgresMajorVersion int) error

Rewind uses pg_rewind to align this data directory with the contents of the primary node. If postgres major version is >= 13, add "--restore-target-wal" option

func (Instance) Run 

func (instance Instance) Run() (*execlog.StreamingCmd, error)

Run this instance returning an OS process needed to control the instance execution

func (*Instance) SetCanCheckReadiness 

func (instance *Instance) SetCanCheckReadiness(enabled bool)

SetCanCheckReadiness marks whether the instance should be checked for readiness

func (*Instance) SetFencing 

func (instance *Instance) SetFencing(enabled bool)

SetFencing marks whether the instance is fenced, if enabling, marks also any down to be tolerated

func (*Instance) SetMightBeUnavailable 

func (instance *Instance) SetMightBeUnavailable(enabled bool)

SetMightBeUnavailable marks whether the instance being down should be tolerated

func (*Instance) Shutdown 

func (instance *Instance) Shutdown(options ShutdownOptions) error

Shutdown shuts down a PostgreSQL instance which was previously started with Startup. This function will return an error whether PostgreSQL is still up after the shutdown request.

func (*Instance) ShutdownConnections 

func (instance *Instance) ShutdownConnections()

ShutdownConnections tears down database connections

func (*Instance) Startup 

func (instance *Instance) Startup() error

Startup starts up a PostgreSQL instance and wait for the instance to be started

func (*Instance) TryGetPgStatWAL added in v1.15.1 

func (instance *Instance) TryGetPgStatWAL() (*PgStatWal, error)

TryGetPgStatWAL retrieves pg_wal_stat on pg version 14 and further

func (*Instance) WaitForConfigReloaded 

func (instance *Instance) WaitForConfigReloaded() error

WaitForConfigReloaded waits until the config has been reloaded

func (*Instance) WaitForPrimaryAvailable 

func (instance *Instance) WaitForPrimaryAvailable() error

WaitForPrimaryAvailable waits until we can connect to the primary

func (*Instance) WaitForSuperuserConnectionAvailable 

func (instance *Instance) WaitForSuperuserConnectionAvailable() error

WaitForSuperuserConnectionAvailable waits until we can connect to this instance using the superuser account

func (Instance) WithActiveInstance 

func (instance Instance) WithActiveInstance(inner func() error) error

WithActiveInstance execute the internal function while this PostgreSQL instance is running

type InstanceCommand 

type InstanceCommand string

InstanceCommand are commands for the goroutine managing postgres 

const (
	// RestartSmartFast means the instance has to be restarted by first issuing
	// a smart shutdown and in case it doesn't work, a fast shutdown
	RestartSmartFast InstanceCommand = "RestartSmartFast"

	// FenceOn means the instance has to be restarted by first issuing
	// a smart shutdown and in case it doesn't work, a fast shutdown
	FenceOn InstanceCommand = "FenceOn"

	// FenceOff means the instance has to be restarted by first issuing
	// a smart shutdown and in case it doesn't work, a fast shutdown
	FenceOff InstanceCommand = "FenceOff"

	// ShutDownFastImmediate means the instance has to be shut down by first
	// issuing a fast shut down and in case of errors an immediate one
	ShutDownFastImmediate InstanceCommand = "ShutDownFastImmediate"
)

type PgStatWal added in v1.15.1 

type PgStatWal struct {
	WalRecords     int64
	WalFpi         int64
	WalBytes       int64
	WALBuffersFull int64
	WalWrite       int64
	WalSync        int64
	WalWriteTime   int64
	WalSyncTime    int64
	StatsReset     string
}

PgStatWal is a representation of the pg_stat_wal table

type ShutdownMode 

type ShutdownMode string

ShutdownMode represent a way to request the postmaster shutdown

type ShutdownOptions 

type ShutdownOptions struct {
	// Mode is the method we require for the shutdown
	Mode ShutdownMode

	// Wait is true whether we want to wait for the shutdown to complete
	Wait bool

	// Timeout is the maximum number of seconds to wait for the shutdown to complete
	// Used only if Wait is true. Defaulted by PostgreSQL to 60 seconds.
	Timeout *int32
}

ShutdownOptions is the configuration of a shutdown request to PostgreSQL

Source Files

View all Source files

Directories

Path Synopsis
Package constants provides the needed constants in the postgres package
 Package constants provides the needed constants in the postgres package
Package logicalimport contains the logic needed to import a logical snapshot
 Package logicalimport contains the logic needed to import a logical snapshot
Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.
 Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.
Package metrics enables to expose a set of metrics and collectors on a given postgres instance
 Package metrics enables to expose a set of metrics and collectors on a given postgres instance
histogram
Package histogram contain histogram-metrics related functions
 Package histogram contain histogram-metrics related functions
Package pool contain an implementation of a connection pool to multiple database pointing to the same instance
 Package pool contain an implementation of a connection pool to multiple database pointing to the same instance
Package utils holds generic utils about postgres instances
 Package utils holds generic utils about postgres instances
Package webserver contains the web server powering probes, backups and metrics
 Package webserver contains the web server powering probes, backups and metrics
metricserver
Package metricserver contains the web server powering metrics
 Package metricserver contains the web server powering metrics

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.