postgres

package
v1.37.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 14, 2024 License: Apache-2.0 Imports: 48 Imported by: 0

README

PostgreSQL Datastore

Minimum required version can be found here defined as MinimumSupportedPostgresVersion in version.go

PostgreSQL is a traditional relational database management system that is very popular. This datastore implementation allows you to use a PostgreSQL database as the backing durable storage for SpiceDB. Recommended usage: when you are comfortable with having all permissions data stored in a single region.

Configuration

track_commit_timestamp must be set to on for the Watch API to be enabled.

Implementation Caveats

While PostgreSQL uses MVCC to implement its ACID properties, it doesn't offer users the ability to read dirty data without adding an extension. For that reason, the PostgreSQL datastore driver implements a second layer of MVCC where we can manually control all writes to the database. This allows us to track all revisions of the database explicitly and perform point-in-time snapshot queries.

Documentation

Index

Constants

View Source 
const (
	Engine = "postgres"
)
View Source 
const MaxLegacyXIPDelta = 1000

MaxLegacyXIPDelta is the maximum allowed delta between the xmin and xmax revisions IDs on a *legacy* revision stored as a revision decimal. This is set to prevent a delta that is too large from blowing out the memory usage of the allocated slice, or even causing a panic in the case of a VERY large delta (which can be produced by, for example, a CRDB revision being given to a Postgres datastore accidentally).

Variables

This section is empty.

Functions

func DefaultQueryExecMode added in v1.35.0 

func DefaultQueryExecMode(poolConfig *pgxpool.Config) *pgxpool.Config

DefaultQueryExecMode parses a Postgres URI and determines if a default_query_exec_mode has been specified. If not, it defaults to "exec". SpiceDB queries have high variability of arguments and rarely benefit from using prepared statements. The default and recommended query exec mode is 'exec', which has shown the best performance under various synthetic workloads. See more in https://spicedb.dev/d/query-exec-mode.

The docs for the different execution modes offered by pgx may be found here: https://pkg.go.dev/github.com/jackc/pgx/v5#QueryExecMode

func NewPostgresDatastore 

func NewPostgresDatastore(
	ctx context.Context,
	url string,
	options ...Option,
) (datastore.Datastore, error)

NewPostgresDatastore initializes a SpiceDB datastore that uses a PostgreSQL database by leveraging manual book-keeping to implement revisioning.

This datastore is also tested to be compatible with CockroachDB.

func NewReadOnlyPostgresDatastore added in v1.35.0 

func NewReadOnlyPostgresDatastore(
	ctx context.Context,
	url string,
	index uint32,
	options ...Option,
) (datastore.StrictReadDatastore, error)

NewReadOnlyPostgresDatastore initializes a SpiceDB datastore that uses a PostgreSQL database by leveraging manual book-keeping to implement revisioning. This version is read only and does not allow for write transactions.

func ParseRevisionString added in v1.27.0 

func ParseRevisionString(revisionStr string) (rev datastore.Revision, err error)

ParseRevisionString parses a revision string into a Postgres revision.

func RegisterTypes added in v1.19.0 

func RegisterTypes(m *pgtype.Map)

RegisterTypes registers pgSnapshot and xid8 with a pgtype.ConnInfo.

Types

type Option added in v1.0.0 

type Option func(*postgresOptions)

Option provides the facility to configure how clients within the Postgres datastore interact with the running Postgres database.

func CredentialsProviderName added in v1.32.0 

func CredentialsProviderName(credentialsProviderName string) Option

CredentialsProviderName is the name of the CredentialsProvider implementation to use for dynamically retrieving the datastore credentials at runtime

Empty by default.

func DebugAnalyzeBeforeStatistics added in v1.6.0 

func DebugAnalyzeBeforeStatistics() Option

DebugAnalyzeBeforeStatistics signals to the Statistics method that it should run Analyze on the database before returning statistics. This should only be used for debug and testing.

Disabled by default.

func EnableTracing 

func EnableTracing() Option

EnableTracing enables trace-level logging for the Postgres clients being used by the datastore.

Tracing is disabled by default.

func FilterMaximumIDCount added in v1.35.0 

func FilterMaximumIDCount(filterMaximumIDCount uint16) Option

FilterMaximumIDCount is the maximum number of IDs that can be used to filter IDs in queries

func GCEnabled added in v1.13.0 

func GCEnabled(isGCEnabled bool) Option

GCEnabled indicates whether garbage collection is enabled.

GC is enabled by default.

func GCInterval added in v1.1.0 

func GCInterval(interval time.Duration) Option

GCInterval is the the interval at which garbage collection will occur.

This value defaults to 3 minutes.

func GCMaxOperationTime added in v1.1.0 

func GCMaxOperationTime(time time.Duration) Option

GCMaxOperationTime is the maximum operation time of a garbage collection pass before it times out.

This value defaults to 1 minute.

func GCWindow 

func GCWindow(window time.Duration) Option

GCWindow is the maximum age of a passed revision that will be considered valid.

This value defaults to 24 hours.

func MaxRetries added in v1.8.0 

func MaxRetries(maxRetries uint8) Option

MaxRetries is the maximum number of times a retriable transaction will be client-side retried. Default: 10

func MaxRevisionStalenessPercent added in v1.7.0 

func MaxRevisionStalenessPercent(stalenessPercent float64) Option

MaxRevisionStalenessPercent is the amount of time, expressed as a percentage of the revision quantization window, that a previously computed rounded revision can still be advertised after the next rounded revision would otherwise be ready.

This value defaults to 0.1 (10%).

func MigrationPhase added in v1.14.0 

func MigrationPhase(phase string) Option

MigrationPhase configures the postgres driver to the proper state of a multi-phase migration.

Steady-state configuration (e.g. fully migrated) by default

func ReadConnHealthCheckInterval added in v1.18.0 

func ReadConnHealthCheckInterval(interval time.Duration) Option

ReadConnHealthCheckInterval is the frequency at which both idle and max lifetime connections are checked, and also the frequency at which the minimum number of connections is checked.

This happens asynchronously.

This is not the only approach to evaluate these counts; "connection idle/max lifetime" is also checked when connections are released to the pool.

There is no guarantee connections won't last longer than their specified idle/max lifetime. It's largely dependent on the health-check goroutine being able to pull them from the connection pool.

The health-check may not be able to clean up those connections if they are held by the application very frequently.

This value defaults to 30s.

func ReadConnMaxIdleTime added in v1.18.0 

func ReadConnMaxIdleTime(idle time.Duration) Option

ReadConnMaxIdleTime is the duration after which an idle read connection will be automatically closed by the health check.

This value defaults to having no maximum.

func ReadConnMaxLifetime added in v1.18.0 

func ReadConnMaxLifetime(lifetime time.Duration) Option

ReadConnMaxLifetime is the duration since creation after which a read connection will be automatically closed.

This value defaults to having no maximum.

func ReadConnMaxLifetimeJitter added in v1.19.0 

func ReadConnMaxLifetimeJitter(jitter time.Duration) Option

ReadConnMaxLifetimeJitter is an interval to wait up to after the max lifetime to close the connection.

This value defaults to 20% of the max lifetime.

func ReadConnsMaxOpen added in v1.18.0 

func ReadConnsMaxOpen(conns int) Option

ReadConnsMaxOpen is the maximum size of the connection pool used for reads.

This value defaults to having no maximum.

func ReadConnsMinOpen added in v1.18.0 

func ReadConnsMinOpen(conns int) Option

ReadConnsMinOpen is the minimum size of the connection pool used for reads.

The health check will increase the number of connections to this amount if it had dropped below.

This value defaults to the maximum open connections.

func ReadStrictMode added in v1.35.0 

func ReadStrictMode(readStrictMode bool) Option

ReadStrictMode sets whether strict mode is used for reads in the Postgres reader. If enabled, an assertion is added into the WHERE clause of all read queries to ensure that the revision being read is available on the read connection.

Strict mode is disabled by default, as the default behavior is to read from the primary.

func RevisionQuantization added in v1.7.0 

func RevisionQuantization(quantization time.Duration) Option

RevisionQuantization is the time bucket size to which advertised revisions will be rounded.

This value defaults to 5 seconds.

func WatchBufferLength 

func WatchBufferLength(watchBufferLength uint16) Option

WatchBufferLength is the number of entries that can be stored in the watch buffer while awaiting read by the client.

This value defaults to 128.

func WatchBufferWriteTimeout added in v1.29.0 

func WatchBufferWriteTimeout(watchBufferWriteTimeout time.Duration) Option

WatchBufferWriteTimeout is the maximum timeout for writing to the watch buffer, after which the caller to the watch will be disconnected.

func WithEnablePrometheusStats added in v1.7.0 

func WithEnablePrometheusStats(enablePrometheusStats bool) Option

WithEnablePrometheusStats marks whether Prometheus metrics provided by the Postgres clients being used by the datastore are enabled.

Prometheus metrics are disabled by default.

func WithQueryInterceptor added in v1.18.1 

func WithQueryInterceptor(interceptor pgxcommon.QueryInterceptor) Option

WithQueryInterceptor adds an interceptor to all underlying postgres queries

By default, no query interceptor is used.

func WriteConnHealthCheckInterval added in v1.18.0 

func WriteConnHealthCheckInterval(interval time.Duration) Option

WriteConnHealthCheckInterval is the frequency at which both idle and max lifetime connections are checked, and also the frequency at which the minimum number of connections is checked.

This happens asynchronously.

This is not the only approach to evaluate these counts; "connection idle/max lifetime" is also checked when connections are released to the pool.

There is no guarantee connections won't last longer than their specified idle/max lifetime. It's largely dependent on the health-check goroutine being able to pull them from the connection pool.

The health-check may not be able to clean up those connections if they are held by the application very frequently.

This value defaults to 30s.

func WriteConnMaxIdleTime added in v1.18.0 

func WriteConnMaxIdleTime(idle time.Duration) Option

WriteConnMaxIdleTime is the duration after which an idle write connection will be automatically closed by the health check.

This value defaults to having no maximum.

func WriteConnMaxLifetime added in v1.18.0 

func WriteConnMaxLifetime(lifetime time.Duration) Option

WriteConnMaxLifetime is the duration since creation after which a write connection will be automatically closed.

This value defaults to having no maximum.

func WriteConnMaxLifetimeJitter added in v1.19.0 

func WriteConnMaxLifetimeJitter(jitter time.Duration) Option

WriteConnMaxLifetimeJitter is an interval to wait up to after the max lifetime to close the connection.

This value defaults to 20% of the max lifetime.

func WriteConnsMaxOpen added in v1.18.0 

func WriteConnsMaxOpen(conns int) Option

WriteConnsMaxOpen is the maximum size of the connection pool used for writes.

This value defaults to having no maximum.

func WriteConnsMinOpen added in v1.18.0 

func WriteConnsMinOpen(conns int) Option

WriteConnsMinOpen is the minimum size of the connection pool used for writes.

The health check will increase the number of connections to this amount if it had dropped below.

This value defaults to the maximum open connections.

type SnapshotCodec added in v1.19.0 

type SnapshotCodec struct {
	pgtype.TextCodec
}

func (SnapshotCodec) DecodeValue added in v1.19.0 

func (SnapshotCodec) DecodeValue(tm *pgtype.Map, oid uint32, format int16, src []byte) (interface{}, error)

type Uint64Codec added in v1.19.0 

type Uint64Codec struct{}

func (Uint64Codec) DecodeDatabaseSQLValue added in v1.19.0 

func (c Uint64Codec) DecodeDatabaseSQLValue(m *pgtype.Map, oid uint32, format int16, src []byte) (driver.Value, error)

func (Uint64Codec) DecodeValue added in v1.19.0 

func (c Uint64Codec) DecodeValue(m *pgtype.Map, oid uint32, format int16, src []byte) (any, error)

func (Uint64Codec) FormatSupported added in v1.19.0 

func (Uint64Codec) FormatSupported(format int16) bool

func (Uint64Codec) PlanEncode added in v1.19.0 

func (Uint64Codec) PlanEncode(_ *pgtype.Map, _ uint32, format int16, value any) pgtype.EncodePlan

func (Uint64Codec) PlanScan added in v1.19.0 

func (Uint64Codec) PlanScan(_ *pgtype.Map, _ uint32, format int16, target any) pgtype.ScanPlan

func (Uint64Codec) PreferredFormat added in v1.19.0 

func (Uint64Codec) PreferredFormat() int16

type Uint64Scanner added in v1.19.0 

type Uint64Scanner interface {
	ScanUint64(v xid8) error
}

Adapted from https://github.com/jackc/pgx/blob/ca022267dbbfe7a8ba7070557352a5cd08f6cb37/pgtype/uint32.go

type Uint64Valuer added in v1.19.0 

type Uint64Valuer interface {
	Uint64Value() (xid8, error)
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.