README
¶
Welcome to the CloudNativePG project!
CloudNativePG is an open source operator designed to manage PostgreSQL workloads on any supported Kubernetes cluster running in private, public, hybrid, or multi-cloud environments.
CloudNativePG was originally built and sponsored by EDB.
Table of content
Getting Started
The best way to get started is with the "Quickstart" section in the documentation.
Scope
The goal of CloudNativePG is to increase the adoption of PostgreSQL, one of the most loved DBMS in traditional VM and bare metal environments, inside Kubernetes, thus making the database an integral part of the development process and GitOps CI/CD automated pipelines.
In scope
CloudNativePG has been designed by Postgres experts with Kubernetes administrators in mind. Put simply, it leverages Kubernetes by extending its controller and by defining, in a programmatic way, all the actions that a good DBA would normally do when managing a highly available PostgreSQL database cluster.
Since the inception, our philosophy has been to adopt a Kubernetes native approach to PostgreSQL cluster management, making incremental decisions that would answer the fundamental question: "What would a Kubernetes user expect from a Postgres operator?".
The most important decision we made is to have the status of a PostgreSQL
cluster directly available in the Cluster resource, so to inspect it through
the Kubernetes API. We've fully embraced the operator pattern and eventual
consistency, two of the core principles upon which Kubernetes is built for
managing complex applications.
As a result, the operator is responsible for managing the status of the
Cluster resource, keeping it up to date with the information that each
PostgreSQL instance manager regularly reports back through the API server. Such
changes might trigger, for example, actions like:
-
a PostgreSQL failover where, after an unexpected failure of a cluster's primary instance, the operator itself elects the new primary, updates the status, and directly coordinates the operation through the reconciliation loop, by relying on the instance managers
-
scaling up or down the number of read-only replicas, based on a positive or negative variation in the number of desired instances in the cluster, so that the operator creates or removes the required resources to run PostgreSQL, such as persistent volumes, persistent volume claims, pods, secrets, config maps, and then coordinates cloning and streaming replication tasks
-
updates of the endpoints of the PostgreSQL services that applications rely on to interact with the database, as Kubernetes represents the single source of truth and authority
-
updates of container images in a rolling fashion, following a change in the image name, by first updating the pods where replicas are running, and then the primary, issuing a switchover first
The latter example is based on another pillar of CloudNativePG: immutable application containers - as explained in the blog article "Why EDB Chose Immutable Application Containers".
The above list can be extended. However, the gist is that CloudNativePG exclusively relies on the Kubernetes API server and the instance manager to coordinate the complex operations that need to take place in a business continuity PostgreSQL cluster, without requiring any assistance from an intermediate management tool responsible for high availability and failover management like similar open source operators.
CloudNativePG also manages additional resources to help the Cluster resource
manage PostgreSQL - currently ScheduledBackup, Pooler, and Backup.
Fully embracing Kubernetes also means that, in case of failure of the whole Kubernetes cluster, the operator won’t do anything, postponing any decision to when the cluster is back up again. In the meantime, Postgres instances should continue running according to the last known state of the cluster.
Out of scope
CloudNativePG is exclusively focused on the PostgreSQL database management system maintained by the PostgreSQL Global Development Group (PGDG). We are not currently considering adding to CloudNativePG extensions or capabilities that are included in forks of the PostgreSQL database management system, unless in the form of extensible or pluggable frameworks.
CloudNativePG doesn't intend to pursue database independence (e.g. control a MariaDB cluster).
Communications
Resources
Useful links
- Data on Kubernetes (DoK) Community
- "Recommended Architectures for PostgreSQL in Kubernetes" by Gabriele Bartolini
- "How CloudNativePG manages physical replication slots for PostgreSQL in Kubernetes" by Gabriele Bartolini (September 2023)
- "The Current State of Major PostgreSQL Upgrades with CloudNativePG" by Gabriele Bartolini (August 2023)
- "The Rise of the Kubernetes Native Database" by Jeff Carpenter (December 2022)
- "Why Run Postgres in Kubernetes?" by Gabriele Bartolini (May 2022)
- "Shift-Left Security: The Path To PostgreSQL On Kubernetes" by Gabriele Bartolini (April 2021)
- "Local Persistent Volumes and PostgreSQL usage in Kubernetes" by Gabriele Bartolini (June 2020)
Maintainers
The current maintainers of the CloudNativePG project are:
- Gabriele Bartolini (EDB)
- Francesco Canovai (EDB)
- Leonardo Cecchi (EDB)
- Jonathan Gonzalez (EDB)
- Marco Nenciarini (EDB)
- Armando Ruocco (EDB)
- Philippe Scorsolini (upbound)
They are listed in the CODEOWNERS file.
Adopters
A list of publicly known users of the CloudNativePG operator is in ADOPTERS.md. Help us grow our community and CloudNativePG by adding yourself and your organization to this list!
Star History
Trademarks
Postgres, PostgreSQL and the Slonik Logo are trademarks or registered trademarks of the PostgreSQL Community Association of Canada, and used with their permission.
Directories
¶
| Path | Synopsis |
|---|---|
|
api
|
|
|
v1
Package v1 contains API Schema definitions for the postgresql v1 API group +kubebuilder:object:generate=true +groupName=postgresql.cnpg.io
|
Package v1 contains API Schema definitions for the postgresql v1 API group +kubebuilder:object:generate=true +groupName=postgresql.cnpg.io |
|
cmd
|
|
|
kubectl-cnpg
kubectl-cnp is a plugin to manage your CloudNativePG clusters
|
kubectl-cnp is a plugin to manage your CloudNativePG clusters |
|
manager
The manager command is the main entrypoint of CloudNativePG operator.
|
The manager command is the main entrypoint of CloudNativePG operator. |
|
Package controllers contains the controller of the CRD
|
Package controllers contains the controller of the CRD |
|
internal
|
|
|
cmd/manager/backup
Package backup implement the "controller backup" command
|
Package backup implement the "controller backup" command |
|
cmd/manager/bootstrap
Package bootstrap implement the "controller bootstrap" command
|
Package bootstrap implement the "controller bootstrap" command |
|
cmd/manager/controller
Package controller implement the command used to start the operator
|
Package controller implement the command used to start the operator |
|
cmd/manager/instance
Package instance implements the "instance" subcommand of the operator
|
Package instance implements the "instance" subcommand of the operator |
|
cmd/manager/instance/initdb
Package initdb implements the "instance init" subcommand of the operator
|
Package initdb implements the "instance init" subcommand of the operator |
|
cmd/manager/instance/join
Package join implements the "instance join" subcommand of the operator
|
Package join implements the "instance join" subcommand of the operator |
|
cmd/manager/instance/pgbasebackup
Package pgbasebackup implement the pgbasebackup bootstrap method
|
Package pgbasebackup implement the pgbasebackup bootstrap method |
|
cmd/manager/instance/restore
Package restore implements the "instance restore" subcommand of the operator
|
Package restore implements the "instance restore" subcommand of the operator |
|
cmd/manager/instance/restoresnapshot
Package restoresnapshot implements the "instance restoresnapshot" subcommand of the operator
|
Package restoresnapshot implements the "instance restoresnapshot" subcommand of the operator |
|
cmd/manager/instance/run
Package run implements the "instance run" subcommand of the operator
|
Package run implements the "instance run" subcommand of the operator |
|
cmd/manager/instance/run/lifecycle
Package lifecycle contains the code to run and manage the lifecycle of a postgres Instance
|
Package lifecycle contains the code to run and manage the lifecycle of a postgres Instance |
|
cmd/manager/instance/status
Package status implement the "instance status" subcommand of the operator
|
Package status implement the "instance status" subcommand of the operator |
|
cmd/manager/pgbouncer
Package pgbouncer implements the "pgbouncer" subcommand of the operator
|
Package pgbouncer implements the "pgbouncer" subcommand of the operator |
|
cmd/manager/pgbouncer/run
Package run implements the "pgbouncer run" subcommand of the operator
|
Package run implements the "pgbouncer run" subcommand of the operator |
|
cmd/manager/show
Package show implement the show command subfeatures
|
Package show implement the show command subfeatures |
|
cmd/manager/show/walarchivequeue
Package walarchivequeue implement the wal-archive-queue command
|
Package walarchivequeue implement the wal-archive-queue command |
|
cmd/manager/walarchive
Package walarchive implement the wal-archive command
|
Package walarchive implement the wal-archive command |
|
cmd/manager/walrestore
Package walrestore implement the walrestore command
|
Package walrestore implement the walrestore command |
|
cmd/plugin
Package plugin contains the common behaviors of the kubectl-cnpg subcommand
|
Package plugin contains the common behaviors of the kubectl-cnpg subcommand |
|
cmd/plugin/backup
Package backup implements a command to request an on-demand backup for a PostgreSQL cluster
|
Package backup implements a command to request an on-demand backup for a PostgreSQL cluster |
|
cmd/plugin/certificate
Package certificate implement the kubectl-cnpg certificate command
|
Package certificate implement the kubectl-cnpg certificate command |
|
cmd/plugin/destroy
Package destroy implements a command to destroy an instances of a cluster and its associated PVC
|
Package destroy implements a command to destroy an instances of a cluster and its associated PVC |
|
cmd/plugin/fence
Package fence implements a command to fence instances in a cluster
|
Package fence implements a command to fence instances in a cluster |
|
cmd/plugin/fio
Package fio implements the fio job creation via deployment
|
Package fio implements the fio job creation via deployment |
|
cmd/plugin/hibernate
Package hibernate implements the hibernation feature
|
Package hibernate implements the hibernation feature |
|
cmd/plugin/install
Package install implements the install plugin command
|
Package install implements the install plugin command |
|
cmd/plugin/logs
Package logs implements the kubectl-cnpg logs command
|
Package logs implements the kubectl-cnpg logs command |
|
cmd/plugin/maintenance
Package maintenance implements the kubectl-cnpg maintenance sub-command
|
Package maintenance implements the kubectl-cnpg maintenance sub-command |
|
cmd/plugin/pgbench
Package pgbench implements the pgbench job creation
|
Package pgbench implements the pgbench job creation |
|
cmd/plugin/promote
Package promote implement the kubectl-cnpg promote command
|
Package promote implement the kubectl-cnpg promote command |
|
cmd/plugin/psql
Package psql implements the `kubectl cnpg psql` command
|
Package psql implements the `kubectl cnpg psql` command |
|
cmd/plugin/reload
Package reload implements a command to trigger a reconciliation loop for a cluster
|
Package reload implements a command to trigger a reconciliation loop for a cluster |
|
cmd/plugin/report
Package report implements the kubectl-cnpg report command
|
Package report implements the kubectl-cnpg report command |
|
cmd/plugin/restart
Package restart implements a command to rollout restart a cluster or restart a single instance
|
Package restart implements a command to rollout restart a cluster or restart a single instance |
|
cmd/plugin/snapshot
Package snapshot implements the snapshot feature
|
Package snapshot implements the snapshot feature |
|
cmd/plugin/status
Package status implements the kubectl-cnpg status command
|
Package status implements the kubectl-cnpg status command |
|
cmd/versions
Package versions builds the version subcommand for both manager and plugins
|
Package versions builds the version subcommand for both manager and plugins |
|
configuration
Package configuration contains the configuration of the operator, reading if from environment variables and from the ConfigMap
|
Package configuration contains the configuration of the operator, reading if from environment variables and from the ConfigMap |
|
management/cache
Package cache contains the constants and functions for reading/writing to the process local cache some specific supported objects
|
Package cache contains the constants and functions for reading/writing to the process local cache some specific supported objects |
|
management/cache/client
Package client contains the constants and functions for reading supported objects from cache or building them in case of cache miss.
|
Package client contains the constants and functions for reading supported objects from cache or building them in case of cache miss. |
|
management/controller
Package controller contains the functions in PostgreSQL instance manager that reacts to changes to the Cluster resource.
|
Package controller contains the functions in PostgreSQL instance manager that reacts to changes to the Cluster resource. |
|
management/controller/externalservers
Package externalservers contains the reconciler of external servers, taking care of enabling connectivity to every server defined in the relative section of the CR
|
Package externalservers contains the reconciler of external servers, taking care of enabling connectivity to every server defined in the relative section of the CR |
|
management/controller/roles
Package roles contains the code needed to reconcile roles with PostgreSQL
|
Package roles contains the code needed to reconcile roles with PostgreSQL |
|
management/controller/slots/infrastructure
Package infrastructure contains the structs and interfaces needed to manage replication slots
|
Package infrastructure contains the structs and interfaces needed to manage replication slots |
|
management/controller/slots/reconciler
Package reconciler contains all the logic needed to reconcile replication slots
|
Package reconciler contains all the logic needed to reconcile replication slots |
|
management/controller/slots/runner
Package runner contains the runner that replicates slots from the primary to the replicas
|
Package runner contains the runner that replicates slots from the primary to the replicas |
|
management/controller/tablespaces
Package tablespaces contains the runner to declarative tablespace
|
Package tablespaces contains the runner to declarative tablespace |
|
management/controller/tablespaces/infrastructure
Package infrastructure contains the structs and interfaces needed to manage declarative tablespace
|
Package infrastructure contains the structs and interfaces needed to manage declarative tablespace |
|
management/istio
Package istio implements functions needed to integrate with istio-proxy
|
Package istio implements functions needed to integrate with istio-proxy |
|
management/linkerd
Package linkerd implements functions needed to integrate with linkerd-proxy
|
Package linkerd implements functions needed to integrate with linkerd-proxy |
|
management/utils
Package utils contains uncategorized utilities only used by the instance manager of PostgreSQL and PgBouncer
|
Package utils contains uncategorized utilities only used by the instance manager of PostgreSQL and PgBouncer |
|
pgbouncer/management/controller
Package controller contains the functions in pgbouncer instance manager that reacts to changes in the Pooler resource.
|
Package controller contains the functions in pgbouncer instance manager that reacts to changes in the Pooler resource. |
|
plugin/resources
Package resources contains reusable functions for the plugin commands
|
Package resources contains reusable functions for the plugin commands |
|
scheme
Package scheme offers a builder capable of generating a scheme with the resources known by the CNP manager
|
Package scheme offers a builder capable of generating a scheme with the resources known by the CNP manager |
|
pkg
|
|
|
certs
Package certs handle the PKI infrastructure of the operator
|
Package certs handle the PKI infrastructure of the operator |
|
cnpgerrors
Package cnpgerrors contains error types for CNPG
|
Package cnpgerrors contains error types for CNPG |
|
concurrency
Package concurrency contains utilities for goroutines coordination
|
Package concurrency contains utilities for goroutines coordination |
|
conditions
Package conditions contains functions useful to update the conditions on the resources managed by the operator
|
Package conditions contains functions useful to update the conditions on the resources managed by the operator |
|
configfile
Package configfile contains primitives needed to manage a configuration file with the syntax of PostgreSQL
|
Package configfile contains primitives needed to manage a configuration file with the syntax of PostgreSQL |
|
configparser
Package configparser contains the code required to fill a Go structure representing the configuration information from several sources like:
|
Package configparser contains the code required to fill a Go structure representing the configuration information from several sources like: |
|
executablehash
Package executablehash detect the SHA256 of the running binary
|
Package executablehash detect the SHA256 of the running binary |
|
fileutils
Package fileutils contains the utility functions about file management
|
Package fileutils contains the utility functions about file management |
|
fileutils/compatibility
Package compatibility provides a layer to cross-compile with other OS than Linux
|
Package compatibility provides a layer to cross-compile with other OS than Linux |
|
management
Package management contains all the features needed by the instance manager that runs in each Pod as PID 1
|
Package management contains all the features needed by the instance manager that runs in each Pod as PID 1 |
|
management/barman
Package barman contains the utilities to interact with barman-cloud.
|
Package barman contains the utilities to interact with barman-cloud. |
|
management/barman/archiver
Package archiver manages the WAL archiving process
|
Package archiver manages the WAL archiving process |
|
management/barman/capabilities
Package capabilities stores the definition of the type for Barman capabilities
|
Package capabilities stores the definition of the type for Barman capabilities |
|
management/barman/credentials
Package credentials is used to build environment for barman cloud commands
|
Package credentials is used to build environment for barman cloud commands |
|
management/barman/restorer
Package restorer manages the WAL restore process
|
Package restorer manages the WAL restore process |
|
management/barman/spool
Package spool implements a WAL pooler keeping track of which WALs we have archived
|
Package spool implements a WAL pooler keeping track of which WALs we have archived |
|
management/catalog
Package catalog is the implementation of a backup catalog
|
Package catalog is the implementation of a backup catalog |
|
management/execlog
Package execlog handles stdout and stderr pipes of started commands and logs them in JSON using the provided logger
|
Package execlog handles stdout and stderr pipes of started commands and logs them in JSON using the provided logger |
|
management/external
Package external contains the functions needed to manage servers which are external to this PostgreSQL cluster
|
Package external contains the functions needed to manage servers which are external to this PostgreSQL cluster |
|
management/external/internal/pgpass
Package pgpass allows the user to generate a PostgreSQL .pgpass file See: https://www.postgresql.org/docs/current/libpq-pgpass.html
|
Package pgpass allows the user to generate a PostgreSQL .pgpass file See: https://www.postgresql.org/docs/current/libpq-pgpass.html |
|
management/log
Package log contains the logging subsystem of the instance manager
|
Package log contains the logging subsystem of the instance manager |
|
management/logtest
Package logtest contains the testing utils for the logging subsystem of the instance manager
|
Package logtest contains the testing utils for the logging subsystem of the instance manager |
|
management/pgbouncer/config
Package config contains the code related to the generation of the PgBouncer configuration
|
Package config contains the code related to the generation of the PgBouncer configuration |
|
management/pgbouncer/metricsserver
Package metricsserver contains the web server powering metrics
|
Package metricsserver contains the web server powering metrics |
|
management/postgres
Package postgres contains the function about starting up, shutting down and managing a PostgreSQL instance.
|
Package postgres contains the function about starting up, shutting down and managing a PostgreSQL instance. |
|
management/postgres/constants
Package constants provides the needed constants in the postgres package
|
Package constants provides the needed constants in the postgres package |
|
management/postgres/logicalimport
Package logicalimport contains the logic needed to import a logical snapshot
|
Package logicalimport contains the logic needed to import a logical snapshot |
|
management/postgres/logpipe
Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.
|
Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON. |
|
management/postgres/metrics
Package metrics enables to expose a set of metrics and collectors on a given postgres instance
|
Package metrics enables to expose a set of metrics and collectors on a given postgres instance |
|
management/postgres/metrics/histogram
Package histogram contain histogram-metrics related functions
|
Package histogram contain histogram-metrics related functions |
|
management/postgres/pool
Package pool contain an implementation of a connection pool to multiple database pointing to the same instance
|
Package pool contain an implementation of a connection pool to multiple database pointing to the same instance |
|
management/postgres/utils
Package utils holds generic utils about postgres instances
|
Package utils holds generic utils about postgres instances |
|
management/postgres/webserver
Package webserver contains the web server powering probes, backups and metrics
|
Package webserver contains the web server powering probes, backups and metrics |
|
management/postgres/webserver/metricserver
Package metricserver contains the web server powering metrics
|
Package metricserver contains the web server powering metrics |
|
management/upgrade
Package upgrade manages the in-place upgrade of the instance manager
|
Package upgrade manages the in-place upgrade of the instance manager |
|
management/url
Package url holds the constants for webserver routing
|
Package url holds the constants for webserver routing |
|
multicache
Package multicache implements a cache that is able to work on multiple namespaces but also able to read data from a namespace which is beside the specified ones.
|
Package multicache implements a cache that is able to work on multiple namespaces but also able to read data from a namespace which is beside the specified ones. |
|
podspec
Package podspec contains various utilities to deal with Pod Specs
|
Package podspec contains various utilities to deal with Pod Specs |
|
postgres
Package postgres contains the function covering the PostgreSQL integrations and the relative data types
|
Package postgres contains the function covering the PostgreSQL integrations and the relative data types |
|
reconciler/backup/volumesnapshot
Package volumesnapshot contains the VolumeSnapshot reconciler that is used to take VolumeSnapshot backups
|
Package volumesnapshot contains the VolumeSnapshot reconciler that is used to take VolumeSnapshot backups |
|
reconciler/hibernation
Package hibernation contains all the logic to hibernate a CNPG cluster
|
Package hibernation contains all the logic to hibernate a CNPG cluster |
|
reconciler/instance
Package instance contains all the logic to reconcile an instance pod
|
Package instance contains all the logic to reconcile an instance pod |
|
reconciler/persistentvolumeclaim
Package persistentvolumeclaim contains all the logic to reconcile and build PVCS
|
Package persistentvolumeclaim contains all the logic to reconcile and build PVCS |
|
resources
Package resources contains a set of Kubernetes generic utilities that are used by the operator
|
Package resources contains a set of Kubernetes generic utilities that are used by the operator |
|
resources/instance
Package instance contains the client capable of querying the HTTP instances endpoints
|
Package instance contains the client capable of querying the HTTP instances endpoints |
|
specs
Package specs contains the specification of the K8s resources generated by the CloudNativePG operator
|
Package specs contains the specification of the K8s resources generated by the CloudNativePG operator |
|
specs/pgbouncer
Package pgbouncer contains the specification of the K8s resources generated by the CloudNativePG operator related to pgbouncer poolers
|
Package pgbouncer contains the specification of the K8s resources generated by the CloudNativePG operator related to pgbouncer poolers |
|
stringset
Package stringset implements a basic set of strings
|
Package stringset implements a basic set of strings |
|
system
Package system provides an interface with the operating system
|
Package system provides an interface with the operating system |
|
system/compatibility
Package compatibility provides a layer to cross-compile with other OS than Linux
|
Package compatibility provides a layer to cross-compile with other OS than Linux |
|
utils
Package utils contains otherwise uncategorized kubernetes relative functions
|
Package utils contains otherwise uncategorized kubernetes relative functions |
|
utils/hash
Package hash allows the user to get a hash number for a given Kubernetes object.
|
Package hash allows the user to get a hash number for a given Kubernetes object. |
|
utils/logs
Package logs contains code to fetch logs from Kubernetes pods
|
Package logs contains code to fetch logs from Kubernetes pods |
|
versions
Package versions contains the version of the CloudNativePG operator and the software that is used by it
|
Package versions contains the version of the CloudNativePG operator and the software that is used by it |
|
Package releases contains the filesystem with operator manifests with all the released manifests
|
Package releases contains the filesystem with operator manifests with all the released manifests |
|
Package tests contains the test infrastructure of the CloudNativePG operator
|
Package tests contains the test infrastructure of the CloudNativePG operator |
|
utils
Package utils contains helper functions/methods for e2e
|
Package utils contains helper functions/methods for e2e |
|
utils/nodes
Package nodes contains the helper methods/functions for nodes
|
Package nodes contains the helper methods/functions for nodes |
Click to show internal directories.
Click to hide internal directories.