operator

README

Operator

See the binary documentation for deployment instructions.

Testing

The operator has an end-to-end test suite to run functional tests against a real Kubernetes cluster.

To run the tests a kubeconfig pointing to a GKE cluster is required. This is generally already taken care of while setting up a GKE cluster (instructions). Use kubectl config {current,set}-context to verify or change which cluster the tests will execute against.

The tests require that the CRD definition and ClusterRole gmp-system:collector already exist in the cluster. (They are part of deploying the operator itself, we make this manual for tests to not unknowingly deploy resources with cluster-wide effects.) All other resources are created and cleaned up by the test suite. To setup the resources:

kubectl apply -f ../../cmd/operator/deploy/operator/crds.yaml
kubectl apply -f ../../cmd/operator/deploy/operator/clusterrole.yaml
kubectl apply -f ../../cmd/operator/deploy/operator/priority_class.yaml

The tests verify the metric data written into GCM, for which information about the GKE cluster must be provided. Execute the tests with:

go test ./e2e/ \
    --project-id=$PROJECT_ID --cluster=$CLUSTER_NAME --location=$LOCATION

Credentials

Tests by default assume to run in a GKE cluster and that they can infer sufficient credentials from the environment.

If that's not the case the --skip-gcm flag disables tests paths that require connectivity to the GMP and GCM backends.

Alternatively, the --gcp-service-account=<filepath> flag allows providing a GCP service account credentials file which is used for deployed components instead of relying on the environment. The service account needs permission to read and write metric data against the --project-id. Running the test on GKE with and without this option provides more comprehensive coverage.

Code Generation

To update generated code when changes to Custom Resource Definitions are made run:

make codegen
make crds

The generated CRD YAMLs may require manual editing. Especially consider whether fields are correctly marked as required or not.

Documentation

Index

• Constants
• type Operator
  • func New(logger logr.Logger, clientConfig *rest.Config, registry prometheus.Registerer, ...) (*Operator, error)
  • func (o *Operator) Run(ctx context.Context, ors ...metav1.OwnerReference) error
• type Options

Constants

const (
	// DefaultOperatorNamespace is the namespace in which all resources owned by the operator are installed.
	DefaultOperatorNamespace = "gmp-system"
	// DefaultPublicNamespace is the namespace where the operator will check for user-specified
	// configuration data.
	DefaultPublicNamespace = "gmp-public"

	// Fixed names used in various resources managed by the operator.
	NameOperator = "gmp-operator"

	// The well-known app name label.
	LabelAppName = "app.kubernetes.io/name"
	// The component name, will be exposed as metric name.
	AnnotationMetricName = "components.gke.io/component-name"

	// The official images to be used with this version of the operator. For debugging
	// and emergency use cases they may be overwritten through options.
	ImageCollector      = "gke.gcr.io/prometheus-engine/prometheus:v2.28.1-gmp.3-gke.0"
	ImageConfigReloader = "gke.gcr.io/prometheus-engine/config-reloader:v0.2.3-gke.0"
	ImageRuleEvaluator  = "gke.gcr.io/prometheus-engine/rule-evaluator:v0.2.3-gke.0"

	// The k8s Application, will be exposed as component name.
	KubernetesAppName    = "app"
	CollectorAppName     = "managed-prometheus-collector"
	RuleEvaluatorAppName = "managed-prometheus-rule-evaluator"
)

const (
	NameOperatorConfig = "config"
	NameRuleEvaluator  = "rule-evaluator"
	NameCollector      = "collector"
)

Base resource names which may be used for multiple different resource kinds related to the given component.

const (
	RulesSecretName      = "rules"
	CollectionSecretName = "collection"

	RuleEvaluatorPort = 19092
)

Types

type Operator

type Operator struct {
	// contains filtered or unexported fields
}

Operator to implement managed collection for Google Prometheus Engine.

func New

func New(logger logr.Logger, clientConfig *rest.Config, registry prometheus.Registerer, opts Options) (*Operator, error)

New instantiates a new Operator.

func (*Operator) Run

func (o *Operator) Run(ctx context.Context, ors ...metav1.OwnerReference) error

Run the reconciliation loop of the operator. The passed owner references are set on cluster-wide resources created by the operator.

type Options

type Options struct {
	// ID of the project of the cluster.
	ProjectID string
	// Location of the cluster.
	Location string
	// Name of the cluster the operator acts on.
	Cluster string
	// Disable exporting to GCM (mostly for testing).
	DisableExport bool
	// Namespace to which the operator deploys any associated resources.
	OperatorNamespace string
	// Namespace to which the operator looks for user-specified configuration
	// data, like Secrets and ConfigMaps.
	PublicNamespace string
	// Listening port of the collector. Configurable to allow multiple
	// simultanious collector deployments for testing purposes while each
	// collector runs on the host network.
	CollectorPort int32
	// Image for the Prometheus collector container.
	ImageCollector string
	// Image for the Prometheus config reloader.
	ImageConfigReloader string
	// Image for the Prometheus rule-evaluator.
	ImageRuleEvaluator string
	// Whether to deploy pods with hostNetwork enabled. This allow pods to run with the GCE compute
	// default service account even on GKE clusters with Workload Identity enabled.
	// It must be set to false for GKE Autopilot clusters.
	HostNetwork bool
	// Priority class for the collector pods.
	PriorityClass string
	// Certificate of the server in base 64.
	TLSCert string
	// Key of the server in base 64.
	TLSKey string
	// Endpoint of the Cloud Monitoring API to be used by all collectors.
	CloudMonitoringEndpoint string
	// Webhook serving address.
	ListenAddr string
}

Options for the Operator.