provisioner

package
v0.44.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 6, 2024 License: Apache-2.0 Imports: 26 Imported by: 0

README

provisioner

This directory contains the provisioner package. On a high level the provisioner's main responsibility is to allocate resources on a runtime to a deployment. Currently there is two different types of provisioners, static and kubernetes. The static type will allocate a runtime from a statically pre-defined pool of runtimes and the kubernetes type will dynamically provision a dedicated runtime in Kubernetes and allocate it to the deployment.

Configuration

The provisioner is configured using RILL_ADMIN_PROVISIONER_SET_JSON with a named set of provisioners using a format like the following example. More provisioners of the same type can be configured, this is a useful for example to support deployments to different Kubernetes clusters. Furthermore the name of the default provisioner needs to be specified with RILL_ADMIN_DEFAULT_PROVISIONER, this provisioner will be used for all deployed projects where a provisioner is not explicitly chosen.

{
  "static-example":
    {
      "type": "static",
      "spec":
        {
          "runtimes":
            [
              {
                "host": "http://localhost:9091",          // Runtime host
                "slots": 50,                              // Amount of slots in the pre-provisioned runtime
                "data_dir": "/mnt/data",                  // Directory to use for data storage like DB files etc.
                "audience_url": "http://localhost:8081"   // Audience URL (JWT)
              }
            ]
        }
    },

  "kubernetes-example":
    {
      "type": "kubernetes",
      "spec":
        {
          "timeout_seconds": 30,                              // Maximum time to wait for the runtime to become ready
          "data_dir": "/mnt/data",                            // Directory to use for data storage like DB files etc.
          "host": "http://node-*.localhost",                  // The wildcard '*' will be replaced with the deployment's 'provision_id'
          "namespace": "cloud-runtime",                       // Namespace to use in the K8s cluster
          "image": "rilldata/rill",                           // Rill Docker image
          "kubeconfig_path": "kubeconfig.yaml",               // K8s config file to authenticate against the cluster
          "template_paths":
            {
              "http_ingress": "templates/http_ingress.yaml",  // Ingress resource template for HTTP
              "grpc_ingress": "templates/grpc_ingress.yaml",  // Ingress resource template for GRCP
              "service": "templates/service.yaml",            // Service resource template
              "statefulset": "templates/statefulset.yaml"     // Statefulset resource template
            }
        }
    }
}

Development

Be aware that the runtimes provisioned in Kubernetes will need to be able to communicate with the admin server to function correctly, so if the admin server is running locally and you setup provisioning to an external cluster, you'll need to make sure there's an available network path from the runtimes to your local admin server.

Templates

The Kubernetes resource templates provides a high level of flexibility, but they will need to be adapted to the specific Kubernetes environment. The simplified examples below will provide a good starting point.

Note: For internal Rill users refer to our private infra repos containing environment specific configurations and templates.

statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
spec:
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Delete
    whenScaled: Retain
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: {{ .StorageBytes }}
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: {{ .Names.StatefulSet }}
  serviceName: cloud-runtime
  template:
    metadata:
      labels:
        app.kubernetes.io/name: {{ .Names.StatefulSet }}
    spec:
      securityContext:
        fsGroup: 1000
      containers:
      - args:
        - runtime
        - start
        command:
        - rill
        env:
        - name: RILL_RUNTIME_GRPC_PORT
          value: "9090"
        - name: RILL_RUNTIME_HTTP_PORT
          value: "8080"

        ########################################################################
        # Add all the relevant runtime configuration environment variables here
        ########################################################################

        image: "{{ .Image }}:{{ .ImageTag }}"
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /v1/ping
            port: 8080
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: cloud-runtime
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 9090
          protocol: TCP
        resources:
          limits:
            cpu: {{ .CPU }}
            memory: {{ .MemoryGB }}Gi
          requests:
            cpu: {{ .CPU }}
            memory: {{ .MemoryGB }}Gi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - all
          runAsNonRoot: true
          runAsUser: 1000
        terminationMessagePolicy: FallbackToLogsOnError
        volumeMounts:
        - mountPath: {{ .DataDir }}
          name: data
service.yaml
apiVersion: v1
kind: Service
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: grpc
    port: 9090
    targetPort: 9090
  selector:
    app.kubernetes.io/name: {{ .Names.StatefulSet }}
grpc_ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: GRPC
spec:
  ingressClassName: nginx
  rules:
  - host: {{ .Host }}
    http:
      paths:
      - backend:
          service:
            name: {{ .Names.Service }}
            port:
              number: 9090
        path: /
        pathType: Prefix
http_ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
spec:
  ingressClassName: nginx
  rules:
  - host: {{ .Host }}
    http:
      paths:
      - backend:
          service:
            name: {{ .Names.Service }}
            port:
              number: 8080
        path: /v1
        pathType: Prefix

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewSet added in v0.43.0 

func NewSet(set string, db database.DB, logger *zap.Logger) (map[string]Provisioner, error)

Types

type Allocation added in v0.24.3 

type Allocation struct {
	Host         string
	Audience     string
	CPU          int
	MemoryGB     int
	StorageBytes int64
}

type KubernetesProvisioner added in v0.43.0 

type KubernetesProvisioner struct {
	Spec *KubernetesSpec
	// contains filtered or unexported fields
}

func NewKubernetes added in v0.43.0 

func NewKubernetes(spec json.RawMessage) (*KubernetesProvisioner, error)

func (*KubernetesProvisioner) AwaitReady added in v0.43.0 

func (p *KubernetesProvisioner) AwaitReady(ctx context.Context, provisionID string) error

func (*KubernetesProvisioner) CheckCapacity added in v0.43.0 

func (p *KubernetesProvisioner) CheckCapacity(ctx context.Context) error

func (*KubernetesProvisioner) Deprovision added in v0.43.0 

func (p *KubernetesProvisioner) Deprovision(ctx context.Context, provisionID string) error

func (*KubernetesProvisioner) Provision added in v0.43.0 

func (p *KubernetesProvisioner) Provision(ctx context.Context, opts *ProvisionOptions) (*Allocation, error)

func (*KubernetesProvisioner) Update added in v0.43.0 

func (p *KubernetesProvisioner) Update(ctx context.Context, provisionID, newVersion string) error

type KubernetesSpec added in v0.43.0 

type KubernetesSpec struct {
	Host           string                   `json:"host"`
	Image          string                   `json:"image"`
	Namespace      string                   `json:"namespace"`
	TimeoutSeconds int                      `json:"timeout_seconds"`
	KubeconfigPath string                   `json:"kubeconfig_path"`
	TemplatePaths  *KubernetesTemplatePaths `json:"template_paths"`
}

type KubernetesTemplatePaths added in v0.43.0 

type KubernetesTemplatePaths struct {
	HTTPIngress string `json:"http_ingress"`
	GRPCIngress string `json:"grpc_ingress"`
	Service     string `json:"service"`
	StatefulSet string `json:"statefulset"`
}

type ProvisionOptions 

type ProvisionOptions struct {
	ProvisionID    string
	RuntimeVersion string
	Slots          int
	Annotations    map[string]string
}

type Provisioner 

type Provisioner interface {
	Provision(ctx context.Context, opts *ProvisionOptions) (*Allocation, error)
	Deprovision(ctx context.Context, provisionID string) error
	AwaitReady(ctx context.Context, provisionID string) error
	Update(ctx context.Context, provisionID string, newVersion string) error
	CheckCapacity(ctx context.Context) error
}

type ProvisionerSpec added in v0.43.0 

type ProvisionerSpec struct {
	Type string          `json:"type"`
	Spec json.RawMessage `json:"spec"`
}

type ResourceNames added in v0.43.0 

type ResourceNames struct {
	HTTPIngress string
	GRPCIngress string
	Service     string
	StatefulSet string
}

type StaticProvisioner added in v0.26.0 

type StaticProvisioner struct {
	Spec *StaticSpec
	// contains filtered or unexported fields
}

func NewStatic 

func NewStatic(spec json.RawMessage, db database.DB, logger *zap.Logger) (*StaticProvisioner, error)

func (*StaticProvisioner) AwaitReady added in v0.43.0 

func (p *StaticProvisioner) AwaitReady(ctx context.Context, provisionID string) error

func (*StaticProvisioner) CheckCapacity added in v0.43.0 

func (p *StaticProvisioner) CheckCapacity(ctx context.Context) error

func (*StaticProvisioner) Deprovision added in v0.43.0 

func (p *StaticProvisioner) Deprovision(ctx context.Context, provisionID string) error

func (*StaticProvisioner) Provision added in v0.26.0 

func (p *StaticProvisioner) Provision(ctx context.Context, opts *ProvisionOptions) (*Allocation, error)

func (*StaticProvisioner) Update added in v0.43.0 

func (p *StaticProvisioner) Update(ctx context.Context, provisionID, newVersion string) error

type StaticRuntimeSpec added in v0.26.0 

type StaticRuntimeSpec struct {
	Host     string `json:"host"`
	Slots    int    `json:"slots"`
	Audience string `json:"audience_url"`
}

type StaticSpec added in v0.26.0 

type StaticSpec struct {
	Runtimes []*StaticRuntimeSpec `json:"runtimes"`
}

type TemplateData added in v0.43.0 

type TemplateData struct {
	Image        string
	ImageTag     string
	Host         string
	CPU          int
	MemoryGB     int
	StorageBytes int64
	Slots        int
	Names        ResourceNames
	Annotations  map[string]string
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.