web

README

Web provider for Virtual Kubelet

Virtual Kubelet providers are written using the Go programming language. While Go is a great general purpose programming language, it is however a fact that other programming languages exist. The problem that Virtual Kubelet solves is as applicable to applications written in those languages as it is for those written using Go. This provider aims to serve as a bridge between technology stacks and programming languages, as it were, by adapting the Virtual Kubelet provider interface using a web endpoint, i.e., this provider is a thin layer that forwards all calls that Kubernetes makes to the virtual kubelet to a pre-configured HTTP endpoint. This frees the provider's implementor to write their code in any programming language and technology stack as they see fit.

The providers/web/web-rust folder contains a sample provider implemented in the Rust programming language. Here's a diagram that depicts the interaction between Kubernetes, the virtual kubelet web provider and the Rust app.

+----------------+         +---------------------------+          +------------------------------+
|                |         |                           |   HTTP   |                              |
|   Kubernetes   | <-----> |   Virtual Kubelet: Web    | <------> |   Provider written in Rust   |
|                |         |                           |          |                              |
+----------------+         +---------------------------+          +------------------------------+

Provider interface

The web provider uses an environment variable to determine the endpoint to use for forwarding requests. The environment variable must be named WEB_ENDPOINT_URL and must implement the following HTTP API:

Path	Verb	Query	Request	Response	Description
/createPod	POST	-	Pod JSON	HTTP status code	Create a new pod
/updatePod	PUT	-	Pod JSON	HTTP status code	Update pod spec
/deletePod	DELETE	-	Pod JSON	HTTP status code	Delete an existing pod
/getPod	GET	namespace, name	-	Pod JSON	Given a pod namespace and name, return the pod JSON
/getContainerLogs	GET	namespace, podName, containerName, tail	-	Container logs	Given the namespace, pod name and container name, return tail log lines
/getPodStatus	GET	namespace, name	-	Pod status JSON	Given a pod namespace and name, return the pod's status JSON
/getPods	GET	-	-	Array of pod JSON strings	Fetch list of created pods
/capacity	GET	-	-	JSON map containing resource name and values	Fetch resource capacity values
/nodeConditions	GET	-	-	Array of node condition JSON strings	Get list of node conditions (Ready, OutOfDisk etc)
/nodeAddresses	GET	-	-	Array of node address values (type/address pairs)	Fetch a list of addresses for the node status

A typical deployment configuration for this setup would be to have the provider implementation be deployed as a container in the same pod as the virtual kubelet itself (as a "sidecar").

Take her for a spin

A sample web provider implementation is included in this repo in order to showcase what this enables. The sample has been implemented in Rust. The easiest way to get this up and running is to use the Helm chart available at providers/web/charts/web-rust. Open a terminal and install the chart like so:

$ cd providers/web/charts
$ helm install -n web-provider ./web-rust

$ kubectl get pods
NAME                                                READY     STATUS    RESTARTS   AGE
web-provider-virtual-kubelet-web-6b5b7446f6-279xl   2/2       Running   0          3h

If you list the nodes in the cluster after this you should see something that looks like this:

$ kubectl get nodes
NAME                       STATUS    ROLES     AGE       VERSION
aks-nodepool1-35187879-0   Ready     agent     37d       v1.8.2
aks-nodepool1-35187879-1   Ready     agent     37d       v1.8.2
aks-nodepool1-35187879-3   Ready     agent     37d       v1.8.2
virtual-kubelet-web        Ready     agent     3h        v1.8.3

In case the name of the node didn't give it away, the last entry in the output above is the virtual kubelet. If you try to list the containers in the pod that represents the virtual kubelet you should be able to see the sidecar Rust container:

$ kubectl get pods -o=custom-columns=NAME:.metadata.name,CONTAINERS:.spec.containers[*].name
NAME                                                CONTAINERS
web-provider-virtual-kubelet-web-6b5b7446f6-279xl   webrust,virtualkubelet

In the output above, webrust is the sidecar container and virtualkubelet is the broker that forwards requests to webrust. You can run a query on the /getPods HTTP endpoint on the webrust container to see a list of the pods that it has been asked to create. To do this we first use kubectl to setup a port forwarding server like so:

$ kubectl port-forward web-provider-virtual-kubelet-web-6b5b7446f6-279xl 3000:3000

Now if we run curl on the http://localhost:3000/getPods URL you should see the pod JSON getting dumped to the terminal. I ran my test on a Kubernetes cluster deployed on Azure which happens to deploy a daemonset with some, what I imagine are "system" pods to every node in the cluster. You can filter the output to see just the pod names using the jq tool like so:

$ curl -s http://localhost:3000/getPods | jq '.[] | { name: .metadata.name }'
{
    "name": "kube-proxy-czz57"
}
{
    "name": "kube-svc-redirect-7qlpd"
}

You can deploy workloads to the virtual kubelet as you normally do. Here's a sample pod spec that uses nodeSelector to cause the deployment to be scheduled on the virtual kubelet.

apiVersion: v1
kind: Pod
metadata:
        name: vk-pod
        labels:
                foo: bar
spec:
        containers:
        - name: web1
        image: nginx
        nodeSelector:
                type: virtual-kubelet

Let's go ahead and deploy the pod and run our /getPods query again:

$ kubectl apply -f ~/tmp/pod1.yaml
pod "vk-pod" created

$ curl -s http://localhost:3000/getPods | jq '.[] | { name: .metadata.name }'
{
    "name": "kube-proxy-czz57"
}
{
    "name": "kube-svc-redirect-7qlpd"
}
{
    "name": "vk-pod"
}

$ kubectl get pods
NAME                                                READY     STATUS    RESTARTS   AGE
vk-pod                                              0/1       Running   0          1m
web-provider-virtual-kubelet-web-6b5b7446f6-279xl   2/2       Running   6          4h

As you can tell, a new pod has been scheduled to run on our virtual kubelet instance. Deleting pods works as one would expect:

$ kubectl delete -f ~/tmp/pod1.yaml
pod "vk-pod" deleted

$ curl -s http://localhost:3000/getPods | jq '.[] | { name: .metadata.name }'
{
    "name": "kube-proxy-czz57"
}
{
    "name": "kube-svc-redirect-7qlpd"
}

Documentation

Overview

Package web provides an implementation of the virtual kubelet provider interface by forwarding all calls to a web endpoint. The web endpoint to which requests must be forwarded must be specified through an environment variable called `WEB_ENDPOINT_URL`. This endpoint must implement the following HTTP APIs:

• POST /createPod
• PUT /updatePod
• DELETE /deletePod
• GET /getPod?namespace=[namespace]&name=[pod name]
• GE /getContainerLogs?namespace=[namespace]&podName=[pod name]&containerName=[container name]&tail=[tail value]
• GET /getPodStatus?namespace=[namespace]&name=[pod name]
• GET /getPods
• GET /capacity
• GET /nodeConditions
• GET /nodeAddresses

Index

• type BrokerProvider
  • func NewBrokerProvider(nodeName, operatingSystem string, daemonEndpointPort int32) (*BrokerProvider, error)
  • func (p *BrokerProvider) Capacity(ctx context.Context) v1.ResourceList
  • func (p *BrokerProvider) CreatePod(ctx context.Context, pod *v1.Pod) error
  • func (p *BrokerProvider) DeletePod(ctx context.Context, pod *v1.Pod) error
  • func (p *BrokerProvider) GetContainerLogs(ctx context.Context, namespace, podName, containerName string, ...) (io.ReadCloser, error)
  • func (p *BrokerProvider) GetPod(ctx context.Context, namespace, name string) (*v1.Pod, error)
  • func (p *BrokerProvider) GetPodFullName(namespace string, pod string) string
  • func (p *BrokerProvider) GetPodStatus(ctx context.Context, namespace, name string) (*v1.PodStatus, error)
  • func (p *BrokerProvider) GetPods(ctx context.Context) ([]*v1.Pod, error)
  • func (p *BrokerProvider) NodeAddresses(ctx context.Context) []v1.NodeAddress
  • func (p *BrokerProvider) NodeConditions(ctx context.Context) []v1.NodeCondition
  • func (p *BrokerProvider) NodeDaemonEndpoints(ctx context.Context) *v1.NodeDaemonEndpoints
  • func (p *BrokerProvider) OperatingSystem() string
  • func (p *BrokerProvider) RunInContainer(ctx context.Context, namespace, name, container string, cmd []string, ...) error
  • func (p *BrokerProvider) UpdatePod(ctx context.Context, pod *v1.Pod) error

Types

type BrokerProvider

type BrokerProvider struct {
	// contains filtered or unexported fields
}

BrokerProvider implements the virtual-kubelet provider interface by forwarding kubelet calls to a web endpoint.

func NewBrokerProvider

func NewBrokerProvider(nodeName, operatingSystem string, daemonEndpointPort int32) (*BrokerProvider, error)

NewBrokerProvider creates a new BrokerProvider

func (*BrokerProvider) Capacity

func (p *BrokerProvider) Capacity(ctx context.Context) v1.ResourceList

Capacity returns a resource list containing the capacity limits

func (*BrokerProvider) CreatePod

func (p *BrokerProvider) CreatePod(ctx context.Context, pod *v1.Pod) error

CreatePod accepts a Pod definition and forwards the call to the web endpoint

func (*BrokerProvider) DeletePod

func (p *BrokerProvider) DeletePod(ctx context.Context, pod *v1.Pod) error

DeletePod accepts a Pod definition and forwards the call to the web endpoint

func (*BrokerProvider) GetContainerLogs

func (p *BrokerProvider) GetContainerLogs(ctx context.Context, namespace, podName, containerName string, opts api.ContainerLogOpts) (io.ReadCloser, error)

GetContainerLogs returns the logs of a container running in a pod by name.

func (*BrokerProvider) GetPod

func (p *BrokerProvider) GetPod(ctx context.Context, namespace, name string) (*v1.Pod, error)

GetPod returns a pod by name that is being managed by the web server

func (*BrokerProvider) GetPodFullName

func (p *BrokerProvider) GetPodFullName(namespace string, pod string) string

Get full pod name as defined in the provider context TODO: Implementation

func (*BrokerProvider) GetPodStatus

func (p *BrokerProvider) GetPodStatus(ctx context.Context, namespace, name string) (*v1.PodStatus, error)

GetPodStatus retrieves the status of a given pod by name.

func (*BrokerProvider) GetPods

func (p *BrokerProvider) GetPods(ctx context.Context) ([]*v1.Pod, error)

GetPods retrieves a list of all pods scheduled to run.

func (*BrokerProvider) NodeAddresses

func (p *BrokerProvider) NodeAddresses(ctx context.Context) []v1.NodeAddress

NodeAddresses returns a list of addresses for the node status within Kubernetes.

func (*BrokerProvider) NodeConditions

func (p *BrokerProvider) NodeConditions(ctx context.Context) []v1.NodeCondition

NodeConditions returns a list of conditions (Ready, OutOfDisk, etc), for updates to the node status

func (*BrokerProvider) NodeDaemonEndpoints

func (p *BrokerProvider) NodeDaemonEndpoints(ctx context.Context) *v1.NodeDaemonEndpoints

NodeDaemonEndpoints returns NodeDaemonEndpoints for the node status within Kubernetes.

func (*BrokerProvider) OperatingSystem

func (p *BrokerProvider) OperatingSystem() string

OperatingSystem returns the operating system for this provider.

func (*BrokerProvider) RunInContainer

func (p *BrokerProvider) RunInContainer(ctx context.Context, namespace, name, container string, cmd []string, attach api.AttachIO) error

RunInContainer executes a command in a container in the pod, copying data between in/out/err and the container's stdin/stdout/stderr. TODO: Implementation

func (*BrokerProvider) UpdatePod

func (p *BrokerProvider) UpdatePod(ctx context.Context, pod *v1.Pod) error

UpdatePod accepts a Pod definition and forwards the call to the web endpoint