nomad

README

HashiCorp Nomad Provider for Virtual Kubelet

HashiCorp Nomad provider for Virtual Kubelet connects your Kubernetes cluster with Nomad cluster by exposing the Nomad cluster as a node in Kubernetes. By using the provider, pods that are scheduled on the virtual Nomad node registered on Kubernetes will run as jobs on Nomad clients as they would on a Kubernetes node.

This is an experimental project. This project isn't production ready.

Demo

Prerequisites

This guide assumes the following:

• A Nomad cluster up and running.
• A Kubernetes cluster up and running.
• The Nomad API is accessible from the Kubernetes cluster.
• kubectl installed.

Usage

The Nomad provider accepts the following two environment variables:

• NOMAD_ADDR - The Nomad API address. Set to 127.0.0.1:4646 by default.
• NOMAD_REGION - The Nomad region. Set to global by default.

export NOMAD_ADDR="127.0.0.1:4646"
export NOMAD_REGION="global"

Run Virtual Kubelet with Nomad Provider

VK_TAINT_KEY="hashicorp.com/nomad" ./virtual-kubelet --provider="nomad"

Validate that the virtual kubelet node is registered.

kubectl get nodes

Expected output.

NAME              STATUS    ROLES     AGE       VERSION
minikube          Ready     master    55d       v1.10.0
virtual-kubelet   Ready     agent     1m        v1.13.1-vk-N/A

Create a Pod in Kubernetes

kubectl apply -f pods/nginx-pod.yaml

Validate pod.

kubectl get pods

Expected output.

NAME      READY     STATUS    RESTARTS   AGE
nginx     1/1       Running   0          1m

Validate Nomad job.

nomad status

Expected output.

ID                           Type     Priority  Status   Submit Date
nomad-virtual-kubelet-nginx  service  100       running  2018-12-31T16:52:52+05:30

Configuration Options

The Nomad provider has support for annotations to define Nomad datacenters.

Here is an example usage of the Nomad datacenter annotations in a pod spec.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  annotations:
    "nomad.hashicorp.com/datacenters": "us-east1,us-west1"
spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: nginx
    ports:
    - containerPort: 80
    - containerPort: 443
  dnsPolicy: ClusterFirst
  nodeSelector:
    kubernetes.io/role: agent
    beta.kubernetes.io/os: linux
    type: virtual-kubelet
  tolerations:
  - key: virtual-kubelet.io/provider
    operator: Exists
  - key: hashicorp.com/nomad
    effect: NoSchedule

Documentation

Index

• type Provider
  • func NewProvider(rm *manager.ResourceManager, nodeName, operatingSystem string) (*Provider, error)
  • func (p *Provider) Capacity(ctx context.Context) v1.ResourceList
  • func (p *Provider) CreatePod(ctx context.Context, pod *v1.Pod) error
  • func (p *Provider) DeletePod(ctx context.Context, pod *v1.Pod) (err error)
  • func (p *Provider) GetContainerLogs(ctx context.Context, namespace, podName, containerName string, ...) (io.ReadCloser, error)
  • func (p *Provider) GetPod(ctx context.Context, namespace, name string) (pod *v1.Pod, err error)
  • func (p *Provider) GetPodFullName(ctx context.Context, namespace string, pod string) string
  • func (p *Provider) GetPodStatus(ctx context.Context, namespace, name string) (*v1.PodStatus, error)
  • func (p *Provider) GetPods(ctx context.Context) ([]*v1.Pod, error)
  • func (p *Provider) NodeAddresses(ctx context.Context) []v1.NodeAddress
  • func (p *Provider) NodeConditions(ctx context.Context) []v1.NodeCondition
  • func (p *Provider) NodeDaemonEndpoints(ctx context.Context) *v1.NodeDaemonEndpoints
  • func (p *Provider) OperatingSystem() string
  • func (p *Provider) RunInContainer(ctx context.Context, namespace, name, container string, cmd []string, ...) error
  • func (p *Provider) UpdatePod(ctx context.Context, pod *v1.Pod) error

Types

type Provider

type Provider struct {
	// contains filtered or unexported fields
}

Provider implements the virtual-kubelet provider interface and communicates with the Nomad API.

func NewProvider

func NewProvider(rm *manager.ResourceManager, nodeName, operatingSystem string) (*Provider, error)

NewProvider creates a new Provider

func (*Provider) Capacity

func (p *Provider) Capacity(ctx context.Context) v1.ResourceList

Capacity returns a resource list containing the capacity limits set for Nomad.

func (*Provider) CreatePod

func (p *Provider) CreatePod(ctx context.Context, pod *v1.Pod) error

CreatePod accepts a Pod definition and creates a Nomad job

func (*Provider) DeletePod

func (p *Provider) DeletePod(ctx context.Context, pod *v1.Pod) (err error)

DeletePod accepts a Pod definition and deletes a Nomad job.

func (*Provider) GetContainerLogs

func (p *Provider) GetContainerLogs(ctx context.Context, namespace, podName, containerName string, opts api.ContainerLogOpts) (io.ReadCloser, error)

GetContainerLogs retrieves the logs of a container by name from the provider.

func (*Provider) GetPod

func (p *Provider) GetPod(ctx context.Context, namespace, name string) (pod *v1.Pod, err error)

GetPod returns the pod running in the Nomad cluster. returns nil if pod is not found.

func (*Provider) GetPodFullName

func (p *Provider) GetPodFullName(ctx context.Context, namespace string, pod string) string

GetPodFullName as defined in the provider context

func (*Provider) GetPodStatus

func (p *Provider) GetPodStatus(ctx context.Context, namespace, name string) (*v1.PodStatus, error)

GetPodStatus returns the status of a pod by name that is running as a job in the Nomad cluster returns nil if a pod by that name is not found.

func (*Provider) GetPods

func (p *Provider) GetPods(ctx context.Context) ([]*v1.Pod, error)

GetPods returns a list of all pods known to be running in Nomad nodes.

func (*Provider) NodeAddresses

func (p *Provider) NodeAddresses(ctx context.Context) []v1.NodeAddress

NodeAddresses returns a list of addresses for the node status within Kubernetes.

func (*Provider) NodeConditions

func (p *Provider) NodeConditions(ctx context.Context) []v1.NodeCondition

NodeConditions returns a list of conditions (Ready, OutOfDisk, etc), for updates to the node status within Kubernetes.

func (*Provider) NodeDaemonEndpoints

func (p *Provider) NodeDaemonEndpoints(ctx context.Context) *v1.NodeDaemonEndpoints

NodeDaemonEndpoints returns NodeDaemonEndpoints for the node status within Kubernetes.

func (*Provider) OperatingSystem

func (p *Provider) OperatingSystem() string

OperatingSystem returns the operating system for this provider. This is a noop to default to Linux for now.

func (*Provider) RunInContainer

func (p *Provider) RunInContainer(ctx context.Context, namespace, name, container string, cmd []string, attach api.AttachIO) error

RunInContainer executes a command in a container in the pod, copying data between in/out/err and the container's stdin/stdout/stderr. TODO: Implementation

func (*Provider) UpdatePod

func (p *Provider) UpdatePod(ctx context.Context, pod *v1.Pod) error

UpdatePod is a noop, nomad does not support live updates of a pod.