crd/

directory
v1.7.0-alpha.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 12, 2021 License: Apache-2.0

README

CRD controller

This directory contains code for custom Kubernetes CRDs and controllers used by the Kubeflow pipelines system. Currently there are 2 such systems:

  • ScheduledWorkflow
  • Viewer

The following are guidelines on developing and running these controllers.

Prerequisites

First, we need to generate the API client code from the API specification.

Get the dependencies:

dep ensure
go get -u ./...

Generate the API client code from the API specification:

./hack/update-codegen.sh

Running ScheduledWorkflow controller from the command line.

The following assumes that your Kubernetes configuration file is located at '$HOME/.kube/config'.

To create the resource for the CRD, execute:

kubectl create -f ./install/manifests/scheduledworkflow-crd.yaml

Output:

customresourcedefinition.apiextensions.k8s.io "scheduledworkflows.kubeflow.org" created

To run the controller locally, execute: configuration file

go run ./controller/scheduledworkflow/*.go -kubeconfig=$HOME/.kube/config -alsologtostderr=true

Output:

Starting workers
Started workers
Wait for shut down

To run a sample workflow on a schedule, execute:

kubectl create -f ./samples/scheduledworkflow/every-minute-cron.yaml

Output:

scheduledworkflow.kubeflow.org "every-minute-cron" created

To see the current list of ScheduledWorkflows, execute:

kubectl get swf

Output:

NAME                AGE
every-minute-cron   1m

To see the current status of the ScheduledWorklfow named 'every-minute-cron', execute:

kubectl describe swf every-minute-cron

Output:

Name:         every-minute-cron
Namespace:    default
Labels:       scheduledworkflows.kubeflow.org/enabled=true
              scheduledworkflows.kubeflow.org/status=Enabled
Annotations:  <none>
API Version:  kubeflow.org/v1beta1
Kind:         ScheduledWorkflow
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-06-06T01:24:55Z
  Generation:          0
  Initializers:        <nil>
  Resource Version:    3056202
  Self Link:           /apis/kubeflow.org/v1beta1/namespaces/default/scheduledworkflows/every-minute-cron
  UID:                 6b11874e-6928-11e8-9fd5-42010a8a0021
Spec:
  Enabled:      true
  Max History:  10
  Trigger:
    Cron Schedule:
      Cron:  1 * * * * *
  Workflow:
    Spec:
      Arguments:
        Parameters:
          Name:    message
          Value:   hello world
      Entrypoint:  whalesay
      Templates:
        Container:
          Args:
            {{inputs.parameters.message}}
          Command:
            cowsay
          Image:  docker/whalesay
          Name:
          Resources:
        Inputs:
          Parameters:
            Name:  message
        Metadata:
        Name:  whalesay
        Outputs:
Status:
  Conditions:
    Last Heartbeat Time:   2018-06-06T01:41:40Z
    Last Transition Time:  2018-06-06T01:41:40Z
    Message:               The schedule is enabled.
    Reason:                Enabled
    Status:                True
    Type:                  Enabled
  Trigger:
    Last Index:           17
    Last Triggered Time:  2018-06-06T01:41:01Z
    Next Triggered Time:  2018-06-06T01:42:01Z
  Workflow History:
    Completed:
      Phase:         Succeeded
      Created At:    2018-06-06T01:41:10Z
      Finished At:   2018-06-06T01:41:13Z
      Index:         17
      Name:          every-minute-cron-17-2173648469
      Namespace:     default
      Scheduled At:  2018-06-06T01:41:01Z
      Self Link:     /apis/argoproj.io/v1alpha1/namespaces/default/workflows/every-minute-cron-17-2173648469
      Started At:    2018-06-06T01:41:10Z
      UID:           b0b63a82-692a-11e8-9fd5-42010a8a0021
      [...]

Running Viewer controller from the command line.

The following assumes that your Kubernetes configuration file is located at '$HOME/.kube/config'.

To create the resource for the CRD, execute:

$ kubectl create -f ./install/manifests/viewer-crd.yaml
customresourcedefinition.apiextensions.k8s.io/viewers.kubeflow.org created

To run the controller locally, execute:

go run ./controller/viewer/ -kubecfg=$HOME/.kube/config -alsologtostderr=true

Now, let's create a simple Tensorboard viewer using the supplied sample:

$ kubectl create -f samples/viewer/mnist.yaml
viewer.kubeflow.org/viewer-75tkf created

$ kubectl get viewers -n kubeflow
NAME           AGE
viewer-75tkf   108s

Verify that the viewer created a deployment and service to house the Tensorboard instance:

$ kubectl -n kubeflow get deployments -l app=viewer
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
viewer-75tkf-deployment   1/1     1            1           3m31s

$ kubectl -n kubeflow get services -l app=viewer
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
viewer-75tkf-service   ClusterIP   10.35.249.82   <none>        80/TCP    3m45s

There are two ways one can access the running Tensorboard instance, via Ambassador, or via direct port forwarding, as described below.

Access via Ambassador

The Tensorboard instance's service has the right annotations to allow it to be routed via Ambassador. Set up port forwarding to the Ambassador instance in your cluster:

$ kubectl port-forward -n kubeflow \
  $(kubectl get pods -n kubeflow --selector service=ambassador -o jsonpath='{.items[0].metadata.name}')  \
  8000:80
Forwarding from 127.0.0.1:8000 -> 80
Forwarding from [::1]:8000 -> 80

Note: The above assumes ambassador is installed under namespace kubeflow.

The Tensorboard instance should now be accessible at http://localhost:8000/tensorboard/viewer-75tkf/. Note that the last path corresponds to the new viewer name, and the URL must end with the trailing slash.

Access via port-forwarding

You can also access the instance directly by setting up port-forwarding to the Pod running the Tensorboard instance. Note that the serving path will still be under /tensorboard/viewer-75tkf, similar to the Ambassador routing scenario above.

To set up port-forwarding to the viewer named viewer-75ktf, run the following command:

kubectl port-forward -n kubeflow \
  $(kubectl get pods -n kubeflow --selector=viewer=viewer-t6qst -o jsonpath='{.items[0].metadata.name}') \
  8000:6006

Notice that we port-forward the remote port 6006, which is the port that Tensorboard viewers are started on.

The Tensorboard instance should now be accessible at http://localhost:8000/tensorboard/viewer-75tkf/.

Directories

Path Synopsis
controller
viewer
Package main is the main binary for running the Viewer Kubernetes CRD controller.
Package main is the main binary for running the Viewer Kubernetes CRD controller.
viewer/reconciler
Package reconciler describes a Reconciler for working with Viewer CRDs.
Package reconciler describes a Reconciler for working with Viewer CRDs.
pkg
apis/scheduledworkflow/v1beta1
Package v1beta1 is the v1beta1 version of the API.
Package v1beta1 is the v1beta1 version of the API.
apis/viewer
Package viewer contains types that define the Viewer CRD for Kubeflow Pipelines.
Package viewer contains types that define the Viewer CRD for Kubeflow Pipelines.
apis/viewer/v1beta1
Package v1beta1 is the v1beta1 version of the API.
Package v1beta1 is the v1beta1 version of the API.
client/clientset/versioned
This package has the automatically generated clientset.
This package has the automatically generated clientset.
client/clientset/versioned/fake
This package has the automatically generated fake clientset.
This package has the automatically generated fake clientset.
client/clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
This package contains the scheme of the automatically generated clientset.
client/clientset/versioned/typed/scheduledworkflow/v1beta1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/scheduledworkflow/v1beta1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL