ibm-licensing-operator

README

Table of Contents generated with DocToc

• IBM Licensing Operator
  • Overview
  • Installation
    • Install the IBM Licensing Operator On OCP 4.2+
      • 1. Create OperatorSource
      • 2. Create a Namespace ibm-common-services
      • 3. Install IBM Licensing Operator Package in the OperatorHub
    • Install the IBM Licensing Operator on Kubernetes from scratch
      • 1. Install Operator Lifecycle Manager
      • 2. Install the Operator Marketplace
      • 3. Create the OperatorSource
      • 4. View Available Operators
      • 5. Create an OperatorGroup
      • 6. Create a Subscription
      • 7. Verify Operator health
  • Using IBM Licensing Service
    • Create instance on OCP 4.2+
    • Create instance from console
    • Check Components
    • Show and use API
  • Uninstall
    • Uninstall from any kubernetes cluster
      • 1. Delete IBMLicensing custom resource
      • 2. Delete subscription
      • 3. Delete CSV
      • 4. Delete CRD
      • 5. Delete OperatorGroup
      • 6. Delete OperatorSource
      • 7. Delete OperatorMarketplace
      • 8. Uninstall OLM
  • Troubleshoot
    • CreateContainerConfigError Marketplace Operator error

IBM Licensing Operator

In this document, we will show you how to deploy and use IBM Licensing Service inside your Cluster using Operator Lifecycle Manager.

Overview

• IBM Licensing Service is a tool that collects licensing data from IBM Products across all cluster. It provides API to see the usage and generate Audit Snapshots.
• You can read more about Operators and Operator Lifecycle Manager here

Installation

IBM Licensing Operator can be installed either on vanilla kubernetes, or on OpenShift Container Platform. It is also a part of bigger project called IBM Cloud Platform Common Services, and could be installed with other Services using operand-deployment-lifecycle-manager

• If You have OpenShift Container Platform with Version 4.2+ install with this
• Otherwise on you can install it with this

Install the IBM Licensing Operator On OCP 4.2+

1. Create OperatorSource

Before install IBM Licensing Operator, this operator source should be created to get operator bundles from quay.io.

apiVersion: operators.coreos.com/v1
kind: OperatorSource
metadata:
  name: opencloud-operators
  namespace: openshift-marketplace
spec:
  authorizationToken: {}
  displayName: IBMCS Operators
  endpoint: https://quay.io/cnr
  publisher: IBM
  registryNamespace: opencloudio
  type: appregistry

Click the plus button, and then copy the above operator source into the editor.

2. Create a Namespace ibm-common-services

Open the OperatorHub page in OCP console left menu, then Create Project named ibm-common-services.

3. Install IBM Licensing Operator Package in the OperatorHub

Open OperatorHub and search IBM Licensing Operator to find the operator, and install it by clicking install.

Select specific namespace ibm-common-services that was created in step Create Project and click subscribe.

Now after waiting about 1 minute and clicking installed operators you should see IBM Licensing Operator and it should have status InstallSucceeded

Install the IBM Licensing Operator on Kubernetes from scratch

1. Install Operator Lifecycle Manager

Make sure You are connected to your cluster f.e. using:

kubectl get node
# check if this shows your nodes

Download OLM Release from here For versions newer than 13.0 process might differ, here is script to install OLM v13.0:

curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.13.0/install.sh | bash -s 0.13.0

In case you have any error you might have old kubernetes version. You can try either upgrading your kubernetes server version or using older version of OLM.

2. Install the Operator Marketplace

Clone the following github repo:

git clone --single-branch --branch release-4.6 https://github.com/operator-framework/operator-marketplace.git

Then we need to change marketplace namespace to olm in order to be able to create subscriptions to our operatorsource/catalogsource from different namespace. In case your cluster's global catalog namespace is different than olm, change it below. You can check it at OLM packageserver* pod yaml somewhere in your cluster, using grep global-namespace.

For linux users:

export GLOBAL_CATALOG_NAMESPACE=olm
# change all resources namespace to olm
sed -i 's/namespace: .*/namespace: '"${GLOBAL_CATALOG_NAMESPACE}"'/g' operator-marketplace/deploy/upstream/*
# change namespace to olm
sed -i 's/name: .*/name: '"${GLOBAL_CATALOG_NAMESPACE}"'/g' operator-marketplace/deploy/upstream/01_namespace.yaml

For MAC users:

export GLOBAL_CATALOG_NAMESPACE=olm
sed -i "" 's/namespace: .*/namespace: '"${GLOBAL_CATALOG_NAMESPACE}"'/g' operator-marketplace/deploy/upstream/*
sed -i "" 's/name: .*/name: '"${GLOBAL_CATALOG_NAMESPACE}"'/g' operator-marketplace/deploy/upstream/01_namespace.yaml

Install Operator Marketplace into the cluster in the $GLOBAL_CATALOG_NAMESPACE namespace:

kubectl apply -f operator-marketplace/deploy/upstream

If you get unknown field "preserveUnknownFields" error, try to delete preserveUnknownFields from yaml files inside operator-marketplace/deploy/upstream/ catalog or consider upgrading kubernetes server version:

For linux users:

sed -i '/.*preserveUnknownFields.*/d' operator-marketplace/deploy/upstream/*
kubectl apply -f operator-marketplace/deploy/upstream

For MAC users:

sed -i "" '/.*preserveUnknownFields.*/d' operator-marketplace/deploy/upstream/*
kubectl apply -f operator-marketplace/deploy/upstream

3. Create the OperatorSource

An OperatorSource object is used to define the external datastore we are using to store operator bundles. More information including example can be found in the documentation included in the operator-marketplace repository.

Before install IBM Licensing Operator, this operator source should be created to get operator bundles from quay.io.

cat <<EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1
kind: OperatorSource
metadata:
  name: opencloud-operators
  namespace: $GLOBAL_CATALOG_NAMESPACE
spec:
  authorizationToken: {}
  displayName: IBMCS Operators
  endpoint: https://quay.io/cnr
  publisher: IBM
  registryNamespace: opencloudio
  type: appregistry
EOF

The operator-marketplace controller should successfully process this object (status should have value Succeeded):

$ kubectl get operatorsource opencloud-operators -n $GLOBAL_CATALOG_NAMESPACE
NAME                  TYPE          ENDPOINT              REGISTRY      DISPLAYNAME       PUBLISHER   STATUS      MESSAGE                                       AGE
opencloud-operators   appregistry   https://quay.io/cnr   opencloudio   IBMCS Operators   IBM         Succeeded   The object has been successfully reconciled   1m32s

Additionally, a CatalogSource is created in the $GLOBAL_CATALOG_NAMESPACE namespace:

$ kubectl get catalogsource -n $GLOBAL_CATALOG_NAMESPACE
NAME                           DISPLAY                        TYPE   PUBLISHER   AGE
opencloud-operators            IBMCS Operators                grpc   IBM         20m
[...]

If everything goes well You should see these pods:

$ kubectl get pod -n $GLOBAL_CATALOG_NAMESPACE
NAME                                            READY   STATUS    RESTARTS   AGE
marketplace-operator-6576b4ddc8-dqcgr           1/1     Running   0          84s
opencloud-operators-66df4d97ff-4rhjj            1/1     Running   0          80s
upstream-community-operators-7ffb6b674b-7qlvx   1/1     Running   0          80s
[...]

If something seems wrong check troubleshoot

4. View Available Operators

Once the OperatorSource and CatalogSource are deployed, the following command can be used to list the available operators, this should include ibm-licensing-operator-app

The command below assumes opencloud-operators as the name of the OperatorSource object. Adjust accordingly.

$ kubectl get opsrc opencloud-operators -o=custom-columns=NAME:.metadata.name,PACKAGES:.status.packages -n $GLOBAL_CATALOG_NAMESPACE
NAME                  PACKAGES
opencloud-operators   ibm-meta-operator-bridge-app,ibm-commonui-operator-app,ibm-catalog-ui-operator-app,ibm-metering-operator-app,ibm-helm-repo-operator-app,ibm-iam-operator-app,ibm-elastic-stack-operator-app,ibm-monitoring-exporters-operator-app,ibm-monitoring-prometheusext-operator-app,cp4foobar-operator-app,ibm-healthcheck-operator-app,ibm-platform-api-operator-app,ibm-management-ingress-operator-app,ibm-helm-api-operator-app,ibm-licensing-operator-app,ibm-ingress-nginx-operator-app,ibm-monitoring-grafana-operator-app,ibm-auditlogging-operator-app,operand-deployment-lifecycle-manager-app,ibm-mgmt-repo-operator-app,ibm-mongodb-operator-app,ibm-cert-manager-operator-app

5. Create an OperatorGroup

An OperatorGroup is used to denote which namespaces your Operator should be watching. It must exist in the namespace where your operator should be deployed, we'll use ibm-common-services. First create namespace for IBM Licensing Operator:

kubectl create namespace ibm-common-services

Deploy the OperatorGroup resource:

cat <<EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
  name: operatorgroup
  namespace: ibm-common-services
spec:
  targetNamespaces:
  - ibm-common-services
EOF

6. Create a Subscription

The last piece ties together all of the previous steps. A Subscription is created to the operator. Create the Subscription using:

cat <<EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: ibm-licensing-operator-app
  namespace: ibm-common-services
spec:
  channel: stable-v1
  name: ibm-licensing-operator-app
  source: opencloud-operators
  sourceNamespace: $GLOBAL_CATALOG_NAMESPACE
EOF

7. Verify Operator health

Watch IBM Licensing Operator being deployed by OLM from the catalog source created by Operator Marketplace with the following command:

$ kubectl get clusterserviceversion -n ibm-common-services
NAME                            DISPLAY                  VERSION   REPLACES                        PHASE
ibm-licensing-operator.v1.0.0   IBM Licensing Operator   1.0.0     ibm-licensing-operator.v0.0.0   Succeeded

The above command assumes you have created the Subscription in the ibm-common-services namespace. If your Operator deployment (CSV) shows a Succeeded in the InstallPhase status, your Operator is deployed successfully. If that's not the case check the ClusterServiceVersion objects status for details. Optional also check your Operator's deployment:

kubectl get deployment -n ibm-common-services | grep ibm-licensing-operator

Using IBM Licensing Service

After you successfully installed IBM Licensing Operator you can create IBMLicensing instance that will make IBM Licensing Service running in cluster.

Create instance on OCP 4.2+

If you have OCP 4.2+ you can create the instance from the OCP UI->Installed Operators->IBM Licensing Operator->IBM Licensing tab->Create IBMLicensing

This should look like this:

Then after clicking Create IBMLicensing you can edit your parameters:

Make sure to use metering datasource if IBM Metering Service exits in your cluster. Otherwise you need to change datasource to datacollector:

More about parameters from IBMLicensing instance are described here: IBMLicensingOperatorParameters

If you are sure about your parameters click create. Later if you want to edit your instance go to: OCP UI->Administration->Custom Resource Definitions->select IBMLicensing->instances->Edit IBMLicensing

In case instance is not updated properly try deleting the instance and creating new one with new parameters.

Create instance from console

First edit your custom resource at deploy/crds/operator.ibm.com_v1alpha1_ibmlicensing_cr.yaml

Make sure to use metering datasource if IBM Metering Service exits in your cluster. Otherwise you need to change datasource to datacollector More about parameters from IBMLicensing instance are described here: IBMLicensingOperatorParameters

Apply it using:

kubectl apply -f deploy/crds/operator.ibm.com_v1alpha1_ibmlicensing_cr.yaml

Check Components

First check if pod is created. You can see the logs at OCP UI->Workloads->Pods->search for licensing in ibm-common-services project:

Check if pod is running, for further investigating you can click it to check logs, and events.

From console you can use:

podName=`kubectl get pod -n ibm-common-services -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}" | grep ibm-licensing-service-instance`
kubectl logs $podName -n ibm-common-services
kubectl describe pod $podName -n ibm-common-services

You can also check OCP UI->Networking->Service,Route*

* or Ingress if you changed instance parameters

From console you can use for example this to check Ingress:

kubectl get ingress -n ibm-common-services -o yaml

Show and use API

First you need to get your URL from Route or Ingress, if you use default options the route should be created: Go to OCP UI->Networking->Route and search for ibm-licensing-service-instance in project ibm-common-services:

Here you can just click the Location and you should be redirected to IBM Licensing Service API.

To use API you need to get token, you can get it at OCP UI->Workloads->Secret->ibm-licensing-token. Click ibm-licensing-token and get token from Data token field:

You can copy it from console using:

kubectl get secret -n ibm-common-services ibm-licensing-token -o jsonpath="{.data.token}" | base64 --decode | grep ""

If everything works you can see your IBM Licensing Service UI at URL from your Ingress or Route:

Uninstall

Uninstall from any kubernetes cluster

These steps assume you have deployed IBM Licensing Service in ibm-common-services namespace

1. Delete IBMLicensing custom resource

Delete instance after which operator should clean its resources. Check what ibmlicensing instances you have:

licensingNamespace=ibm-common-services
kubectl get ibmlicensing -n ${licensingNamespace} -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}"

Above command should return one instance. Delete it if exists:

licensingNamespace=ibm-common-services
instanceName=`kubectl get ibmlicensing -n ${licensingNamespace} -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}"`
kubectl delete ibmlicensing ${instanceName} -n ${licensingNamespace}

2. Delete subscription

Show subscriptions with:

licensingNamespace=ibm-common-services
kubectl get subscription -n ${licensingNamespace} -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}"

Delete ibm-licensing-operator-app subscription:

licensingNamespace=ibm-common-services
subName=ibm-licensing-operator-app
kubectl delete subscription ${subName} -n ${licensingNamespace}

3. Delete CSV

Delete CSV that manages Operator image. Get your CSV name, look for ibm-licensing-operator:

licensingNamespace=ibm-common-services
kubectl get clusterserviceversion -n ${licensingNamespace} -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}"
kubectl get clusterserviceversion -n ${licensingNamespace} -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}" | grep ibm-licensing-operator

Delete it:

licensingNamespace=ibm-common-services
csvName=`kubectl get clusterserviceversion -n ${licensingNamespace} -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}" | grep ibm-licensing-operator`
kubectl delete clusterserviceversion ${csvName} -n ${licensingNamespace}

4. Delete CRD

Delete custom resource definition

kubectl delete CustomResourceDefinition ibmlicensings.operator.ibm.com

5. Delete OperatorGroup

If you also have other subscriptions tied with that operatorGroup do not delete it. IBM Licensing Operator should be uninstalled now but you can also cleanup operatorgroup you created for subscription:

licensingNamespace=ibm-common-services
operatorGroupName=operatorgroup
kubectl delete OperatorGroup ${operatorGroupName} -n ${licensingNamespace}

6. Delete OperatorSource

If you have other services using opencloudio catalog source do not delete the OperatorSource. Otherwise if you want to delete it use:

export GLOBAL_CATALOG_NAMESPACE=olm
opencloudioSourceName=opencloud-operators
kubectl delete OperatorSource ${opencloudioSourceName} -n ${GLOBAL_CATALOG_NAMESPACE}

7. Delete OperatorMarketplace

WARNING!!! Make sure you are not using it elsewhere.

export GLOBAL_CATALOG_NAMESPACE=olm
kubectl delete -f operator-marketplace/deploy/upstream

8. Uninstall OLM

WARNING!!! Make sure you are not using it elsewhere.

export GLOBAL_CATALOG_NAMESPACE=olm
kubectl delete crd clusterserviceversions.operators.coreos.com \
installplans.operators.coreos.com \
subscriptions.operators.coreos.com \
catalogsources.operators.coreos.com \
operatorgroups.operators.coreos.com
kubectl delete namespace ${GLOBAL_CATALOG_NAMESPACE}

Troubleshoot

CreateContainerConfigError Marketplace Operator error

If something goes wrong while installing operator-marketplace look for pods in marketplace. If you see error like this:

$ kubectl get pod -n $GLOBAL_CATALOG_NAMESPACE
NAME                                    READY   STATUS                       RESTARTS   AGE
marketplace-operator-7d4c5bdb5-mxsj6    0/1     CreateContainerConfigError   0          1m36s

You can check what the problem is using yaml (using your pod name)

kubectl get pod marketplace-operator-7d4c5bdb5-mxsj6 -n $GLOBAL_CATALOG_NAMESPACE -o yaml

If it will show error like container has runAsNonRoot and image has non-numeric user (marketplace-operator), cannot verify user is non-root in status, You can fix it by adding securityContext to operator-marketplace/deploy/upstream:

vim operator-marketplace/deploy/upstream/08_operator.yaml

Then append lines mentioned here:

...
      containers:
        - name: marketplace-operator
          securityContext: # <- this
            runAsUser: 65534 # <- and this
          image: quay.io/openshift/origin-operator-marketplace:latest
...

And then apply the configuration:

kubectl apply -f operator-marketplace/deploy/upstream/08_operator.yaml