cloud-pubsub/

README

Service Catalog Sample - Cloud Pub/Sub

This sample demonstrates how to build a simple Kubernetes web application using the Kubernetes Service Catalog and the Google Cloud Platform Service Broker, an implementation of the Open Service Broker API.

The sample highlights a number of Kubernetes and Open Service Broker concepts:

• Using the Service Catalog and the Service Broker to provision a service instance.
• Binding the provisioned service instance to a Kubernetes application.
• Use of the service binding by the application to access the service instance.

The sample contains two web applications using Cloud Pub/Sub; it allows users to publish messages, and then to receive the published messages.

At the core of the sample is a Cloud Pub/Sub topic and a subscription to the topic accessed by different components of the sample. The Cloud Pub/Sub topic, which is a corresponding resource of Cloud Pub/Sub service instance, is provisioned by the Service Broker and the subscription is completed by the binding to the service instance. The service instance is accessed by two Kubernetes applications: a publisher web deployment and a subscriber web deployment.

The publisher web deployment uses a binding which allows it to publish messages to the Cloud Pub/Sub topic.

The subscriber web deployment uses a binding which not only creates a subscription to the topic, but also grants subscriber-level access to the Pub/Sub topic - to receive the messages from its subscribed topic.

Objectives

To deploy and run the sample application, you must:

1. Create a new Kubernetes namespace.
2. Create a Cloud Pub/Sub topic.
3. Deploy the publisher application:
   1. Create a Cloud Pub/Sub service binding for publisher.
   2. Create the Kubernetes publisher deployment.
4. Deploy the subscriber application:
   1. Provision a Cloud IAM service account instance and bind to it.
   2. Create a Cloud Pub/Sub service binding for subscriber.
   3. Create the Kubernetes subscriber deployment.
5. Use the publisher and subscriber applications.
6. Clean up.

Before you begin

Review the information applicable to all Service Catalog samples, including prerequisites and troubleshooting.

Successful use of this sample requires:

• A Kubernetes cluster, minimum version 1.8.x.
• Kubernetes Service Catalog and the Service Broker installed.
• The Service Catalog CLI (svcat) installed.
• The Cloud Pub/Sub API enabled.

Step 1: Create a New Kubernetes Namespace

kubectl create namespace pubsub

TIP: For convenience, also set the NAMESPACE environment variable in your shell: export NAMESPACE=pubsub. svcat will use this namespace by default so you don't have to use --namespace pubsub on every command below.

Step 2: Create a Cloud Pub/Sub Topic

To create a Cloud Pub/Sub topic, run:

svcat provision pubsub-instance \
    --class cloud-pubsub \
    --plan beta \
    --param topicId=comments \
    --namespace pubsub

This command will use the Kubernetes Service Catalog to provision a Cloud Pub/Sub service instance with the parameters set by --param flag. The corresponding resource is a Pub/Sub topic called "comments".

Check on the status of the service instance provisioning:

svcat get instance pubsub-instance --namespace pubsub

The service instance is provisioned when its status is Ready. You can also examine the Cloud Pub/Sub topic in the Google Cloud console.

Step 3: Deploy the Publisher Application

The publisher deployment checks the existence of the Pub/Sub topic and publishes messages to the topic. To do so, it requires permissions from the Pub/Sub publisher and viewer roles.

To express the intent of granting the permissions to publisher, you will create a service binding using the parameters in publisher-binding.yaml. Creating the service binding will:

• Create a service account to authenticate with Cloud Pub/Sub.
• Grant the service account the roles/pubsub.publisher and roles/pubsub.viewer roles.
• Store the service account private key (privateKeyData) and the Pub/Sub topic information (projectId, topicId) in a Kubernetes secret.

The publisher deployment consumes the information stored in the secret via environment variables GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_CLOUD_PROJECT, and PUBSUB_TOPIC. Review the publisher deployment configuration in publisher-deployment.yaml.

Step 3.1: Create a Cloud Pub/Sub Service Binding for Publisher

Create the publisher binding:

kubectl create -f manifests/publisher-binding.yaml

The command will use the Kubernetes Service Catalog to create a binding to the Pub/Sub service instance provisioned earlier.

Check on the status of the binding operation:

svcat get binding publisher --namespace pubsub

Once the binding status is Ready, view the Kubernetes secret containing the result of the binding (the default name of the secret is the same as the name of the binding resource - publisher).

kubectl get secret --namespace pubsub publisher -oyaml

Notice the values privateKeyData, projectId, and topicId which contain the result of the binding, ready for the publisher deployment to use.

Step 3.2: Create the Kubernetes Publisher Deployment

Create the Kubernetes deployment using configuration in publisher-deployment.yaml:

kubectl create -f ./manifests/publisher-deployment.yaml

Wait for the deployment to complete and then find the the Kubernetes service external IP address:

kubectl get service publisher-service --namespace pubsub
PUB_IP= ... # External IP address of the Kubernetes load balancer.

or:

PUB_IP=$(kubectl get service --namespace pubsub publisher-service -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')

Step 4: Deploy the Subscriber Application

The subscriber deployment uses the subscription to the Pub/Sub topic, pulls the messages published to the topic, and displays the received messages. To do so, it requires permissions from the subscriber and viewer roles.

In order to acquire the permissions, subscriber will run as a single service account with subscriber and viewer roles granted. Next, you will:

• Create a service account by provisioning a service instance for "Cloud IAM service account".
• Create a binding to the service account instance. This will:
  • Create a service account private key.
  • Store the private key in the Kubernetes secret as privateKeyData.
• Create a binding to Cloud Pub/Sub service instance, using the subscription ID and referencing the service account. This will:
  • Create a subscription to the corresponding topic of the Pub/Sub service instance.
  • Grant the service account the roles needed to subcribe to the topic and view the published messages.
  • Store the Cloud Pub/Sub subscription information (projectId, subscriptionId) in a Kubernetes secret.

The subscriber deployment consumes both secrets via environment variables GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_CLOUD_PROJECT, and PUBSUB_SUBSCRIPTION. Review the subscriber deployment configuration in subscriber-deployment.yaml.

Step 4.1: Provision a Cloud IAM Service Account Instance and Bind to It

Create a subscriber service account:

svcat provision subscriber-account \
    --class cloud-iam-service-account \
    --plan beta \
    --param accountId=pubsub-subscriber \
    --namespace pubsub

Check on the status of the service account provisioning:

svcat get instance --namespace pubsub subscriber-account

Once the status is Ready, create a binding to make the service account private key available in a Kubernetes secret:

svcat bind subscriber-account \
    --namespace pubsub \
    --name subscriber-account-credentials

Check the binding status:

svcat get binding --namespace pubsub subscriber-account-credentials

When the binding status is Ready, view the secret containing the service account credentials:

kubectl get secret --namespace pubsub subscriber-account-credentials -oyaml

Notice the privateKeyData value which contains the service account private key.

Step 4.2: Create a Cloud Pub/Sub Service Binding for Subscriber

Create the subscriber binding to the Cloud Pub/Sub service instance using the configuration in subscriber-binding.yaml:

kubectl create -f ./manifests/subscriber-binding.yaml

Check the binding status:

svcat get binding --namespace pubsub subscriber

Once the binding status is Ready, view the secret containing the result of the binding (the default name of the secret is the same as the name of the binding resource - subscriber):

kubectl get secret --namespace pubsub subscriber -oyaml

Notice the projectId and subscriptionId values. They are referenced from subscriber-deployment.yaml and tell the subscriber deployment which subscription to use.

Step 4.3: Create the Kubernetes Subscriber Deployment

Create the Kubernetes deployment using configuration in subscriber-deployment.yaml. The configuration uses two secrets to obtain service account credentials (from secret subscriber-account-credentials ) and Pub/Sub subscription projectId and subscriptionId (from secrect subscriber).

kubectl create -f ./manifests/subscriber-deployment.yaml

Wait for the deployment to complete and then find the the Kubernetes service external IP address:

kubectl get service subscriber-service --namespace pubsub
SUB_IP= ... # External IP address of the Kubernetes load balancer.

or:

SUB_IP=$(kubectl get service --namespace pubsub subscriber-service -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')

Step 5: Use the Publisher and Subscriber Applications

Use the IP addresses of the Kubernetes load balancer services in the web browser to access the application.

Enter the IP address of subscriber-service, ${SUB_IP}, into your browser. No messages have been received.

Enter the IP address of publisher-service, ${PUB_IP}, into your browser and, in the simple UI, enter messages to send to the subscriber.

Refresh the subscriber browser window - the messages have now been received.

Step 6: Clean up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this sample, delete and deprovision all resources.

An expedient way is to delete the Kubernetes namespace; however make sure that the namespace doesn't contain any resources you want to keep:

kubectl delete namespace pubsub

Alternatively, delete all resources individually by running the following commands:

Note: You may have to wait several minutes between steps to allow for the previous operations to complete.

Delete the publisher deployment and subscriber deployment. They will stop serving user traffic and will cease using the Cloud Pub/Sub topic.

kubectl delete -f ./manifests/publisher-deployment.yaml
kubectl delete -f ./manifests/subscriber-deployment.yaml

Delete the subscriber binding to the Cloud Pub/Sub service instance:

kubectl delete -f ./manifests/subscriber-binding.yaml

Delete the publisher binding to the Cloud Pub/Sub service instance. This also deletes the service account created for the publisher binding.

kubectl delete -f ./manifests/publisher-binding.yaml

Unbind the subscriber service account instance:

svcat unbind subscriber-account \
    --name subscriber-account \
    --namespace pubsub

Deprovision the subscriber service account instance:

svcat deprovision subscriber-account --namespace pubsub

Deprovision the Cloud Pub/Sub service instance:

svcat deprovision pubsub-instance --namespace pubsub

If the pubsub namespace contains no resource you wish to keep, delete it:

kubectl delete namespace pubsub

Troubleshooting

Please find the troubleshooting information here.