bosh-google-cpi/

README

BOSH Google Compute Engine CPI

This is an external BOSH CPI for Google Compute Engine that is jointly developed by Pivotal and Google.

Usage

Deployment

This is the implementation of the CPI, and is part of the BOSH Google CPI release repository. See the README at the root of this repository for instructions on deploying the release with this CPI.

Installation

The source for this CPI is not intended to be deployed except as a BOSH deployment and is not go getable. To build or install the CPI locally for development or test purposes, you can symlink the repository into your Go workspace.

From the root of the bosh-google-cpi-boshrelease repository:

ln -s $(pwd)/src/bosh-google-cpi $GOPATH/src/

You can now go build or go install the "main" package.

Configuration

Create a configuration file:

{
  "cloud": {
    "plugin": "google",
    "properties": {
      "google": {
        "project": "my-gce-project",
        "json_key": "{\"private_key_id\": \"...\"}",
        "default_root_disk_size_gb": 20,
        "default_root_disk_type": ""
      },
      "agent": {
        "mbus": "https:\/\/mbus:mbus@0.0.0.0:6868",
        "ntp": [
          "169.254.169.254"
        ],
        "blobstore": {
          "provider": "local",
          "options": {
          }
        }
      },
      "registry": {
        "use_gce_metadata": true
      }
    }
  }
}

Option	Required	Type	Description
google.project	Y	String	Google Compute Engine Project
google.json_key	N	String	Contents of the Google Compute Engine JSON file. Only required if you are not running the CPI inside a Google Compute Engine VM with compute and devstorage.full_control service scopes and/or the Google Cloud SDK has not been initialized
google.default_root_disk_size_gb	N	Integer	The default size (in Gb) of the instance root disk (default is 10Gb)
google.default_root_disk_type	N	String	The name of the default Google Compute Engine Disk Type the CPI will use when creating the instance root disk
actions.agent.mbus.endpoint	Y	String	BOSH Message Bus URL used by deployed BOSH agents
actions.agent.ntp	Y	Array<String>	List of NTP servers used by deployed BOSH agents
actions.agent.blobstore.type	Y	String	Provider type for the BOSH Blobstore used by deployed BOSH agents (e.g. dav, s3)
actions.agent.blobstore.options	Y	Hash	Options for the BOSH Blobstore used by deployed BOSH agents
actions.registry.protocol	Y	String	BOSH Registry Protocol (http or https)
actions.registry.host	Y	String	BOSH Registry Host
actions.registry.port	Y	Integer	BOSH Registry port
actions.registry.username	Y	String	BOSH Registry username
actions.registry.password	Y	String	BOSH Registry password
actions.registry.tls.insecure_skip_verify	When https	Boolean	Skip BOSH Registry server's certificate chain and host name verification
actions.registry.tls.certfile	When https	String	BOSH Registry Client Certificate (PEM format) file location
actions.registry.tls.keyfile	When https	String	BOSH Registry Client Key (PEM format) file location
actions.registry.tls.cacertfile	When https	String	BOSH Registry Client Root CA certificates (PEM format) file location

Run

Run the cpi using the previously created configuration file:

$ echo "{\"method\": \"method_name\", \"arguments\": []}" | cpi -configFile="/path/to/configuration_file.json"

Features

BOSH Network options

The BOSH Google Compute Engine CPI supports these BOSH Networks Types:

Type	Description
manual	To use manually- or BOSH-assigned private IPs
dynamic	To use DHCP-assigned private IPs from Google Compute Engine
vip	To use previously allocated Google Compute Engine Static IPs

These options are specified under cloud_properties at the networks section of a BOSH deployment manifest and are only valid for manual or dynamic networks:

Option	Required	Type	Example	Description
network_name	N	String	cf	The name of the Google Compute Engine Network the CPI will use when creating the instance (if not set, by default it will use the default network)
xpn_host_project_id	N	String	my-other-project	The project id that owns the network resource to support Shared VPC Networks (XPN) (if not set, it will default to the project hosting the compute resources)
subnetwork_name	N	String	cf-east	The name of the Google Compute Engine Subnet Network the CPI will use when creating the instance. If the network is in legacy mode, do not provide this property. If the network is in auto subnet mode, providing the subnetwork is optional. If the network is in custom subnet mode, then this field is required.
ephemeral_external_ip	N	Boolean	false	If instances must have an ephemeral external IP (false by default). Can be overridden in resource_pools.
ip_forwarding	N	Boolean	false	If instances must have IP forwarding enabled (false by default). Can be overridden in resource_pools.
tags	N	Array<String>	["foo","bar"]	A list of tags to apply to the instances, useful if you want to apply firewall or routes rules based on tags. Will be merged with tags in resource_pools.

BOSH Resource pool options

These options are specified under cloud_properties at the resource_pools section of a BOSH deployment manifest:

Option	Required	Type	Example	Description
machine_type	Y	String	n1-standard-1	The name of the Google Compute Engine Machine Type the CPI will use when creating the instance (required if not using cpu and ram)
cpu	Y	Integer	2	Number of vCPUs (Google Compute Engine Custom Machine Types) the CPI will use when creating the instance (required if not using machine_type)
ram	Y	Integer	2048	Amount of memory in MBs (Google Compute Engine Custom Machine Types) the CPI will use when creating the instance (required if not using machine_type)
zone	N	String	us-west1-a	The name of the Google Compute Engine Zone where the instance must be created
root_disk_size_gb	N	Integer	10	The size (in Gb) of the instance root disk (default is 10Gb)
root_disk_type	N	String	pd-standard	The name of the Google Compute Engine Disk Type the CPI will use when creating the instance root disk
automatic_restart	N	Boolean	false	If the instances should be restarted automatically if they are terminated for non-user-initiated reasons (false by default)
on_host_maintenance	N	String	MIGRATE	Instance behavior on infrastructure maintenance that may temporarily impact instance performance (supported values are MIGRATE (default) or TERMINATE)
preemptible	N	Boolean	false	If the instances should be preemptible (false by default)
service_account	N	String	service-account-name@project-name.iam.gserviceaccount.com	The full service account address of the service account to launch the VM with. If a value is provided, service_scopes will default to https://www.googleapis.com/auth/cloud-platform unless it is explicitly set. See service account permissions for more details. To use the default service account, leave this field empty and specify service_scopes.
service_scopes	N	Array<String>	cloud-platform	If this value is specified and service_account is empty, default will be used for service_account. This value supports both short (e.g., cloud-platform) and fully-qualified (e.g., https://www.googleapis.com/auth/cloud-platform formats. See Authorization scope names for more details.
target_pool	N	String	cf-router	The name of the Google Compute Engine Target Pool the instances should be added to
backend_service	N	String OR Map<String,String>	cf-router (external), {name: "cf-internal", scheme: "INTERNAL"} (internal)	The name of the Google Compute Engine Backend Service the instances should be added to. The backend service must already be configured with an Instance Group in the same zone as this instance. To set up Internal Load Balancing use a map and set scheme to INTERNAL and name to the name of the backend service.
ephemeral_external_ip	N	Boolean	false	Overrides the equivalent option in the networks section
ip_forwarding	N	Boolean	false	Overrides the equivalent option in the networks section
tags	N	Array<String>	["foo","bar"]	Merged with tags from the networks section
labels	N	Map<String,String>	{"foo":"bar"}	A dictionary of (key,value) labels applied to the VM

BOSH Persistent Disks options

These options are specified under cloud_properties at the disk_pools section of a BOSH deployment manifest:

Option	Required	Type	Description
type	N	String	The name of the Google Compute Engine Disk Type

Deployment Manifest Example - Dynamic Networking

This is an example of how Google Compute Engine CPI specific properties are used in a BOSH deployment manifest with dynamic networking:

---
name: example
director_uuid: 38ce80c3-e9e9-4aac-ba61-97c676631b91

# ...

networks:
  - name: private
    type: dynamic
    dns:
      - 8.8.8.8
      - 8.8.4.4
    cloud_properties:
      network_name: default
      subnetwork_name: my-subnetwork
      ephemeral_external_ip: false
      ip_forwarding: false
      target_pool: my-load-balancer
      tags:
        - bosh

  - name: public
    type: vip
    cloud_properties: {}

resource_pools:
  - name: vms
    network: private
    stemcell:
      name: bosh-google-kvm-ubuntu-trusty-go_agent
      version: latest
    cloud_properties:
      instance_type: n1-standard-2
      zone: us-central1-a
      root_disk_size_gb: 20
      root_disk_type: pd-ssd
      automatic_restart: false
      on_host_maintenance: MIGRATE
      service_scopes:
        - compute.readonly
        - devstorage.read_write


disk_pools:
  - name: disks
    disk_size: 32_768
    cloud_properties:
      type: pd-ssd

Deployment Manifest Example - Manual Networking

This is an example of how Google Compute Engine CPI specific properties are used in a BOSH deployment manifest with manual networking. This assumes you've created a networked named custom-network and a subnetwork named custom-subnetwork with a CIDR of 10.0.0.0/24:

---
name: example
director_uuid: 38ce80c3-e9e9-4aac-ba61-97c676631b91

...

networks:
  - name: private
      type: manual
      subnets:
      - range: 10.0.0.0/24
        gateway: 10.0.0.1
        static: [10.0.0.2-10.0.0.100]
        cloud_properties:
          network_name: custom-network
          subnetwork_name: custom-subnetwork
          ephemeral_external_ip: true
          tags:
            - bosh

  - name: public
    type: vip
    cloud_properties: {}
...

resource_pools:
  - name: vms
    network: private
    stemcell:
      name: bosh-google-kvm-ubuntu-trusty-go_agent
      version: latest
    cloud_properties:
      instance_type: n1-standard-2
      zone: us-central1-a
      root_disk_size_gb: 20
      root_disk_type: pd-ssd
      automatic_restart: false
      on_host_maintenance: MIGRATE
      service_scopes:
        - compute.readonly
        - devstorage.read_write
...

disk_pools:
  - name: disks
    disk_size: 32_768
    cloud_properties:
      type: pd-ssd
...