cf-networking-release

README

netman-release

A garden-runc add-on that provides container networking.

Project links

• Design doc for Container Networking Policy
• Engineering backlog
• Chat with us at the #container-networking channel on CloudFoundry Slack
• CI pipeline and config

Deploy and run high-level acceptance test errand on bosh-lite

pushd ~/workspace
  git clone https://github.com/cloudfoundry-incubator/diego-release
  git clone https://github.com/cloudfoundry/cf-release
  git clone https://github.com/cloudfoundry-incubator/netman-release
popd

pushd ~/workspace/netman-release
  ./scripts/deploy-to-bosh-lite
popd

bosh run errand netman-cf-acceptance

Deploy to AWS

1. Upload stemcell with Linux kernel 4.4 to bosh director

• Current AWS stemcells only have 3.19
• For now you will need a special stemcell that can be found here

1. Create netman stubs

• netman requires additional information in several stubs.

• Add under properties: uaa in stubs/cf/properties.yml:

  scim:
    users:
    - admin|<admin-password>|scim.write,scim.read,openid,cloud_controller.admin,doppler.firehose
    - network-admin|<network-admin-password>|openid,network.admin
  clients:
    cf:
      scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write,doppler.firehose,uaa.user,routing.router_groups.read,network.admin
  

• Add under properties in stubs/cf/properties.yml:

  acceptance_tests:
    admin_password: <admin-password>
    admin_user: admin
    api: api.<system-domain>
    apps_domain: <apps-domain>
    nodes: 1
    skip_ssl_validation: true
    use_http: true
  

• Create a cf_creds_stub.yml

  ---
  properties:
    netman-cf-acceptance:
      admin_password: <admin-password>
      admin_user: admin
      api: api.<system-domain>
      apps_domain: <apps-domain>
      nodes: 1
      skip_ssl_validation: true
      use_http: true
        test_user_password: <test-user-password>
    uaa:
      clients:
        network-policy:
          secret: <uaa-client-secret>
    policy-server:
      database_password: <db-password>
  

1. Generate diego with netman manifest

• Run generate-deployment-manifest. Set environment_path to the directory containing your stubs for cf, diego, and netman. Set output_path to the directory you want your manifest to be created in. Set diego_release_path to your local copy of the diego-release repository.

set -e -x -u

environment_path=
output_path=
diego_release_path=

pushd cf-release
  ./scripts/generate_deployment_manifest aws \
    ${environment_path}/stubs/director-uuid.yml \
    ${diego_release_path}/examples/aws/stubs/cf/diego.yml \
    ${environment_path}/stubs/cf/properties.yml \
    ${environment_path}/stubs/cf/instance-count-overrides.yml \
    ${environment_path}/stubs/cf/stub.yml \
    > ${output_path}/cf.yml
popd

pushd diego-release
  ./scripts/generate-deployment-manifest \
    -g \
    -c ${output_path}/cf.yml \
    -i ${environment_path}/stubs/diego/iaas-settings.yml \
    -p ${environment_path}/stubs/diego/property-overrides.yml \
    -n ${environment_path}/stubs/diego/instance-count-overrides.yml \
    -v ${environment_path}/stubs/diego/release-versions.yml \
    > ${output_path}/diego0.yml
popd

sed 's/\ guardian/\ garden-runc/' < ${output_path}/diego0.yml > ${output_path}/diego1.yml

pushd netman-release
  ./scripts/netmanify \
    ${output_path}/diego1.yml \
    ${environment_path}/stubs/netman/cf_creds_stub.yml \
    ${environment_path}/stubs/cf/stub.yml \
    > ${output_path}/diego.yml
popd

1. Deploy

• Target your bosh director.

bosh target <your-director>

• Set the deployment

bosh deployment ${output_path}/diego.yml

• Deploy

bosh deploy

1. Run the acceptance errand

bosh run errand netman-cf-acceptance

Kicking the tires on the policy server

cf auth network-admin network-admin

# list policies
cf curl /networking/v0/external/policies

# create a new policy
cf curl -X POST /networking/v0/external/policies -d '{ "policies": [ {"source": { "id": "some-app-guid" }, "destination": { "id": "some-other-app-guid", "protocol": "tcp", "port": 8080 } } ] }'

# delete that policy
cf curl -X DELETE /networking/v0/external/policies -d '{ "policies": [ {"source": { "id": "some-app-guid" }, "destination": { "id": "some-other-app-guid", "protocol": "tcp", "port": 8080 } } ] }'

Using your own CNI plugin

1. Remove the following BOSH jobs:

• cni-flannel
• netman-agent

1. Remove the following BOSH packages:

• flannel
• flannel-watchdog
• netman-agent

1. Add in all packages and jobs required by your CNI plugin. At a minimum, you must provide a CNI binary program and a CNI config file.

• For more info on bosh packaging scripts read this.
• For more info on bosh jobs read this.

1. Update the deployment manifest properties

garden-cni:
  adapter:
    cni_plugin_dir: /var/vcap/packages/YOUR_PACKAGE/bin # your CNI binary goes in this directory
    cni_config_dir: /var/vcap/jobs/YOUR_JOB/config/cni  # your CNI config file goes in this directory

Remove any lingering references to flannel or cni-flannel in the deployment manifest.

Development

Running low-level tests

~/workspace/netman-release/scripts/docker-test

Referencing a new library from existing BOSH package

1. Add any new libraries into the submodule from the root of the repo

cd $GOPATH
git submodule add https://github.com/foo/bar src/github.com/foo/bar
./scripts/sync-package-specs

Adding a new BOSH package

1. Add any new libraries into the submodules from the root of the repo

cd $GOPATH
git submodule add https://github.com/foo/bar src/github.com/foo/bar

1. Update the package sync script:

vim $GOPATH/scripts/sync-package-specs

Find or create the sync_package line for baz

1. Run the sync script:

./scripts/sync-package-specs