k8s-app-engine

README

Aptomi simplifies roll-out, operation and control of complex container-based applications. Instead of dealing with thousands of individual containers, Dev and Ops can now talk services. Service-based approach enables Dev to move faster, while providing control and insights/contextual visibility to Ops.

It is particularly useful in environments with multiple teams, clouds and data centers, where intent-based management plays an important role in running large application infrastructure. Aptomi’s current focus is Kubernetes, but it's designed to work with any container runtime and container orchestration technologies.

Table of contents

• Features & Benefits
• Where Aptomi is located in the stack
• User Guide
  • Installation
  • Getting Started
    • Configuring LDAP
    • Creating k8s Clusters
    • Starting Aptomi
    • Running Examples
  • Architecture & How It Works
    • Components
    • State Enforcement
    • Language
  • Aptomi language
• Dev Guide
  • Building From Source
  • Tests & Code Validation
  • Web UI
  • How to contribute
  • Roadmap

Features & Benefits

• Easy way to deploy and manage complex applications
  • See example of a multi-service application based entirely on containers
• Run on top of any container platform
  • k8s, OpenShift (with support coming for AWS ECS, GKE, Docker Datacenter, Mesos)
• Friendly for Dev and Ops
  • Keep using your existing application templates (Helm, k8s YAMLs, ksonnet, Ansible, etc)
  • Speak services, not containers. Collaborate between orgs and rely on services published by other teams
  • Easy changes to the running code -- seconds to propagate updated parameters, relocate the whole stack to another cluster, restrict access, etc
• Continuous state enforcement
  • Desired state of all services is rendered as a system and continuously validated/enforced
  • Built-in service discovery ensures all dependencies always are up to date
• Lazy allocation of resources
  • Containers are running only when needed (i.e. when someone declared an intent to consume the corresponding service)
• Flexible rule engine. See examples of user-defined rules:
  • Production Instances get deployed to us-west, Staging Instances get deployed to us-west
  • Web and Mobile teams always share the same small flavor of Analytics service in Staging, while Healthcare team gets a dedicated high-performance instance of the same service
  • Development teams can never deploy to Production
  • Personal development instances of MyApp can only be running from 7am to 11pm and should be terminated overnight for all developers
• Insights & Contextual visibility
  • UI to understand what services are running, why they were instantiated, visualize dependencies and impact of changes. No need to deal with thousands of individual containers

Where Aptomi is located in the stack

Aptomi sits in between CI/CD and container orchestration. Being in deployment path for applications, it can apply higher-level policy rules (see examples above) and configure the underlying infrastructure components accordingly.

User Guide

Installation

The best way to install Aptomi is to download its latest release, which contains compiled server and client binaries for various platforms:

• Aptomi Server is an all-in-one binary with embedded DB store, which serves API requests, runs UI, as well as does deployment and continuous state enforcement
• Aptomi Client is a client for talking to Aptomi Server. It allows end-users of Aptomi to feed YAML files into Aptomi Server over REST API

You can run those binaries locally.

Getting Started

Configuring LDAP

Aptomi needs to be configured with user data source in order to enable UI login and make policy decisions based on users' labels/properties. It's recommended to start with LDAP, which is also required by Aptomi examples and smoke tests.

1. LDAP Server with sample users is provided in a docker container. To download and start the published LDAP server image, run:

   ./tools/demo-ldap.sh
   

2. Even though it's not required, you may want to download and install Apache Directory Studio to familiarize yourself with the user data in provided in sample LDAP server. Once installed, follow these step-by-step instructions to connect to LDAP and browse it. Use default credentials given in the manual.

Creating k8s Clusters

You need to have access to k8s cluster in order to deploy services from the provided examples. Two k8s clusters will enable you to take full advantage of Aptomi policy engine and use cluster-based rules.

1. If you don't have k8s clusters set up, follow these instructions and run the provided script to create them in Google Cloud.

   ./tools/demo-gke.sh up
   

Starting Aptomi

1. Download the latest release of Aptomi from releases. It comes with server and client binaries as well as examples directory and needed tools. Unpack it into some directory:

   TODO:
   

2. Create config for Aptomi server and start it. It will serve API and UI :

   mkdir /etc/aptomi
   sudo cp examples/config/server.yaml /etc/aptomi/config.yaml
   aptomi server
   

3. Create config for Aptomi client and make sure it can connect to the server:

   mkdir ~/.aptomi
   cp examples/config/client.yaml ~/.aptomi/config.yaml
   aptomictl -u Sam policy show
   

   You should be able to see:

   &{{policy} {1 2017-11-19 00:00:05.613151 -0800 PST aptomi} map[]}
   

4. Open Web UI and log in (use 'sam:sam', it's a user from sample LDAP server running locally in a container) http://127.0.0.1:8080/

Running Examples

Once Aptomi is up and running and k8s clusters are set up, you can get started by running the following examples:

Example	Description	Diagram
examples/01	Description of Example 1	Diagram for example #1
examples/02	Description of Example 2	Diagram for example #2
examples/03-twitter-analytics	Twitter Analytics Application, 2 k8s clusters	Diagram for example #3

Architecture & How It Works

Components

State Enforcement

Language

Aptomi language

TODO: policy documentation

Dev Guide

Building From Source

In order to build Aptomi from source you will need Go (the latest 1.9.x) and a couple of external dependencies:

• glide - all Go dependencies for Aptomi are managed via Glide
• docker - to run Aptomi in container, as well as to run sample LDAP server with user data
• kubernetes-cli and kubernetes-helm for using Kubernetes with Helm
• npm - to build UI, as well as automatically generate table of contents in README.md
• telnet - for the script which runs smoke tests

If you are on macOS, install Homebrew and Docker For Mac, then run:

brew install go glide docker kubernetes-cli kubernetes-helm npm telnet

Check out Aptomi source code from the repo:

mkdir -p $GOPATH/src/github.com/Aptomi
cd $GOPATH/src/github.com/Aptomi
git clone https://github.com/Aptomi/aptomi.git

Install Helm, Kubectl clients:

./tools/install-clients.sh

In order to build Aptomi, you must first tell Glide to fetch all of its dependencies. It will read the list of dependencies defined in glide.lock and fetch them into a local "vendor" folder. After that, you must run Go to build and install the binaries. There are convenient Makefile targets for both, run them:

make vendor 
make install

Tests & Code Validation

Command	Action	LDAP Required
make test	Unit tests	No
make alltest	Integration + Unit tests	Yes
make smoke	Smoke tests + Integration + Unit tests	Yes
make profile-engine	Profile engine for CPU usage	No
make coverage	Calculate code coverage by unit tests	No
make coverage-full	Calculate code coverage by unit & integration tests	Yes

Command	Action	Description
make fmt	Re-format code	Re-formats all code according to Go standards
make lint	Examine code	Run linters to examine Go source code and reports suspicious constructs

Web UI

Source code is available in webui

Make sure you have latest node and npm. We have tested with node v8.9.1 and npm 5.5.1 and it's known to work with these.

Command	Action
npm install	Install dependencies
npm run dev	Serve with hot reload at localhost:8080
npm run build	Build for production with minification
npm run build --report	Build for production and view the bundle analyzer report
npm run unit	Run unit tests: coming soon
npm run e2e	Run e2e tests: coming soon
npm run test	Run all tests: coming soon

How to contribute

Report a bug. Send us a pull request.

List of areas where we could use help:

• Feedback from Dev & Ops teams on service & rule definitions
• Adding support for additional cloud providers (AWS ECS, GKE, Docker Datacenter, Mesos)
• Also, see Feature Backlog

Roadmap

We will soon publish the list of items for Q4 2017 and Q1 2018. In the meantime, Feature Backlog is a good place to look at the roadmap items which are being considered.

If you have any questions, please contact us on Slack.