README
¶
Aptomi simplifies roll-out, operation and control of container-based applications on k8s. It introduces a service-centric abstraction that allows Dev and Ops to collaborate asynchronously. It enables teams to create and operate services, share them across the organization, fully control their lifecycle while enforcing Ops/Governance policies. Changes and updates are executed with a goal of minimizing disruptive impact on depending services.
It is particularly useful in environments with multiple teams, clouds and data centers, where intent-based management plays an important role in running large application infrastructure. Aptomi’s current focus is Kubernetes, but it's designed to work with any container runtime and container orchestration technologies.
Demo
Short demo (5 minutes, Asciinema)
Detailed demo (13 minutes, Youtube)
Table of contents
Features & Benefits
- Deploy and manage container-based applications with ease
- Dev and Ops think applications and services, not infrastructure primitives and thousands of containers
- Built-in service discovery ensures all dependencies always are up to date
- No need to change existing application templates (Helm, Ksonnet, k8s YAMLs, etc)
- Run on k8s, OpenShift (support for AWS ECS, GKE, Docker Datacenter, Mesos is pluggable)
- Lazy allocation of resources
- Containers are only running when the corresponding service has consumers
- Continuous state enforcement
- Desired state of all services is rendered as a DAG system and continuously validated/enforced
- Changes/rules can be enforced at any time (change service parameters, relocate the whole application w/ dependencies to another cluster, restrict access, etc)
- Disruption impact of change on depending services is minimized
- Flexible rule engine. Examples:
- Production Instances get deployed to us-west, Staging Instances get deployed to us-west
- Web and Mobile teams always share the same small flavor of Analytics service in Staging, while Healthcare team gets a dedicated high-performance instance of the same service
- Development teams can never deploy to Production
- Personal development instances of MyApp can only be running from 7am to 11pm and should be terminated overnight for all developers
- Insights & Contextual visibility
- UI to explore services instances, see why services were instantiated, visualize dependencies and impact of changes
Quickstart
Installation
There are several ways to install Aptomi. We recommend Compact as a starting point, but you may pick one that suits your needs the best:
| Installation Mode | Aptomi | App Deployment | Description |
|---|---|---|---|
| Compact | Local binaries | Yes | Aptomi binaries will be installed on a local machine |
| Kubernetes | Containers on k8s | Yes | Aptomi will be deployed on k8s in container |
You can also install it in a stripped-down mode, mostly to explore concepts and look at API/UI. It will use a fake executor and thus will NOT be able to perform any app deployments:
| Installation Mode | Aptomi / UI | App Deployment | Description |
|---|---|---|---|
| Concepts | Local binaries | No | Use this only if you want get familiar with Aptomi concepts, API and UI. |
Setting up k8s Clusters
You need to have access to a k8s cluster in order to deploy services from the provided examples. Two k8s clusters (or two namespaces within the same cluster) will enable you to take full advantage of Aptomi policy engine and use rules to deploy to different clusters.
| Kubernetes Cluster | When to use | How to run |
|---|---|---|
| Your own | If you already have k8s cluster set up | TODO: is it required to install Helm? write instructions to create 2 NS |
| Google Kubernetes Engine | Useful if you have $300 in free credits @ Google | Use our script to create them |
| k8s / Minikube | Local machine with 16GB+ RAM | TODO: |
| k8s / Docker For Mac | Local machine with 16GB+ RAM | TODO: |
Running Examples
Once Aptomi server is up and k8s clusters are ready, you can get started by running the following examples:
| Example | Description | Diagram |
|---|---|---|
| twitter-analytics | Twitter Analytics Application, multiple components, 2 k8s clusters | Diagram |
More examples are coming.
Architecture & How It Works
Components
State Enforcement
Language
Dev Guide
Building From Source
In order to build Aptomi from source you will need Go (the latest 1.9.x) and a couple of external dependencies:
- glide - all Go dependencies for Aptomi are managed via Glide
- docker - to run Aptomi in container, as well as to run sample LDAP server with user data
- kubernetes-cli and kubernetes-helm for using Kubernetes with Helm
- npm - to build UI, as well as automatically generate table of contents in README.md
- telnet, jq - for the script which runs smoke tests
If you are on macOS, install Homebrew and Docker For Mac, then run:
brew install go glide docker kubernetes-cli kubernetes-helm npm telnet jq
Check out Aptomi source code from the repo:
mkdir -p $GOPATH/src/github.com/Aptomi
cd $GOPATH/src/github.com/Aptomi
git clone https://github.com/Aptomi/aptomi.git
In order to build Aptomi, you must first tell Glide to fetch all of its dependencies. It will read the list of
dependencies defined in glide.lock and fetch them into a local "vendor" folder. After that, you must run Go to
build and install the binaries. There are convenient Makefile targets for both, run them:
make vendor
make install
Tests & Code Validation
| Command | Action | LDAP Required |
|---|---|---|
make test |
Unit tests | No |
make alltest |
Integration + Unit tests | Yes |
make smoke |
Smoke tests + Integration + Unit tests | Yes |
make profile-engine |
Profile engine for CPU usage | No |
make coverage |
Calculate code coverage by unit tests | No |
make coverage-full |
Calculate code coverage by unit & integration tests | Yes |
| Command | Action | Description |
|---|---|---|
make fmt |
Re-format code | Re-formats all code according to Go standards |
make lint |
Examine code | Run linters to examine Go source code and reports suspicious constructs |
Web UI
Source code is available in webui
Make sure you have latest node and npm. We have tested with node v8.9.1 and npm 5.5.1 and it's
known to work with these.
| Command | Action |
|---|---|
npm install |
Install dependencies |
npm run dev |
Serve with hot reload at localhost:8080 |
npm run build |
Build for production with minification |
npm run build --report |
Build for production and view the bundle analyzer report |
npm run unit |
Run unit tests: coming soon |
npm run e2e |
Run e2e tests: coming soon |
npm run test |
Run all tests: coming soon |
How to contribute
Report a bug. Send us a pull request.
List of areas where we could use help:
- Feedback from Dev & Ops teams on service & rule definitions
- Adding support for additional cloud providers (AWS ECS, GKE, Docker Datacenter, Mesos)
- Also, see Feature Backlog
How to release
Use git tag and make release for creating new release.
- Create annotated git tag and push it to github repo. Use commit message like
Aptomi v0.1.2.
git tag -a v0.1.2
git push origin v0.1.2
-
Create GitHub API token with the
reposcope selected to upload artifacts to GitHub release page. You can create one here. This token should be added to the environment variables asGITHUB_TOKEN. -
Run
make release. It'll create everything needed and upload all artifacts to github. -
Go to https://github.com/Aptomi/aptomi/releases/tag/v0.1.2 and fix changelog / description if needed.
Roadmap
We will soon publish the list of items for Q4 2017 and Q1 2018. In the meantime, Feature Backlog is a good place to look at the roadmap items which are being considered.
If you have any questions, please contact us on .
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
aptomi
Package aptomi is an implementation of command-line interface for Aptomi server (pkg/server).
|
Package aptomi is an implementation of command-line interface for Aptomi server (pkg/server). |
|
aptomictl
Package aptomictl is an implementation of command-line interface for Aptomi client (pkg/client).
|
Package aptomictl is an implementation of command-line interface for Aptomi client (pkg/client). |
|
api
Package api implements REST API support for Aptomi, including user-level and admin-level calls.
|
Package api implements REST API support for Aptomi, including user-level and admin-level calls. |
|
config
Package config provides support for Aptomi configuration - Server config, Client config, and their common sections (API config)
|
Package config provides support for Aptomi configuration - Server config, Client config, and their common sections (API config) |
|
engine
Package engine implements core engine of Aptomi, which does policy resolution (resolve), diff calculation between states (diff), and state enforcement by applying actions (apply).
|
Package engine implements core engine of Aptomi, which does policy resolution (resolve), diff calculation between states (diff), and state enforcement by applying actions (apply). |
|
engine/actual
Package actual defines a state updater, which reacts to changes in actual state done by engine applier.
|
Package actual defines a state updater, which reacts to changes in actual state done by engine applier. |
|
engine/apply
Package apply implements state enforcer, which executes a list of actions to move from actual state to desired state, performing actual deployment of services and configuration of the underlying cloud components.
|
Package apply implements state enforcer, which executes a list of actions to move from actual state to desired state, performing actual deployment of services and configuration of the underlying cloud components. |
|
engine/apply/action
Package action defines a base action (actions get born a result of policy diff calculation), as well as context which gets passed to all actions during apply phase.
|
Package action defines a base action (actions get born a result of policy diff calculation), as well as context which gets passed to all actions during apply phase. |
|
engine/apply/action/component
Package component defines all component-specific actions, which get generated by the policy diff when component changes occur for specific component instances.
|
Package component defines all component-specific actions, which get generated by the policy diff when component changes occur for specific component instances. |
|
engine/apply/action/global
Package global defines a global post-processing action, which gets generated by the policy diff only once after processing all component instances.
|
Package global defines a global post-processing action, which gets generated by the policy diff only once after processing all component instances. |
|
engine/diff
Package diff allows Aptomi to determine the difference between actual state (running on the cloud) and desired state (that Aptomi wants to enforce), generating a list of actions to reconcile the difference.
|
Package diff allows Aptomi to determine the difference between actual state (running on the cloud) and desired state (that Aptomi wants to enforce), generating a list of actions to reconcile the difference. |
|
engine/progress
Package progress implements progress indicators, which are used during state enforcement while running actions and plugins to estimate progress and show completion percentage to the user.
|
Package progress implements progress indicators, which are used during state enforcement while running actions and plugins to estimate progress and show completion percentage to the user. |
|
engine/resolve
Package resolve is a heart of Aptomi that performs policy resolution, taking policy and service consumption intents as input, producing a set of components to be instantiated in the cloud (desired state) as output, along with their discovery parameters, code parameters, and component instance graph.
|
Package resolve is a heart of Aptomi that performs policy resolution, taking policy and service consumption intents as input, producing a set of components to be instantiated in the cloud (desired state) as output, along with their discovery parameters, code parameters, and component instance graph. |
|
errors
Package errors defines custom errors which are widely used across Aptomi code base.
|
Package errors defines custom errors which are widely used across Aptomi code base. |
|
event
Package event implements support for Aptomi Event Logs and saving them to console, memory, and external stores (BoltDB).
|
Package event implements support for Aptomi Event Logs and saving them to console, memory, and external stores (BoltDB). |
|
external
Package external implements support for all pluggable data sources which are external to Aptomi, such as users and secrets.
|
Package external implements support for all pluggable data sources which are external to Aptomi, such as users and secrets. |
|
external/secrets
Package secrets implements support for retrieving user Secrets from external sources (File).
|
Package secrets implements support for retrieving user Secrets from external sources (File). |
|
external/users
Package users implements support for retrieving Users and their labels from external sources (LDAP, File).
|
Package users implements support for retrieving Users and their labels from external sources (LDAP, File). |
|
lang
Package lang provides core constructs for describing Aptomi policy, as well as core structures for processing policy.
|
Package lang provides core constructs for describing Aptomi policy, as well as core structures for processing policy. |
|
lang/builder
Package builder provides simple and easy-to-use way to construct Aptomi Policy in the source code, primarily for unit tests.
|
Package builder provides simple and easy-to-use way to construct Aptomi Policy in the source code, primarily for unit tests. |
|
lang/expression
Package expression provides support for evaluating expressions in Aptomi, with support for caching compiled expressions.
|
Package expression provides support for evaluating expressions in Aptomi, with support for caching compiled expressions. |
|
lang/template
Package template provides support for evaluating text templates in Aptomi, with support for caching compiled templates.
|
Package template provides support for evaluating text templates in Aptomi, with support for caching compiled templates. |
|
lang/yaml
Package yaml provides support for marshalling YAML objects and loading/unmarshalling them from YAML files.
|
Package yaml provides support for marshalling YAML objects and loading/unmarshalling them from YAML files. |
|
plugin
Package plugin introduces plugins for Aptomi engine, which are responsible for component instantiation and management on different cloud providers (e.g.
|
Package plugin introduces plugins for Aptomi engine, which are responsible for component instantiation and management on different cloud providers (e.g. |
|
plugin/helm
Package helm implements support for Helm plugin, which can deploy Helm charts onto k8s clusters via Helm API.
|
Package helm implements support for Helm plugin, which can deploy Helm charts onto k8s clusters via Helm API. |
|
runtime/store/core
Package core allows to persist Aptomi engine objects into the object store.
|
Package core allows to persist Aptomi engine objects into the object store. |
|
server
Package server implements Aptomi server, which serves UI front-end, API calls, as well as does policy resolution & continuous state enforcement.
|
Package server implements Aptomi server, which serves UI front-end, API calls, as well as does policy resolution & continuous state enforcement. |
|
util
Package util contains various utility methods for working with data (slice, map, nested map, string, time) as well as system-level primitives (files, exec).
|
Package util contains various utility methods for working with data (slice, map, nested map, string, time) as well as system-level primitives (files, exec). |
|
version
Package version allows to capture and store information about Aptomi binary (version, commit, date) so that we know when and how it was built.
|
Package version allows to capture and store information about Aptomi binary (version, commit, date) so that we know when and how it was built. |
|
visualization
Package visualization implements Aptomi policy & resolution data visualization using network-based diagrams for visjs
|
Package visualization implements Aptomi policy & resolution data visualization using network-based diagrams for visjs |
Click to show internal directories.
Click to hide internal directories.