What is Colonies?
Colonies is a Process Orchestration framework for managing AI/ML workloads across heterogeneous computing platforms.
Colonies provides functionality to establish trusted distributed computing environments, process management, and functionality to orchestrate workloads. It is a building block to implement a Meta-Operating System, an overlay built on top of existing operating systems and platforms to create compute continuums spanning devices, webapps, clouds, and edge and HPC platforms.
The core idea of Colonies is to split up complex workloads in two layers, a Meta-layer and an Execution-layer.
- The Meta-layer makes it possible to describe and manage complex workflows as meta-processes independently of implementation and execution environment.
- The Execution-layer provides a serverless computing environment where developers can implement workers capable of executing certain types of meta-processes. AI applications can then be broken down into composable functions executed by remote workers anywhere on the Internet.
- A build-in zero-trust protocol makes it possible to organize remote workers as a single unit called a Colony, thus making it possible for users to keep control even if workloads are spread out and executed on many different platforms at the same time.
Installation
Presentations
Guides
Design
SDKs
Deployment
More information can also be found here.
Current users
- Colonies is currently being used by RockSigma AB to build a compute engine for automatic seismic processing in underground mines.
Running the tests
Follow the instructions at Installation Guide then type:
make test