cortex

README

Get started: Install • Tutorial • Docs • Examples

Learn more: Website • Blog • Subscribe • Contact

Cortex is a machine learning deployment platform that runs in your AWS account. It takes exported models from S3 and deploys them as web APIs. It also handles autoscaling, rolling updates, log streaming, inference on CPUs or GPUs, and more.

Cortex is maintained by a venture-backed team of infrastructure engineers and we're hiring.

How it works

Define your deployment using declarative configuration:

# cortex.yaml

- kind: api
  name: my-api
  model: s3://my-bucket/my-model.zip
  request_handler: handler.py

Customize request handling (optional):

# handler.py

def pre_inference(sample, metadata):
  # Python code


def post_inference(prediction, metadata):
  # Python code

Deploy to AWS:

$ cortex deploy

Deploying ...
https://amazonaws.com/my-api  # Your API is ready!

Serve real time predictions via scalable JSON APIs:

$ curl -d '{"a": 1, "b": 2, "c": 3}' https://amazonaws.com/my-api

{ prediction: "def" }

Hosting Cortex on AWS

# Download the install script
$ curl -O https://raw.githubusercontent.com/cortexlabs/cortex/master/cortex.sh && chmod +x cortex.sh

# Set your AWS credentials
$ export AWS_ACCESS_KEY_ID=***
$ export AWS_SECRET_ACCESS_KEY=***

# Provision infrastructure on AWS and install Cortex
$ ./cortex.sh install

# Install the Cortex CLI on your machine
$ ./cortex.sh install cli

Key features

• Minimal declarative configuration: Deployments can be defined in a single cortex.yaml file.

• Autoscaling: Cortex can automatically scale APIs to handle production workloads.

• Multi framework: Cortex supports TensorFlow, Keras, PyTorch, Scikit-learn, XGBoost, and more.

• Rolling updates: Cortex updates deployed APIs without any downtime.

• Log streaming: Cortex streams logs from your deployed models to your CLI.

• CPU / GPU support: Cortex can run inference on CPU or GPU infrastructure.