cortex

README

Get started: Install • Tutorial • Docs • Examples

Learn more: Website • Blog • Subscribe • Twitter • Contact

Cortex deploys your machine learning models to your cloud infrastructure. You define your deployment with simple declarative configuration, Cortex containerizes your models, deploys them as autoscaling JSON APIs, and manages their lifecycle in production.

Cortex is actively maintained by Cortex Labs. We're a venture-backed team of infrastructure engineers and we're hiring.

How it works

Define your deployment using declarative configuration:

# cortex.yaml

- kind: api
  name: my-api
  model: s3://my-bucket/my-model.zip
  request_handler: handler.py
  compute:
    min_replicas: 5
    max_replicas: 20

Customize request handling (optional):

# handler.py

def pre_inference(sample, metadata):
  # Python code


def post_inference(prediction, metadata):
  # Python code

Deploy to your cloud infrastructure:

$ cortex deploy

Deploying ...
Ready! https://amazonaws.com/my-api

Serve real time predictions via scalable JSON APIs:

$ curl -d '{"a": 1, "b": 2, "c": 3}' https://amazonaws.com/my-api

{ prediction: "def" }

Key features

• Machine learning deployments as code: Cortex deployments are defined using declarative configuration.

• Autoscaling: Cortex can automatically scale APIs to handle production workloads.

• Multi framework support: Cortex supports TensorFlow, Keras, PyTorch, Scikit-learn, XGBoost, and more.

• CPU / GPU support: Cortex can run inference on CPU or GPU infrastructure.

• Rolling updates: Cortex updates deployed APIs without any downtime.

• Cloud native: Cortex can be deployed on any AWS account in minutes.