hatchet

README

A Distributed, Fault-Tolerant Task Queue

Documentation · Website · Issues

What is Hatchet?

Hatchet replaces difficult to manage legacy queues or pub/sub systems so you can design durable workloads that recover from failure and solve for problems like concurrency, fairness, and rate limiting. Instead of managing your own task queue or pub/sub system, you can use Hatchet to distribute your functions between a set of workers with minimal configuration or infrastructure:

What Makes Hatchet Great?

• ⚡️ Ultra-low Latency and High Throughput Scheduling: Hatchet is built on a low-latency queue (25ms average start), perfectly balancing real-time interaction capabilities with the reliability required for mission-critical tasks.

• ☮️ Concurrency, Fairness, and Rate Limiting: Implement FIFO, LIFO, Round Robin, and Priority Queues with Hatchet’s built-in strategies, designed to circumvent common scaling pitfalls with minimal configuration. Read Docs →

• 🔥🧯 Resilience by Design: With customizable retry policies and integrated error handling, Hatchet ensures your operations recover swiftly from transient failures. You can break large jobs down into small tasks so you can finish a run without rerunning work. Read Docs →

Enhanced Visibility and Control:

• Observability. All of your runs are fully searchable, allowing you to quickly identify issues. We track latency, error rates, or custom metrics in your run.
• (Practical) Durable Execution. Replay events and manually pick up execution from specific steps in your workflow.
• Cron. Set recurring schedules for functions runs to execute.
• One-Time Scheduling. Schedule a function run to execute at a specific time and date in the future.
• Spike Protection. Smooth out spikes in traffic and only execute what your system can handle.
• Incremental Streaming. Subscribe to updates as your functions progress in the background worker.

Example Use Cases:

• Fairness for Generative AI: Don't let busy users overwhelm your system. Hatchet lets you distribute requests to your workers fairly with configurable policies.
• Batch Processing for Document Indexing: Hatchet can handle large-scale batch processing of documents, images, and other data and resume mid-job on failure.
• Workflow Orchestration for Multi-Modal Systems: Hatchet can handle orchestrating multi-modal inputs and outputs, with full DAG-style execution.
• Correctness for Event-Based Processing: Respond to external events or internal events within your system and replay events automatically.

Quick Start

Hatchet supports your technology stack with open-source SDKs for Python, Typescript, and Go. To get started, see the Hatchet documentation here, or check out our quickstart repos:

• Go SDK Quickstart
• Python SDK Quickstart
• Typescript SDK Quickstart

Is there a managed cloud version of Hatchet?

Yes, we are offering a have a cloud version to select companies while in beta who are helping to build and shape the product. Please reach out or request access for more information.

Is there a self-hosted version of Hatchet?

Yes, instructions for self-hosting our open source docker containers can be found in our documentation. Please reach out if you're interested in support.

Issues

Please submit any bugs that you encounter via Github issues. However, please reach out on Discord before submitting a feature request - as the project is very early, we'd like to build a solid foundation before adding more complex features.

I'd Like to Contribute

See the contributing docs here, and please let us know what you're interesting in working on in the #contributing channel on Discord. This will help us shape the direction of the project and will make collaboration much easier!