gocelery

README

gocelery.v2 (UNDER DEVELOPMENT)

Go Client/Server for Celery Distributed Task Queue

Usage

go get gopkg.in/gocelery/gocelery.v2

v2 Design

• Message Protocol v2 support
• structs should be initialized by caller instead of using constructor
• use context.Context

Try it out!

go run example/v2/args/worker.go

python example/v2/client_args.py

Why?

Having being involved in a number of projects migrating server from python to go, I have realized Go can help improve performance of existing python web applications. Celery distributed tasks are used heavily in many python web applications and this library allows you to implement celery workers in Go as well as being able to submit celery tasks in Go.

You can also use this library as pure go distributed task queue.

Contributing

You are more than welcome to make any contributions. Please create Pull Request for any changes.

LICENSE

The gocelery is offered under MIT license.

Documentation

Index

• type Backend
• type Broker
• type Client
• type Parser
• type Worker
  • func NewWorker(ctx context.Context, options *WorkerOptions) (*Worker, error)
  • func (w *Worker) GetTask(name string) (interface{}, error)
  • func (w *Worker) Register(name string, task interface{})
  • func (w *Worker) Start()
  • func (w *Worker) Stop()
• type WorkerOptions

Types

type Backend

type Backend interface {
	Set(string, []byte) error
	Get(string) ([]byte, error) // non-blocking only
}

Backend is interface for all backends

type Broker

type Broker interface {
	Set([]byte) error
	Get() ([]byte, error)
}

Broker is interface for all brokers

type Client

type Client struct {
	Broker  Broker
	Backend Backend
}

Client is client for running celery

type Parser

type Parser interface {
	Decode([]byte) (*protocol.TaskMessage, error)
	ContentType() string
}

Parser is interface for message parsers

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker represents distributed worker

func NewWorker

func NewWorker(ctx context.Context, options *WorkerOptions) (*Worker, error)

NewWorker creates new celery workers number of workers is set to number of cpu cores if numWorkers smaller than zero

func (*Worker) GetTask

func (w *Worker) GetTask(name string) (interface{}, error)

GetTask gets registered task

func (*Worker) Register

func (w *Worker) Register(name string, task interface{})

Register new task

func (*Worker) Start

func (w *Worker) Start()

Start starts all celery workers

func (*Worker) Stop

func (w *Worker) Stop()

Stop gracefully stops all celery workers and waits for all workers to stop

type WorkerOptions

type WorkerOptions struct {
	Broker         Broker
	Backend        Backend
	Parser         Parser
	NumWorkers     int
	MsgProtocolVer int
}

WorkerOptions define options for worker