bullion

README

bullion

A golang implementation of the bull library.

Documentation

Index

• Constants
• func DoAddJob(job *Job, lifo bool) (string, error)
• func DoGetJob(id string) (*map[string]string, error)
• type Bullion
  • func New(options *ConnectOptions) *Bullion
  • func (bull *Bullion) AddJob(job *Job) error
  • func (bull *Bullion) Close() error
  • func (bull *Bullion) GetKey(segment string) string
  • func (bull *Bullion) GetKeyPrefix() string
  • func (bull *Bullion) JobFromData(id string, data map[string]string) (*Job, error)
• type ConnectOptions
• type Job
  • func (job *Job) GetLockKey() string
  • func (job *Job) Serialize() (map[string]string, error)
• type JobOptions

Constants

const (
	// BullionKeyWait is the key name of the wait list
	BullionKeyWait = "wait"
	// BullionKeyActive is the key name of the active list
	BullionKeyActive = "active"
	// BullionKeyDelayed is the key name of the delayed zset
	BullionKeyDelayed = "delayed"
	// BullionKeyCompleted is the key name of the completed set
	BullionKeyCompleted = "completed"
	// BullionKeyFailed is the key name of the failed set
	BullionKeyFailed = "failed"
)

const (
	// BullionJobBackoffNone is the value for no backoff
	BullionJobBackoffNone = "none"
	// BullionJobBackoffFixed is the value for fixed backoff
	BullionJobBackoffFixed = "fixed"
	// BullionJobBackoffExponential is the value for exponential backoff
	BullionJobBackoffExponential = "exponential"
)

const BullionKeyJobs = "jobs"

BullionKeyJobs is the key name of the jobs channel

const BullionKeyJobsCount = "id"

BullionKeyJobsCount is the key for the jobs count

const BullionPrefixDefault = "bull"

BullionPrefixDefault is the default key prefix

Functions

func DoAddJob

func DoAddJob(job *Job, lifo bool) (string, error)

DoAddJob adds a job to the queue via lua script

Frist part of the script, we record the job data

```lua // Increment the global counter, and use it as job id local jobId = redis.call("INCR", #{KEY_COUNTER}) // Use HMSET to store the job data, mapped to its key redis.call("HMSET", #{KEY_PREFIX} .. jobId, #{JOB_FIELD_1}, #{JOB_VALUE_1}, ...) ```

Second part depends on the nature of the job. If a job scheduled for immediate execution, then it's to be added to the wait list directly:

```lua // Use LPUSH (or RPUSH if LIFO) to push job to wait list redis.call("LPUSH", #{KEY_QUEUE_WAIT}, jobId) // Publish an event to signal the job is added redis.call("PUBLISH", #{KEY_JOBS}, jobId) ```

Otherwise, it should be added to the delayed set instead:

```lua // Calculate delay score local timestamp = tonumber(#{ETA_TIMESTAMP}) * 0x1000 + bit.band(jobId, 0xfff) redis.call("ZADD", #{KEY_QUEUE_DELAYED}, timestamp, jobId) // Publish an event to signal the job is added and the delayed amount of the job redis.call("PUBLISH", #{KEY_QUEUE_DELAYED}, (timestamp / 0x1000)) ```

Finally, the job id should

Note: LUA uses 1-based array index

func DoGetJob

func DoGetJob(id string) (*map[string]string, error)

DoGetJob retrieves the job data from the queue FIXME

Types

type Bullion

type Bullion struct {
	Name string // queue name
	// contains filtered or unexported fields
}

Bullion is the main bullion program

func New

func New(options *ConnectOptions) *Bullion

New creates a bullion instance

func (*Bullion) AddJob

func (bull *Bullion) AddJob(job *Job) error

AddJob adds a job into the queue

func (*Bullion) Close

func (bull *Bullion) Close() error

Close terminates all connections to the queue

func (*Bullion) GetKey

func (bull *Bullion) GetKey(segment string) string

GetKey returns the full key for a key segment

func (*Bullion) GetKeyPrefix

func (bull *Bullion) GetKeyPrefix() string

GetKeyPrefix returns the key prefix for redis

func (*Bullion) JobFromData

func (bull *Bullion) JobFromData(id string, data map[string]string) (*Job, error)

JobFromData constructs a job from the key value mapping data

type ConnectOptions

type ConnectOptions struct {
	Address string // redis address
	Auth    string // redis password
	DB      string // redis db number
	Name    string // queue name
}

ConnectOptions is the paramters for connecting to redis

type Job

type Job struct {
	ID           string
	Data         interface{}
	Opts         *JobOptions
	Progress     float64
	Delay        int64
	Timestamp    int64
	Attempts     int
	AttemptsMade int
	Backoff      string
	BackoffDelay int64
	Stacktraces  []interface{}
	ReturnValue  interface{}
	Bull         *Bullion
}

Job is the main struct for the job

func (*Job) GetLockKey

func (job *Job) GetLockKey() string

GetLockKey returns the key of the lock on the job in redis

func (*Job) Serialize

func (job *Job) Serialize() (map[string]string, error)

Serialize returns a key value mapping of the job

type JobOptions

type JobOptions struct {
	LIFO         bool      // last in first out
	ETA          time.Time // job ETA timestamp
	Backoff      string    // backoff type: none, fixed, exponential
	BackoffDelay int       // backoff delay
	Attempts     int       // maximum attempts allowed on job
}

JobOptions is the struct for job options