Documentation ¶
Overview ¶
Package asynq provides a framework for Redis based distrubted task queue.
Asynq uses Redis as a message broker. To connect to redis, specify the connection using one of RedisConnOpt types.
redisConnOpt = asynq.RedisClientOpt{ Addr: "127.0.0.1:6379", Password: "xxxxx", DB: 3, }
The Client is used to enqueue a task.
client := asynq.NewClient(redisConnOpt) // Task is created with two parameters: its type and payload. t := asynq.NewTask( "send_email", map[string]interface{}{"user_id": 42}) // Enqueue the task to be processed immediately. res, err := client.Enqueue(t) // Schedule the task to be processed after one minute. res, err = client.Enqueue(t, asynq.ProcessIn(1*time.Minute))
The Server is used to run the task processing workers with a given handler.
srv := asynq.NewServer(redisConnOpt, asynq.Config{ Concurrency: 10, }) if err := srv.Run(handler); err != nil { log.Fatal(err) }
Handler is an interface type with a method which takes a task and returns an error. Handler should return nil if the processing is successful, otherwise return a non-nil error. If handler panics or returns a non-nil error, the task will be retried in the future.
Example of a type that implements the Handler interface.
type TaskHandler struct { // ... } func (h *TaskHandler) ProcessTask(ctx context.Context, task *asynq.Task) error { switch task.Type { case "send_email": id, err := task.Payload.GetInt("user_id") // send email //... default: return fmt.Errorf("unexpected task type %q", task.Type) } return nil }
Index ¶
- Variables
- func DefaultRetryDelayFunc(n int, e error, t *Task) time.Duration
- func GetMaxRetry(ctx context.Context) (n int, ok bool)
- func GetQueueName(ctx context.Context) (qname string, ok bool)
- func GetRetryCount(ctx context.Context) (n int, ok bool)
- func GetTaskID(ctx context.Context) (id string, ok bool)
- func NotFound(ctx context.Context, task *Task) error
- type Client
- type Config
- type ErrorHandler
- type ErrorHandlerFunc
- type Handler
- type HandlerFunc
- type LogLevel
- type Logger
- type MiddlewareFunc
- type Option
- type OptionType
- type Payload
- func (p Payload) GetBool(key string) (bool, error)
- func (p Payload) GetDuration(key string) (time.Duration, error)
- func (p Payload) GetFloat64(key string) (float64, error)
- func (p Payload) GetInt(key string) (int, error)
- func (p Payload) GetIntSlice(key string) ([]int, error)
- func (p Payload) GetString(key string) (string, error)
- func (p Payload) GetStringMap(key string) (map[string]interface{}, error)
- func (p Payload) GetStringMapBool(key string) (map[string]bool, error)
- func (p Payload) GetStringMapInt(key string) (map[string]int, error)
- func (p Payload) GetStringMapString(key string) (map[string]string, error)
- func (p Payload) GetStringMapStringSlice(key string) (map[string][]string, error)
- func (p Payload) GetStringSlice(key string) ([]string, error)
- func (p Payload) GetTime(key string) (time.Time, error)
- func (p Payload) Has(key string) bool
- func (p Payload) MarshalJSON() ([]byte, error)
- func (p Payload) String() string
- type RedisClientOpt
- type RedisClusterClientOpt
- type RedisConnOpt
- type RedisFailoverClientOpt
- type Result
- type RetryDelayFunc
- type Scheduler
- type SchedulerOpts
- type ServeMux
- func (mux *ServeMux) Handle(pattern string, handler Handler)
- func (mux *ServeMux) HandleFunc(pattern string, handler func(context.Context, *Task) error)
- func (mux *ServeMux) Handler(t *Task) (h Handler, pattern string)
- func (mux *ServeMux) ProcessTask(ctx context.Context, task *Task) error
- func (mux *ServeMux) Use(mws ...MiddlewareFunc)
- type Server
- type Task
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrDuplicateTask = errors.New("task already exists")
ErrDuplicateTask indicates that the given task could not be enqueued since it's a duplicate of another task.
ErrDuplicateTask error only applies to tasks enqueued with a Unique option.
var ErrServerStopped = errors.New("asynq: the server has been stopped")
ErrServerStopped indicates that the operation is now illegal because of the server being stopped.
var SkipRetry = errors.New("skip retry for the task")
SkipRetry is used as a return value from Handler.ProcessTask to indicate that the task should not be retried and should be archived instead.
Functions ¶
func DefaultRetryDelayFunc ¶
DefaultRetryDelayFunc is the default RetryDelayFunc used if one is not specified in Config. It uses exponential back-off strategy to calculate the retry delay.
func GetMaxRetry ¶
GetMaxRetry extracts maximum retry from a context, if any.
Return value n indicates the maximum number of times the assoicated task can be retried if ProcessTask returns a non-nil error.
func GetQueueName ¶
GetQueueName extracts queue name from a context, if any.
Return value qname indicates which queue the task was pulled from.
func GetRetryCount ¶
GetRetryCount extracts retry count from a context, if any.
Return value n indicates the number of times associated task has been retried so far.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
A Client is responsible for scheduling tasks.
A Client is used to register tasks that should be processed immediately or some time in the future.
Clients are safe for concurrent use by multiple goroutines.
func NewClient ¶
func NewClient(r RedisConnOpt) *Client
NewClient returns a new Client instance given a redis connection option.
func (*Client) Enqueue ¶
Enqueue enqueues the given task to be processed asynchronously.
Enqueue returns nil if the task is enqueued successfully, otherwise returns a non-nil error.
The argument opts specifies the behavior of task processing. If there are conflicting Option values the last one overrides others. By deafult, max retry is set to 25 and timeout is set to 30 minutes. If no ProcessAt or ProcessIn options are passed, the task will be processed immediately.
func (*Client) SetDefaultOptions ¶
SetDefaultOptions sets options to be used for a given task type. The argument opts specifies the behavior of task processing. If there are conflicting Option values the last one overrides others.
Default options can be overridden by options passed at enqueue time.
type Config ¶
type Config struct { // Maximum number of concurrent processing of tasks. // // If set to a zero or negative value, NewServer will overwrite the value // to the number of CPUs usable by the currennt process. Concurrency int // Function to calculate retry delay for a failed task. // // By default, it uses exponential backoff algorithm to calculate the delay. RetryDelayFunc RetryDelayFunc // List of queues to process with given priority value. Keys are the names of the // queues and values are associated priority value. // // If set to nil or not specified, the server will process only the "default" queue. // // Priority is treated as follows to avoid starving low priority queues. // // Example: // // Queues: map[string]int{ // "critical": 6, // "default": 3, // "low": 1, // } // // With the above config and given that all queues are not empty, the tasks // in "critical", "default", "low" should be processed 60%, 30%, 10% of // the time respectively. // // If a queue has a zero or negative priority value, the queue will be ignored. Queues map[string]int // StrictPriority indicates whether the queue priority should be treated strictly. // // If set to true, tasks in the queue with the highest priority is processed first. // The tasks in lower priority queues are processed only when those queues with // higher priorities are empty. StrictPriority bool // ErrorHandler handles errors returned by the task handler. // // HandleError is invoked only if the task handler returns a non-nil error. // // Example: // // func reportError(ctx context, task *asynq.Task, err error) { // retried, _ := asynq.GetRetryCount(ctx) // maxRetry, _ := asynq.GetMaxRetry(ctx) // if retried >= maxRetry { // err = fmt.Errorf("retry exhausted for task %s: %w", task.Type, err) // } // errorReportingService.Notify(err) // }) // // ErrorHandler: asynq.ErrorHandlerFunc(reportError) ErrorHandler ErrorHandler // Logger specifies the logger used by the server instance. // // If unset, default logger is used. Logger Logger // LogLevel specifies the minimum log level to enable. // // If unset, InfoLevel is used by default. LogLevel LogLevel // ShutdownTimeout specifies the duration to wait to let workers finish their tasks // before forcing them to abort when stopping the server. // // If unset or zero, default timeout of 8 seconds is used. ShutdownTimeout time.Duration // HealthCheckFunc is called periodically with any errors encountered during ping to the // connected redis server. HealthCheckFunc func(error) // HealthCheckInterval specifies the interval between healthchecks. // // If unset or zero, the interval is set to 15 seconds. HealthCheckInterval time.Duration }
Config specifies the server's background-task processing behavior.
type ErrorHandler ¶
An ErrorHandler handles an error occured during task processing.
type ErrorHandlerFunc ¶
The ErrorHandlerFunc type is an adapter to allow the use of ordinary functions as a ErrorHandler. If f is a function with the appropriate signature, ErrorHandlerFunc(f) is a ErrorHandler that calls f.
func (ErrorHandlerFunc) HandleError ¶
func (fn ErrorHandlerFunc) HandleError(ctx context.Context, task *Task, err error)
HandleError calls fn(ctx, task, err)
type Handler ¶
A Handler processes tasks.
ProcessTask should return nil if the processing of a task is successful.
If ProcessTask return a non-nil error or panics, the task will be retried after delay. One exception to this rule is when ProcessTask returns SkipRetry error. If the returned error is SkipRetry or the error wraps SkipRetry, retry is skipped and task will be archived instead.
func NotFoundHandler ¶
func NotFoundHandler() Handler
NotFoundHandler returns a simple task handler that returns a “not found“ error.
type HandlerFunc ¶
The HandlerFunc type is an adapter to allow the use of ordinary functions as a Handler. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler that calls f.
func (HandlerFunc) ProcessTask ¶
func (fn HandlerFunc) ProcessTask(ctx context.Context, task *Task) error
ProcessTask calls fn(ctx, task)
type LogLevel ¶
type LogLevel int32
LogLevel represents logging level.
It satisfies flag.Value interface.
const ( // DebugLevel is the lowest level of logging. // Debug logs are intended for debugging and development purposes. DebugLevel LogLevel // InfoLevel is used for general informational log messages. InfoLevel // WarnLevel is used for undesired but relatively expected events, // which may indicate a problem. WarnLevel // ErrorLevel is used for undesired and unexpected events that // the program can recover from. ErrorLevel // FatalLevel is used for undesired and unexpected events that // the program cannot recover from. FatalLevel )
type Logger ¶
type Logger interface { // Debug logs a message at Debug level. Debug(args ...interface{}) // Info logs a message at Info level. Info(args ...interface{}) // Warn logs a message at Warning level. Warn(args ...interface{}) // Error logs a message at Error level. Error(args ...interface{}) // Fatal logs a message at Fatal level // and process will exit with status set to 1. Fatal(args ...interface{}) }
Logger supports logging at various log levels.
type MiddlewareFunc ¶
MiddlewareFunc is a function which receives an asynq.Handler and returns another asynq.Handler. Typically, the returned handler is a closure which does something with the context and task passed to it, and then calls the handler passed as parameter to the MiddlewareFunc.
type Option ¶
type Option interface { // String returns a string representation of the option. String() string // Type describes the type of the option. Type() OptionType // Value returns a value used to create this option. Value() interface{} }
Option specifies the task processing behavior.
func Deadline ¶
Deadline returns an option to specify the deadline for the given task. If it reaches the deadline before the Handler returns, then the task will be retried.
If there's a conflicting Timeout option, whichever comes earliest will be used.
func MaxRetry ¶
MaxRetry returns an option to specify the max number of times the task will be retried.
Negative retry count is treated as zero retry.
func ProcessAt ¶
ProcessAt returns an option to specify when to process the given task.
If there's a conflicting ProcessIn option, the last option passed to Enqueue overrides the others.
func ProcessIn ¶
ProcessIn returns an option to specify when to process the given task relative to the current time.
If there's a conflicting ProcessAt option, the last option passed to Enqueue overrides the others.
func Queue ¶
Queue returns an option to specify the queue to enqueue the task into.
Queue name is case-insensitive and the lowercased version is used.
func Timeout ¶
Timeout returns an option to specify how long a task may run. If the timeout elapses before the Handler returns, then the task will be retried.
Zero duration means no limit.
If there's a conflicting Deadline option, whichever comes earliest will be used.
func Unique ¶
Unique returns an option to enqueue a task only if the given task is unique. Task enqueued with this option is guaranteed to be unique within the given ttl. Once the task gets processed successfully or once the TTL has expired, another task with the same uniqueness may be enqueued. ErrDuplicateTask error is returned when enqueueing a duplicate task.
Uniqueness of a task is based on the following properties:
- Task Type
- Task Payload
- Queue Name
type OptionType ¶
type OptionType int
const ( MaxRetryOpt OptionType = iota QueueOpt TimeoutOpt DeadlineOpt UniqueOpt ProcessAtOpt ProcessInOpt )
type Payload ¶
type Payload struct {
// contains filtered or unexported fields
}
Payload holds arbitrary data needed for task execution.
func (Payload) GetBool ¶
GetBool returns a boolean value if a boolean type is associated with the key, otherwise reports an error.
func (Payload) GetDuration ¶
GetDuration returns a duration value if a correct map type is associated with the key, otherwise reports an error.
func (Payload) GetFloat64 ¶
GetFloat64 returns a float64 value if a numeric type is associated with the key, otherwise reports an error.
func (Payload) GetInt ¶
GetInt returns an int value if a numeric type is associated with the key, otherwise reports an error.
func (Payload) GetIntSlice ¶
GetIntSlice returns a slice of ints if a int slice type is associated with the key, otherwise reports an error.
func (Payload) GetString ¶
GetString returns a string value if a string type is associated with the key, otherwise reports an error.
func (Payload) GetStringMap ¶
GetStringMap returns a map of string to empty interface if a correct map type is associated with the key, otherwise reports an error.
func (Payload) GetStringMapBool ¶
GetStringMapBool returns a map of string to boolean if a correct map type is associated with the key, otherwise reports an error.
func (Payload) GetStringMapInt ¶
GetStringMapInt returns a map of string to int if a correct map type is associated with the key, otherwise reports an error.
func (Payload) GetStringMapString ¶
GetStringMapString returns a map of string to string if a correct map type is associated with the key, otherwise reports an error.
func (Payload) GetStringMapStringSlice ¶
GetStringMapStringSlice returns a map of string to string slice if a correct map type is associated with the key, otherwise reports an error.
func (Payload) GetStringSlice ¶
GetStringSlice returns a slice of strings if a string slice type is associated with the key, otherwise reports an error.
func (Payload) GetTime ¶
GetTime returns a time value if a correct map type is associated with the key, otherwise reports an error.
func (Payload) MarshalJSON ¶
MarshalJSON returns the JSON encoding of payload data.
type RedisClientOpt ¶
type RedisClientOpt struct { // Network type to use, either tcp or unix. // Default is tcp. Network string // Redis server address in "host:port" format. Addr string // Username to authenticate the current connection when Redis ACLs are used. // See: https://redis.io/commands/auth. Username string // Password to authenticate the current connection. // See: https://redis.io/commands/auth. Password string // Redis DB to select after connecting to a server. // See: https://redis.io/commands/select. DB int // Dial timeout for establishing new connections. // Default is 5 seconds. DialTimeout time.Duration // Timeout for socket reads. // If timeout is reached, read commands will fail with a timeout error // instead of blocking. // // Use value -1 for no timeout and 0 for default. // Default is 3 seconds. ReadTimeout time.Duration // Timeout for socket writes. // If timeout is reached, write commands will fail with a timeout error // instead of blocking. // // Use value -1 for no timeout and 0 for default. // Default is ReadTimout. WriteTimeout time.Duration // Maximum number of socket connections. // Default is 10 connections per every CPU as reported by runtime.NumCPU. PoolSize int // TLS Config used to connect to a server. // TLS will be negotiated only if this field is set. TLSConfig *tls.Config }
RedisClientOpt is used to create a redis client that connects to a redis server directly.
func (RedisClientOpt) MakeRedisClient ¶
func (opt RedisClientOpt) MakeRedisClient() interface{}
type RedisClusterClientOpt ¶
type RedisClusterClientOpt struct { // A seed list of host:port addresses of cluster nodes. Addrs []string // The maximum number of retries before giving up. // Command is retried on network errors and MOVED/ASK redirects. // Default is 8 retries. MaxRedirects int // Username to authenticate the current connection when Redis ACLs are used. // See: https://redis.io/commands/auth. Username string // Password to authenticate the current connection. // See: https://redis.io/commands/auth. Password string // Dial timeout for establishing new connections. // Default is 5 seconds. DialTimeout time.Duration // Timeout for socket reads. // If timeout is reached, read commands will fail with a timeout error // instead of blocking. // // Use value -1 for no timeout and 0 for default. // Default is 3 seconds. ReadTimeout time.Duration // Timeout for socket writes. // If timeout is reached, write commands will fail with a timeout error // instead of blocking. // // Use value -1 for no timeout and 0 for default. // Default is ReadTimeout. WriteTimeout time.Duration // TLS Config used to connect to a server. // TLS will be negotiated only if this field is set. TLSConfig *tls.Config }
RedisClusterClientOpt is used to creates a redis client that connects to redis cluster.
func (RedisClusterClientOpt) MakeRedisClient ¶
func (opt RedisClusterClientOpt) MakeRedisClient() interface{}
type RedisConnOpt ¶
type RedisConnOpt interface { // MakeRedisClient returns a new redis client instance. // Return value is intentionally opaque to hide the implementation detail of redis client. MakeRedisClient() interface{} }
RedisConnOpt is a discriminated union of types that represent Redis connection configuration option.
RedisConnOpt represents a sum of following types:
- RedisClientOpt
- RedisFailoverClientOpt
- RedisClusterClientOpt
func ParseRedisURI ¶
func ParseRedisURI(uri string) (RedisConnOpt, error)
ParseRedisURI parses redis uri string and returns RedisConnOpt if uri is valid. It returns a non-nil error if uri cannot be parsed.
Three URI schemes are supported, which are redis:, redis-socket:, and redis-sentinel:. Supported formats are:
redis://[:password@]host[:port][/dbnumber] redis-socket://[:password@]path[?db=dbnumber] redis-sentinel://[:password@]host1[:port][,host2:[:port]][,hostN:[:port]][?master=masterName]
Example ¶
package main import ( "fmt" "log" "github.com/duke-cliff/asynq" ) func main() { rconn, err := asynq.ParseRedisURI("redis://localhost:6379/10") if err != nil { log.Fatal(err) } r, ok := rconn.(asynq.RedisClientOpt) if !ok { log.Fatal("unexpected type") } fmt.Println(r.Addr) fmt.Println(r.DB) }
Output: localhost:6379 10
type RedisFailoverClientOpt ¶
type RedisFailoverClientOpt struct { // Redis master name that monitored by sentinels. MasterName string // Addresses of sentinels in "host:port" format. // Use at least three sentinels to avoid problems described in // https://redis.io/topics/sentinel. SentinelAddrs []string // Redis sentinel password. SentinelPassword string // Username to authenticate the current connection when Redis ACLs are used. // See: https://redis.io/commands/auth. Username string // Password to authenticate the current connection. // See: https://redis.io/commands/auth. Password string // Redis DB to select after connecting to a server. // See: https://redis.io/commands/select. DB int // Dial timeout for establishing new connections. // Default is 5 seconds. DialTimeout time.Duration // Timeout for socket reads. // If timeout is reached, read commands will fail with a timeout error // instead of blocking. // // Use value -1 for no timeout and 0 for default. // Default is 3 seconds. ReadTimeout time.Duration // Timeout for socket writes. // If timeout is reached, write commands will fail with a timeout error // instead of blocking. // // Use value -1 for no timeout and 0 for default. // Default is ReadTimeout WriteTimeout time.Duration // Maximum number of socket connections. // Default is 10 connections per every CPU as reported by runtime.NumCPU. PoolSize int // TLS Config used to connect to a server. // TLS will be negotiated only if this field is set. TLSConfig *tls.Config }
RedisFailoverClientOpt is used to creates a redis client that talks to redis sentinels for service discovery and has an automatic failover capability.
func (RedisFailoverClientOpt) MakeRedisClient ¶
func (opt RedisFailoverClientOpt) MakeRedisClient() interface{}
type Result ¶
type Result struct { // ID is a unique identifier for the task. ID string // EnqueuedAt is the time the task was enqueued in UTC. EnqueuedAt time.Time // ProcessAt indicates when the task should be processed. ProcessAt time.Time // Retry is the maximum number of retry for the task. Retry int // Queue is a name of the queue the task is enqueued to. Queue string // Timeout is the timeout value for the task. // Counting for timeout starts when a worker starts processing the task. // If task processing doesn't complete within the timeout, the task will be retried. // The value zero means no timeout. // // If deadline is set, min(now+timeout, deadline) is used, where the now is the time when // a worker starts processing the task. Timeout time.Duration // Deadline is the deadline value for the task. // If task processing doesn't complete before the deadline, the task will be retried. // The value time.Unix(0, 0) means no deadline. // // If timeout is set, min(now+timeout, deadline) is used, where the now is the time when // a worker starts processing the task. Deadline time.Time }
A Result holds enqueued task's metadata.
type RetryDelayFunc ¶
RetryDelayFunc calculates the retry delay duration for a failed task given the retry count, error, and the task.
n is the number of times the task has been retried. e is the error returned by the task handler. t is the task in question.
type Scheduler ¶
type Scheduler struct {
// contains filtered or unexported fields
}
A Scheduler kicks off tasks at regular intervals based on the user defined schedule.
Example ¶
package main import ( "log" "time" "github.com/duke-cliff/asynq" ) func main() { scheduler := asynq.NewScheduler( asynq.RedisClientOpt{Addr: ":6379"}, &asynq.SchedulerOpts{Location: time.Local}, ) if _, err := scheduler.Register("* * * * *", asynq.NewTask("task1", nil)); err != nil { log.Fatal(err) } if _, err := scheduler.Register("@every 30s", asynq.NewTask("task2", nil)); err != nil { log.Fatal(err) } // Run blocks and waits for os signal to terminate the program. if err := scheduler.Run(); err != nil { log.Fatal(err) } }
Output:
func NewScheduler ¶
func NewScheduler(r RedisConnOpt, opts *SchedulerOpts) *Scheduler
NewScheduler returns a new Scheduler instance given the redis connection option. The parameter opts is optional, defaults will be used if opts is set to nil
func (*Scheduler) Register ¶
func (s *Scheduler) Register(cronspec string, task *Task, opts ...Option) (entryID string, err error)
Register registers a task to be enqueued on the given schedule specified by the cronspec. It returns an ID of the newly registered entry.
func (*Scheduler) Run ¶
Run starts the scheduler until an os signal to exit the program is received. It returns an error if scheduler is already running or has been stopped.
func (*Scheduler) Start ¶
Start starts the scheduler. It returns an error if the scheduler is already running or has been stopped.
func (*Scheduler) Stop ¶
Stop stops the scheduler. It returns an error if the scheduler is not currently running.
func (*Scheduler) Unregister ¶
Unregister removes a registered entry by entry ID. Unregister returns a non-nil error if no entries were found for the given entryID.
type SchedulerOpts ¶
type SchedulerOpts struct { // Logger specifies the logger used by the scheduler instance. // // If unset, the default logger is used. Logger Logger // LogLevel specifies the minimum log level to enable. // // If unset, InfoLevel is used by default. LogLevel LogLevel // Location specifies the time zone location. // // If unset, the UTC time zone (time.UTC) is used. Location *time.Location // EnqueueErrorHandler gets called when scheduler cannot enqueue a registered task // due to an error. EnqueueErrorHandler func(task *Task, opts []Option, err error) }
SchedulerOpts specifies scheduler options.
type ServeMux ¶
type ServeMux struct {
// contains filtered or unexported fields
}
ServeMux is a multiplexer for asynchronous tasks. It matches the type of each task against a list of registered patterns and calls the handler for the pattern that most closely matches the task's type name.
Longer patterns take precedence over shorter ones, so that if there are handlers registered for both "images" and "images:thumbnails", the latter handler will be called for tasks with a type name beginning with "images:thumbnails" and the former will receive tasks with type name beginning with "images".
func (*ServeMux) Handle ¶
Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.
func (*ServeMux) HandleFunc ¶
HandleFunc registers the handler function for the given pattern.
func (*ServeMux) Handler ¶
Handler returns the handler to use for the given task. It always return a non-nil handler.
Handler also returns the registered pattern that matches the task.
If there is no registered handler that applies to the task, handler returns a 'not found' handler which returns an error.
func (*ServeMux) ProcessTask ¶
ProcessTask dispatches the task to the handler whose pattern most closely matches the task type.
func (*ServeMux) Use ¶
func (mux *ServeMux) Use(mws ...MiddlewareFunc)
Use appends a MiddlewareFunc to the chain. Middlewares are executed in the order that they are applied to the ServeMux.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is responsible for managing the task processing.
Server pulls tasks off queues and processes them. If the processing of a task is unsuccessful, server will schedule it for a retry. A task will be retried until either the task gets processed successfully or until it reaches its max retry count.
If a task exhausts its retries, it will be moved to the archive and will be kept in the archive for some time until a certain condition is met (e.g., archive size reaches a certain limit, or the task has been in the archive for a certain amount of time).
func NewServer ¶
func NewServer(r RedisConnOpt, cfg Config) *Server
NewServer returns a new Server given a redis connection option and background processing configuration.
func (*Server) Quiet ¶
func (srv *Server) Quiet()
Quiet signals the server to stop pulling new tasks off queues. Quiet should be used before stopping the server.
Example ¶
package main import ( "log" "os" "os/signal" "github.com/duke-cliff/asynq" "golang.org/x/sys/unix" ) func main() { srv := asynq.NewServer( asynq.RedisClientOpt{Addr: ":6379"}, asynq.Config{Concurrency: 20}, ) h := asynq.NewServeMux() // ... Register handlers if err := srv.Start(h); err != nil { log.Fatal(err) } sigs := make(chan os.Signal, 1) signal.Notify(sigs, unix.SIGTERM, unix.SIGINT, unix.SIGTSTP) // Handle SIGTERM, SIGINT to exit the program. // Handle SIGTSTP to stop processing new tasks. for { s := <-sigs if s == unix.SIGTSTP { srv.Quiet() // stop processing new tasks continue } break } srv.Stop() }
Output:
func (*Server) Run ¶
Run starts the background-task processing and blocks until an os signal to exit the program is received. Once it receives a signal, it gracefully shuts down all active workers and other goroutines to process the tasks.
Run returns any error encountered during server startup time. If the server has already been stopped, ErrServerStopped is returned.
Example ¶
package main import ( "log" "github.com/duke-cliff/asynq" ) func main() { srv := asynq.NewServer( asynq.RedisClientOpt{Addr: ":6379"}, asynq.Config{Concurrency: 20}, ) h := asynq.NewServeMux() // ... Register handlers // Run blocks and waits for os signal to terminate the program. if err := srv.Run(h); err != nil { log.Fatal(err) } }
Output:
func (*Server) Start ¶
Start starts the worker server. Once the server has started, it pulls tasks off queues and starts a worker goroutine for each task. Tasks are processed concurrently by the workers up to the number of concurrency specified at the initialization time.
Start returns any error encountered during server startup time. If the server has already been stopped, ErrServerStopped is returned.
func (*Server) Stop ¶
func (srv *Server) Stop()
Stop stops the worker server. It gracefully closes all active workers. The server will wait for active workers to finish processing tasks for duration specified in Config.ShutdownTimeout. If worker didn't finish processing a task during the timeout, the task will be pushed back to Redis.
Example ¶
package main import ( "log" "os" "os/signal" "github.com/duke-cliff/asynq" "golang.org/x/sys/unix" ) func main() { srv := asynq.NewServer( asynq.RedisClientOpt{Addr: ":6379"}, asynq.Config{Concurrency: 20}, ) h := asynq.NewServeMux() // ... Register handlers if err := srv.Start(h); err != nil { log.Fatal(err) } sigs := make(chan os.Signal, 1) signal.Notify(sigs, unix.SIGTERM, unix.SIGINT) <-sigs // wait for termination signal srv.Stop() }
Output:
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package inspeq provides helper types and functions to inspect queues and tasks managed by Asynq.
|
Package inspeq provides helper types and functions to inspect queues and tasks managed by Asynq. |
internal
|
|
asynqtest
Package asynqtest defines test helpers for asynq and its internal packages.
|
Package asynqtest defines test helpers for asynq and its internal packages. |
base
Package base defines foundational types and constants used in asynq package.
|
Package base defines foundational types and constants used in asynq package. |
log
Package log exports logging related types and functions.
|
Package log exports logging related types and functions. |
rdb
Package rdb encapsulates the interactions with redis.
|
Package rdb encapsulates the interactions with redis. |
testbroker
Package testbroker exports a broker implementation that should be used in package testing.
|
Package testbroker exports a broker implementation that should be used in package testing. |