README
¶
Worker Pool v2
A worker pool allows you to setup a pool of finite worker goroutines to perform jobs. This paradigm is especially useful where you need to limit work in progress. For example, if you need to run concurrent HTTP requests, you should limit the number of simultaneous requests because the OS may have limits, and the receieving server may also have limits.
Example
package main
import (
"fmt"
"time"
"github.com/app-nerds/kit/v6/workerpool2"
)
func newWorker(index int) func() error {
return func() error {
fmt.Printf("Worker %d sleeping...\n", index)
time.Sleep(2 * time.Second)
}
}
func main() {
var pool workerpool2.PoolOrchestrator
errChan := make(chan error)
stopErrChan := make(chan struct{})
go func() {
for {
select {
case err := <-errChan:
// Do something useful with any errors
fmt.Printf("we received error '%v'\n", err)
case <-stopErrChan:
return
}
}
}()
pool = workerpool2.NewPool(workerpool2.PoolConfig{
ErrorChan: errChan,
MaxWorkers: 10,
})
pool.Start()
for index := 0; index < 30; index++ {
job := newWorker(index)
pool.QueueJob(job)
}
pool.Wait()
pool.Shutdown()
stopErrChan <- struct{}{}
}
Documentation
¶
Overview ¶
* Copyright (c) 2023. App Nerds LLC. All rights reserved
* Copyright (c) 2023. App Nerds LLC. All rights reserved
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Pool ¶
type Pool struct {
// contains filtered or unexported fields
}
A Pool provides methods for managing a pool of workers who perform jobs. A pool can be configured to have a maximum number of available workers, and will wait up to a configurable amount of time for a worker to become available before returning an error
func (*Pool) PutWorkerInTheQueue ¶
PutWorkerInTheQueue puts a worker in the worker queue
func (*Pool) QueueJob ¶
func (p *Pool) QueueJob(job WorkerFunc)
QueueJob adds a job to the work queue
func (*Pool) Shutdown ¶
func (p *Pool) Shutdown()
Shutdown closes the job queue and waits for current workers to finish
func (*Pool) WriteError ¶
WriteError writes an error to the error channel, if available.
type PoolConfig ¶
PoolConfig provides the ability to configure the worker pool. MaxWorkers specifies the maximum number of workers available. ErrorChan allows you to provide a channel to watch for errors. This is optional
type PoolOrchestrator ¶
type PoolOrchestrator interface {
PutWorkerInTheQueue(worker Worker)
Shutdown()
Start()
QueueJob(job WorkerFunc)
Wait()
WriteError(err error)
}
PoolOrchestrator describes an interface for managing a pool of workers who perform jobs
type PoolWorker ¶
type PoolWorker struct {
// contains filtered or unexported fields
}
A PoolWorker is someone that performs a job. There are a finite number of workers in the pool
func (*PoolWorker) DoJob ¶
func (w *PoolWorker) DoJob(job WorkerFunc)
DoJob executes the provided job. When the work is complete this worker will put itself back in the queue as available. This method execute a goroutine
func (*PoolWorker) RejoinWorkerPool ¶
func (w *PoolWorker) RejoinWorkerPool()
RejoinWorkerPool puts this worker back in the worker queue of the pool. A worker will rejoin the queue when she has finished the job
type Worker ¶
type Worker interface {
DoJob(job WorkerFunc)
RejoinWorkerPool()
}
Worker interface describes a struct that performs a job
type WorkerFunc ¶
type WorkerFunc func() error
Source Files