Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator is a job iterator with support for fetching more than the original set of jobs requested, in case some of these jobs get discarded, according to the configured discarded percentage tolerance.
func New ¶
func New(pickupMap map[string]int, params jobsdb.GetQueryParamsT, getJobsFn func(context.Context, map[string]int, jobsdb.GetQueryParamsT, jobsdb.MoreToken) (*jobsdb.GetAllJobsResult, error), opts ...IteratorOptFn) *Iterator
New returns a new job iterator
- pickupMap: a map of workspaceID to number of jobs to be fetched from that workspace
- params: jobsDB query parameters
- getJobsFn: the function to fetch jobs from jobsDB
- opts: optional iterator options
func (*Iterator) Discard ¶
Discard is called when a job is not processed. By discarding a job we are allowing the iterator to fetch more jobs from jobsDB.
func (*Iterator) HasNext ¶
HasNext returns true when there are more jobs to be fetched by Next(), false otherwise.
func (*Iterator) Next ¶
Next returns the next job to be processed. Never call Next() without checking HasNext() first.
func (*Iterator) Stats ¶
func (ji *Iterator) Stats() IteratorStats
type IteratorOptFn ¶
type IteratorOptFn func(*Iterator)
func WithDiscardedPercentageTolerance ¶
func WithDiscardedPercentageTolerance(discardedPercentageTolerance int) IteratorOptFn
WithDiscardedPercentageTolerance sets the discarded percentage tolerance, i.e. the maximum percentage of discarded jobs that can be tolerated without further querying jobsDB.
func WithLegacyOrderGroupKey ¶
func WithLegacyOrderGroupKey(legacy bool) IteratorOptFn
WithLegacyOrderGroupKeyFn if enabled, sets the order group key function to assume a legacy jobs pickup algorithm.
func WithMaxQueries ¶
func WithMaxQueries(maxQueries int) IteratorOptFn
WithMaxQueries sets the maximum number of queries that can be made to the jobsDB for fetching more jobs.
type IteratorStats ¶
type IteratorStats struct {
// QueryCount is the number of queries made to the jobsDB
QueryCount int
// TotalJobs is the total number of jobs queried
TotalJobs int
// DiscardedJobs is the number of jobs discarded
DiscardedJobs int
}
IteratorStats holds statistics about an iterator
Source Files