resolver

Documentation

Index

• Constants
• Variables
• type PriorityResolver
  • func NewPriorityResolver() *PriorityResolver
  • func (a *PriorityResolver) Resolve(_ context.Context, jobWithDetails []*scheduler.JobWithDetails) error

Constants

const (
	// MinPriorityWeight - what's the minimum weight that we want to give to a DAG.
	// airflow also sets the default priority weight as 1
	MinPriorityWeight = 1

	// MaxPriorityWeight - is the maximus weight a DAG will be given.
	MaxPriorityWeight = 10000

	// PriorityWeightGap - while giving weights to the DAG, what's the GAP
	// do we want to consider. PriorityWeightGap = 1 means, weights will be 1, 2, 3 etc.
	PriorityWeightGap = 10
)

Variables

var (
	// ErrPriorityNotFound is thrown when priority of a given spec is not found
	ErrPriorityNotFound = errors.New("priority weight not found")
)

Types

type PriorityResolver

type PriorityResolver struct{}

PriorityResolver runs a breadth first traversal on DAG/Job dependencies trees and returns highest weight for the DAG that do not have any dependencies, dynamically. eg, consider following DAGs and dependencies: [dag1 <- dag2 <- dag3] [dag4] [dag5 <- dag6] In this example, we've 6 DAGs in which dag1, dag2, dag5 have dependent DAGs. which means, It'd be preferable to run dag1, dag4, dag5 before other DAGs. Results would be: dag1, dag4, dag5 will get highest weight (maxWeight) dag2, dag6 will get weight of maxWeight-1 dag3 will get maxWeight-2 Note: it's crucial that dependencies of all Jobs are already resolved

func NewPriorityResolver

func NewPriorityResolver() *PriorityResolver

NewPriorityResolver create an instance of PriorityResolver

func (*PriorityResolver) Resolve

func (a *PriorityResolver) Resolve(_ context.Context, jobWithDetails []*scheduler.JobWithDetails) error

Resolve takes jobsWithDetails and returns them with resolved priorities