Documentation ¶
Index ¶
- Constants
- type CmpBasedTaskComparator
- type CmpBasedTaskPrioritizer
- type CmpBasedTaskQueues
- type DBTaskDurationEstimator
- type DBTaskFinder
- type DBTaskQueuePersister
- type DeficitBasedHostAllocator
- type DistroScheduleData
- type DurationBasedHostAllocator
- type HostAllocator
- type HostAllocatorData
- type Runner
- type ScheduledDistroTasksData
- type Scheduler
- type TaskDurationEstimator
- type TaskFinder
- type TaskPrioritizer
- type TaskQueuePersister
Constants ¶
const ( // maximum turnaround we want to maintain for all hosts for a given distro MaxDurationPerDistroHost = time.Hour // indicates the fraction of the total duration of shared tasks that we want // to account for when alternate distros are unable to satisfy the // turnaround requirement as determined by MaxDurationPerDistroHost SharedTasksAllocationProportion = 0.8 )
const ( RunnerName = "scheduler" Description = "queue tasks for execution and allocate hosts" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CmpBasedTaskComparator ¶
type CmpBasedTaskComparator struct {
// contains filtered or unexported fields
}
CmpBasedTaskComparator runs the tasks through a slice of comparator functions determining which is more important.
func NewCmpBasedTaskComparator ¶
func NewCmpBasedTaskComparator() *CmpBasedTaskComparator
NewCmpBasedTaskComparator returns a new task prioritizer, using the default set of comparators as well as the setup functions necessary for those comparators.
func (*CmpBasedTaskComparator) Len ¶
func (self *CmpBasedTaskComparator) Len() int
func (*CmpBasedTaskComparator) Less ¶
func (self *CmpBasedTaskComparator) Less(i, j int) bool
func (*CmpBasedTaskComparator) Swap ¶
func (self *CmpBasedTaskComparator) Swap(i, j int)
type CmpBasedTaskPrioritizer ¶
type CmpBasedTaskPrioritizer struct{}
func (*CmpBasedTaskPrioritizer) PrioritizeTasks ¶
func (prioritizer *CmpBasedTaskPrioritizer) PrioritizeTasks( settings *evergreen.Settings, tasks []task.Task) ([]task.Task, error)
PrioritizeTask prioritizes the tasks to run. First splits the tasks into slices based on whether they are part of patch versions or automatically created versions. Then prioritizes each slice, and merges them. Returns a full slice of the prioritized tasks, and an error if one occurs.
type CmpBasedTaskQueues ¶
type CmpBasedTaskQueues struct { HighPriorityTasks []task.Task PatchTasks []task.Task RepotrackerTasks []task.Task }
CmpBasedTaskQueues represents the three types of queues that are created for merging together into one queue. The HighPriorityTasks list represent the tasks that are always placed at the front of the queue PatchTasks and RepotrackerTasks are interleaved after the high priority tasks.
type DBTaskDurationEstimator ¶
type DBTaskDurationEstimator struct{}
DBTaskDurationEstimator retrives the estimated duration of runnable tasks. Implements TaskDurationEstimator.
func (*DBTaskDurationEstimator) GetExpectedDurations ¶
func (self *DBTaskDurationEstimator) GetExpectedDurations( runnableTasks []task.Task) (model.ProjectTaskDurations, error)
GetExpectedDurations returns the expected duration of tasks (by display name) on a project, buildvariant basis.
type DBTaskFinder ¶
type DBTaskFinder struct{}
DBTaskFinder fetches tasks from the database. Implements TaskFinder.
func (*DBTaskFinder) FindRunnableTasks ¶
func (self *DBTaskFinder) FindRunnableTasks() ([]task.Task, error)
FindRunnableTasks finds all tasks that are ready to be run. This works by fetching all undispatched tasks from the database, and filtering out any whose dependencies are not met.
type DBTaskQueuePersister ¶
type DBTaskQueuePersister struct{}
DBTaskQueuePersister saves a queue to the database.
func (*DBTaskQueuePersister) PersistTaskQueue ¶
func (self *DBTaskQueuePersister) PersistTaskQueue(distro string, tasks []task.Task, taskDurations model.ProjectTaskDurations) ([]model.TaskQueueItem, error)
PersistTaskQueue saves the task queue to the database. Returns an error if the db call returns an error.
type DeficitBasedHostAllocator ¶
type DeficitBasedHostAllocator struct{}
DeficitBasedHostAllocator uses the difference between the number of free hosts and the number of tasks that need to be run as a metric for how many new hosts need to be spun up
func (*DeficitBasedHostAllocator) NewHostsNeeded ¶
func (self *DeficitBasedHostAllocator) NewHostsNeeded( hostAllocatorData HostAllocatorData, settings *evergreen.Settings) (map[string]int, error)
NewHostsNeeded decides how many new hosts are needed for a distro by seeing if the number of tasks that need to be run for the distro is greater than the number of hosts currently free to run a task. Returns a map of distro-># of hosts to spawn.
type DistroScheduleData ¶
type DistroScheduleData struct {
// contains filtered or unexported fields
}
DistroScheduleData contains bookkeeping data that is used by distros to determine whether or not to allocate more hosts
type DurationBasedHostAllocator ¶
type DurationBasedHostAllocator struct{}
DurationBasedHostAllocator computes the total time to completion of tasks running - per distro - and then uses that as a heuristic in determining how many new hosts to spin up
func (*DurationBasedHostAllocator) NewHostsNeeded ¶
func (self *DurationBasedHostAllocator) NewHostsNeeded( hostAllocatorData HostAllocatorData, settings *evergreen.Settings) (newHostsNeeded map[string]int, err error)
NewHostsNeeded decides if new hosts are needed for a distro while taking the duration of running/scheduled tasks into consideration. Returns a map of distro to number of hosts to spawn.
type HostAllocator ¶
type HostAllocator interface { NewHostsNeeded(allocatorData HostAllocatorData, settings *evergreen.Settings) (map[string]int, error) }
HostAllocator is responsible for determining how many new hosts should be spun up. Parameters:
distros: a map of distro name -> information on that distro (a model.Distro object) existingDistroHosts: a map of distro name -> currently running hosts on that distro taskQueueItems: a map of distro name -> task queue items for that distro (a TaskQueue object) projectTaskDurations: the expected duration of tasks by project and variant taskRunDistros: a map of task id -> distros the task is allowed to run on
Returns a map of distro name -> how many hosts need to be spun up for that distro.
type HostAllocatorData ¶
type HostAllocatorData struct {
// contains filtered or unexported fields
}
HostAllocatorData is the set of parameters passed to a HostAllocator.
type ScheduledDistroTasksData ¶
type ScheduledDistroTasksData struct {
// contains filtered or unexported fields
}
ScheduledDistroTasksData contains data that is used to compute the expected duration of tasks within a distro's queue
type Scheduler ¶
type Scheduler struct { *evergreen.Settings TaskFinder TaskPrioritizer TaskDurationEstimator TaskQueuePersister HostAllocator }
Responsible for prioritizing and scheduling tasks to be run, on a per-distro basis.
type TaskDurationEstimator ¶
type TaskDurationEstimator interface { GetExpectedDurations(runnableTasks []task.Task) ( model.ProjectTaskDurations, error) }
TaskDurationEstimator is responsible for fetching the expected duration for a given set of runnable tasks.
type TaskFinder ¶
type TaskFinder interface { // Returns a slice of tasks that are ready to be run, and an error if // appropriate. FindRunnableTasks() ([]task.Task, error) }
TaskFinder finds all tasks that are ready to be run.
type TaskPrioritizer ¶
type TaskPrioritizer interface { // Takes in a slice of tasks and the current MCI settings. // Returns the slice of tasks, sorted in the order in which they should // be run, as well as an error if appropriate. PrioritizeTasks(settings *evergreen.Settings, tasks []task.Task) ( []task.Task, error) }
TaskPrioritizer is responsible for taking in a slice of tasks, and ordering them according to which should be run first.
type TaskQueuePersister ¶
type TaskQueuePersister interface { PersistTaskQueue(distro string, tasks []task.Task, taskExpectedDuration model.ProjectTaskDurations) ([]model.TaskQueueItem, error) }
TaskQueuePersister is responsible for taking a task queue for a particular distro and saving it.