Documentation
¶
Index ¶
- Constants
- func AlternateTaskFinder(d distro.Distro) ([]task.Task, error)
- func DeficitBasedHostAllocator(ctx context.Context, hostAllocatorData HostAllocatorData) (int, error)
- func GetDistroQueueInfo(distroID string, tasks []task.Task, maxDurationThreshold time.Duration) model.DistroQueueInfo
- func LegacyFindRunnableTasks(d distro.Distro) ([]task.Task, error)
- func ParallelTaskFinder(d distro.Distro) ([]task.Task, error)
- func PersistTaskQueue(distro string, tasks []task.Task, distroQueueInfo model.DistroQueueInfo) error
- func PlanDistro(ctx context.Context, conf Configuration, s *evergreen.Settings) error
- func PopulateCaches(id string, distroID string, tasks []task.Task) ([]task.Task, error)
- func PrioritizeTasks(id string, d *distro.Distro, tasks []task.Task, isSecondaryQueue bool) ([]task.Task, error)
- func RunnableTasksPipeline(d distro.Distro) ([]task.Task, error)
- func SpawnHosts(ctx context.Context, d distro.Distro, newHostsNeeded int, ...) ([]host.Host, error)
- func UpdateStaticDistro(d distro.Distro) error
- func UtilizationBasedHostAllocator(ctx context.Context, hostAllocatorData HostAllocatorData) (int, error)
- type CmpBasedTaskComparator
- type CmpBasedTaskPrioritizer
- type CmpBasedTaskQueues
- type Configuration
- type HostAllocator
- type HostAllocatorData
- type Scheduler
- type StringSet
- type TaskFinder
- type TaskGroupData
- type TaskList
- type TaskPlan
- type TaskPlanner
- type TaskPrioritizer
- type Unit
- type UnitCache
Constants ¶
const (
RunnerName = "scheduler"
)
Variables ¶
This section is empty.
Functions ¶
func DeficitBasedHostAllocator ¶
func DeficitBasedHostAllocator(ctx context.Context, hostAllocatorData HostAllocatorData) (int, error)
DeficitBasedHostAllocator decides how many new hosts are needed for a distro by seeing if the number of tasks that need to be run for the distro is greater than the number of hosts currently free to run a task. Returns a map of distro-># of hosts to spawn.
func GetDistroQueueInfo ¶
func GetDistroQueueInfo(distroID string, tasks []task.Task, maxDurationThreshold time.Duration) model.DistroQueueInfo
Returns the distroQueueInfo for the given set of tasks having set the task.ExpectedDuration for each task.
func LegacyFindRunnableTasks ¶
The old Task finderDBTaskFinder, with the dependency check implemented in Go, instead of using $graphLookup
func PersistTaskQueue ¶
func PersistTaskQueue(distro string, tasks []task.Task, distroQueueInfo model.DistroQueueInfo) error
PersistTaskQueue saves the task queue to the database. Returns an error if the db call returns an error.
func PlanDistro ¶
func PopulateCaches ¶
PopulateCaches runs setup functions and is used by the new/tunable scheduler to reprocess tasks before running the new planner.
func PrioritizeTasks ¶
func SpawnHosts ¶
func SpawnHosts(ctx context.Context, d distro.Distro, newHostsNeeded int, pool *evergreen.ContainerPool) ([]host.Host, error)
Call out to the embedded Manager to spawn hosts. Takes in a map of distro -> number of hosts to spawn for the distro. Returns a map of distro -> hosts spawned, and an error if one occurs.
func UpdateStaticDistro ¶
func UtilizationBasedHostAllocator ¶
func UtilizationBasedHostAllocator(ctx context.Context, hostAllocatorData HostAllocatorData) (int, error)
Types ¶
type CmpBasedTaskComparator ¶
type CmpBasedTaskComparator struct {
// contains filtered or unexported fields
}
CmpBasedTaskComparator runs the tasks through a slice of comparator functions determining which is more important.
func NewCmpBasedTaskComparator ¶
func NewCmpBasedTaskComparator(id string) *CmpBasedTaskComparator
NewCmpBasedTaskComparator returns a new task prioritizer, using the default set of comparators as well as the setup functions necessary for those comparators.
func (*CmpBasedTaskComparator) Len ¶
func (self *CmpBasedTaskComparator) Len() int
func (*CmpBasedTaskComparator) Less ¶
func (self *CmpBasedTaskComparator) Less(i, j int) bool
func (*CmpBasedTaskComparator) Swap ¶
func (self *CmpBasedTaskComparator) Swap(i, j int)
type CmpBasedTaskPrioritizer ¶
type CmpBasedTaskPrioritizer struct {
// contains filtered or unexported fields
}
func (*CmpBasedTaskPrioritizer) PrioritizeTasks ¶
func (prioritizer *CmpBasedTaskPrioritizer) PrioritizeTasks(distroId string, tasks []task.Task, versions map[string]model.Version) ([]task.Task, error)
PrioritizeTask prioritizes the tasks to run. First splits the tasks into slices based on whether they are part of patch versions or automatically created versions. Then prioritizes each slice, and merges them. Returns a full slice of the prioritized tasks, and an error if one occurs.
type CmpBasedTaskQueues ¶
type CmpBasedTaskQueues struct {
HighPriorityTasks []task.Task
PatchTasks []task.Task
RepotrackerTasks []task.Task
}
CmpBasedTaskQueues represents the three types of queues that are created for merging together into one queue. The HighPriorityTasks list represent the tasks that are always placed at the front of the queue PatchTasks and RepotrackerTasks are interleaved after the high priority tasks.
type Configuration ¶
type HostAllocator ¶
type HostAllocator func(context.Context, HostAllocatorData) (int, error)
HostAllocator is responsible for determining how many new hosts should be spun up.
func GetHostAllocator ¶
func GetHostAllocator(name string) HostAllocator
type HostAllocatorData ¶
type HostAllocatorData struct {
Distro distro.Distro
ExistingHosts []host.Host
FreeHostFraction float64
UsesContainers bool
ContainerPool *evergreen.ContainerPool
DistroQueueInfo model.DistroQueueInfo
}
type Scheduler ¶
type Scheduler struct {
*evergreen.Settings
TaskPrioritizer
HostAllocator
FindRunnableTasks TaskFinder
}
Responsible for prioritizing and scheduling tasks to be run, on a per-distro basis.
type StringSet ¶
type StringSet map[string]struct{}
StringSet provides simple tools for managing sets of strings.
type TaskFinder ¶
func GetTaskFinder ¶
func GetTaskFinder(version string) TaskFinder
type TaskGroupData ¶
type TaskGroupData struct {
Hosts []host.Host
Info model.TaskGroupInfo
}
type TaskList ¶
TaskList implements sort.Interface on top of a slice of tasks. The provided sorting, orders members of task groups, and then prioritizes tasks by the number of dependencies, priority, and expected duration. This sorting is used for ordering tasks within a unit.
type TaskPlan ¶
type TaskPlan []*Unit
TaskPlan provides a sortable interface on top of a slice of schedulable units, with ordering of units provided by the implementation of RankValue.
func PrepareTasksForPlanning ¶
PrepareTasksForPlanning takes a list of tasks for a distro and returns a TaskPlan, grouping tasks into the appropriate units.
type TaskPlanner ¶
type TaskPrioritizer ¶
type TaskPrioritizer interface {
// Takes in a slice of tasks and the current MCI settings.
// Returns the slice of tasks, sorted in the order in which they should
// be run, as well as an error if appropriate.
PrioritizeTasks(distroId string, tasks []task.Task, versions map[string]model.Version) ([]task.Task, error)
}
TaskPrioritizer is responsible for taking in a slice of tasks, and ordering them according to which should be run first.
type Unit ¶
type Unit struct {
// contains filtered or unexported fields
}
Unit is a holder of a group of related tasks which should be scheculded together. Typically these represent task groups, tasks, and their dependencies, or even all tasks of a version. All tasks in a Unit must be unique with regards to their ID.
func MakeUnit ¶
MakeuUnit constructs a new unit, caching a reference to the distro in the unit. It's valid to pass a nil here.
func (*Unit) Export ¶
Export returns an unordered sequence of tasks from unit. All tasks are unique.
func (*Unit) RankValue ¶
RankValue returns a point value for the tasks in the unit that can be used to compare units with eachother.
Generally, higher point values are given to larger units and for units that have been in the queue for longer, with longer expected runtimes. The tasks priority act as a multiplying factor.
type UnitCache ¶
UnitCache stores an unordered collection of schedulable units. The Unit type holds one or more tasks, but is handled by the scheduler as a single object. While the constituent tasks in a unit have an order, the unit themselves are an intermediate abstraction for the planner which represent task groups, tasks with their dependencies, or the tasks from a single version
func (UnitCache) AddNew ¶
AddNew adds an entire unit to a cache with the specified ID. If the cached item exists, AddNew extends the existing unit with the tasks from the passed unit.
func (UnitCache) Create ¶
Create makes a new unit around the existing task, caching it with the specified key, and returning the resulting unit. If there is an existing cache item with the specified ID, then Create extends that unit with this task. In both cases, the resulting unit is returned to the caller.
Source Files