configs

package
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 10, 2024 License: Apache-2.0 Imports: 18 Imported by: 4

Documentation

Index

Constants

View Source 
const (
	// prefixes
	PrefixEvent  = "event."
	PrefixHealth = "health."

	HealthCheckInterval = PrefixHealth + "checkInterval"

	// events
	CMEventTrackingEnabled    = PrefixEvent + "trackingEnabled"    // Application Tracking
	CMEventRequestCapacity    = PrefixEvent + "requestCapacity"    // Request Capacity
	CMEventRingBufferCapacity = PrefixEvent + "ringBufferCapacity" // Ring Buffer Capacity
	CMMaxEventStreams         = PrefixEvent + "maxStreams"
	CMMaxEventStreamsPerHost  = PrefixEvent + "maxStreamsPerHost"
	CMRESTResponseSize        = PrefixEvent + "RESTResponseSize"

	// defaults
	DefaultHealthCheckInterval     = 30 * time.Second
	DefaultEventTrackingEnabled    = true
	DefaultEventRequestCapacity    = 1000
	DefaultEventRingBufferCapacity = 100000
	DefaultEventChannelSize        = 100000
	DefaultMaxStreams              = uint64(100)
	DefaultMaxStreamsPerHost       = uint64(15)
	DefaultRESTResponseSize        = uint64(10000)
)
View Source 
const (
	RootQueue        = "root"
	DOT              = "."
	DotReplace       = "_dot_"
	DefaultPartition = "default"

	ApplicationSortPolicy   = "application.sort.policy"
	ApplicationSortPriority = "application.sort.priority"
	PriorityPolicy          = "priority.policy"
	PriorityOffset          = "priority.offset"
	PreemptionPolicy        = "preemption.policy"
	PreemptionDelay         = "preemption.delay"

	// app sort priority values
	ApplicationSortPriorityEnabled  = "enabled"
	ApplicationSortPriorityDisabled = "disabled"
)

Variables

View Source 
var DefaultPreemptionDelay = 30 * time.Second
View Source 
var DefaultSchedulerConfig = `` /* 169-byte string literal not displayed */

DefaultSchedulerConfig contains the default scheduler configuration; used if no other is provided

View Source 
var GroupRegExp = regexp.MustCompile(`^[_a-zA-Z][a-zA-Z0-9:_.-]*$`)

Groups should have a slightly more restrictive regexp (no # / @ or $ at the end)

View Source 
var MaxPriority int32 = math.MaxInt32
View Source 
var MinPriority int32 = math.MinInt32

Priority

View Source 
var QueueNameRegExp = regexp.MustCompile(`^[a-zA-Z0-9_:#/@-]{1,64}$`)

A queue can be a username with the dot replaced. Most systems allow a 32 character user name. The queue name must thus allow for at least that length with the replacement of dots.

View Source 
var RuleNameRegExp = regexp.MustCompile(`^[_a-zA-Z][a-zA-Z0-9_]*$`)

The rule maps to a go identifier check that regexp only

View Source 
var SpecialRegExp = regexp.MustCompile(`[\^$*+?()\[{}|]`)

all characters that make a name different from a regexp

View Source 
var UserRegExp = regexp.MustCompile(`^[_a-zA-Z][a-zA-Z0-9:#/_.@-]*[$]?$`)

User and group name check: systems allow different things POSIX is the base but we need to be lenient and allow more. A username must start with a letter(uppercase or lowercase), followed by any number of letter(uppercase or lowercase), digits, ':', '#', '/', '_', '.', '@', '-', and may end with '$'.

Functions

func AddConfigMapCallback added in v1.2.0 

func AddConfigMapCallback(id string, callback func())

AddConfigMapCallback registers a callback to detect configuration updates

func GetConfigMap added in v1.2.0 

func GetConfigMap() map[string]string

Gets the ConfigMap

func GetConfigurationString 

func GetConfigurationString(requestBytes []byte) string

func IsQueueNameValid added in v1.6.0 

func IsQueueNameValid(queueName string) error

func RemoveConfigMapCallback added in v1.2.0 

func RemoveConfigMapCallback(id string)

RemoveConfigMapCallback removes a previously registered configuration update callback

func SetChecksum 

func SetChecksum(content []byte, conf *SchedulerConfig)

func SetConfigMap added in v1.2.0 

func SetConfigMap(newConfigMap map[string]string)

Sets the ConfigMap based on configuration refresh

func Validate 

func Validate(newConfig *SchedulerConfig) error

Check the partition configuration. Any parsing issues will return an error which means that the configuration is invalid. This *must* be called before the configuration is activated. Any configuration that does not pass must be rejected. Check performed: - at least 1 partition must be defined - no more than 1 partition called "default" For the sub components: - The queue config is syntax checked - The placement rules are syntax checked - The user objects are syntax checked

Types

type ChildTemplate 

type ChildTemplate struct {
	MaxApplications uint64            `yaml:",omitempty" json:",omitempty"`
	Properties      map[string]string `yaml:",omitempty" json:",omitempty"`
	Resources       Resources         `yaml:",omitempty" json:",omitempty"`
}

type Filter 

type Filter struct {
	Type   string
	Users  []string `yaml:",omitempty" json:",omitempty"`
	Groups []string `yaml:",omitempty" json:",omitempty"`
}

The user and group filter for a rule. - type of filter (allow or deny filter, empty means allow) - list of users to filter (maybe empty) - list of groups to filter (maybe empty) if the list of users or groups is exactly 1 long it is interpreted as a regular expression

type Limit 

type Limit struct {
	Limit           string
	Users           []string          `yaml:",omitempty" json:",omitempty"`
	Groups          []string          `yaml:",omitempty" json:",omitempty"`
	MaxResources    map[string]string `yaml:",omitempty" json:",omitempty"`
	MaxApplications uint64            `yaml:",omitempty" json:",omitempty"`
}

The limit object to specify user and or group limits at different levels in the partition or queues Different limits for the same user or group may be defined at different levels in the hierarchy - limit description (optional) - list of users (maybe empty) - list of groups (maybe empty) - maximum resources as a resource object to allow for the user or group - maximum number of applications the user or group can have running

type Limits 

type Limits struct {
	Limit []Limit
}

A list of limit objects to define limits for a partition or queue

type NodeSortingPolicy 

type NodeSortingPolicy struct {
	Type            string
	ResourceWeights map[string]float64 `yaml:",omitempty" json:",omitempty"`
}

Global Node Sorting Policy section - type: different type of policies supported (binpacking, fair etc)

type PartitionConfig 

type PartitionConfig struct {
	Name           string
	Queues         []QueueConfig
	PlacementRules []PlacementRule           `yaml:",omitempty" json:",omitempty"`
	Limits         []Limit                   `yaml:",omitempty" json:",omitempty"`
	Preemption     PartitionPreemptionConfig `yaml:",omitempty" json:",omitempty"`
	NodeSortPolicy NodeSortingPolicy         `yaml:",omitempty" json:",omitempty"`
}

The partition object for each partition: - the name of the partition - a list of sub or child queues - a list of placement rule definition objects - a list of users specifying limits on the partition - the preemption configuration for the partition

type PartitionPreemptionConfig 

type PartitionPreemptionConfig struct {
	Enabled *bool `yaml:",omitempty" json:",omitempty"`
}

The partition preemption configuration

type PlacementRule 

type PlacementRule struct {
	Name   string
	Create bool           `yaml:",omitempty" json:",omitempty"`
	Filter Filter         `yaml:",omitempty" json:",omitempty"`
	Parent *PlacementRule `yaml:",omitempty" json:",omitempty"`
	Value  string         `yaml:",omitempty" json:",omitempty"`
}

The queue placement rule definition - the name of the rule - create flag: can the rule create a queue - user and group filter to be applied on the callers - rule link to allow setting a rule to generate the parent - value a generic value interpreted depending on the rule type (i.e queue name for the "fixed" rule or the application label name for the "tag" rule)

type QueueConfig 

type QueueConfig struct {
	Name            string
	Parent          bool              `yaml:",omitempty" json:",omitempty"`
	Resources       Resources         `yaml:",omitempty" json:",omitempty"`
	MaxApplications uint64            `yaml:",omitempty" json:",omitempty"`
	Properties      map[string]string `yaml:",omitempty" json:",omitempty"`
	AdminACL        string            `yaml:",omitempty" json:",omitempty"`
	SubmitACL       string            `yaml:",omitempty" json:",omitempty"`
	ChildTemplate   ChildTemplate     `yaml:",omitempty" json:",omitempty"`
	Queues          []QueueConfig     `yaml:",omitempty" json:",omitempty"`
	Limits          []Limit           `yaml:",omitempty" json:",omitempty"`
}

The queue object for each queue: - the name of the queue - a resources object to specify resource limits on the queue - the maximum number of applications that can run in the queue - a set of properties, exact definition of what can be set is not part of the yaml - ACL for submit and or admin access - a list of sub or child queues - a list of users specifying limits on a queue

type Resources 

type Resources struct {
	Guaranteed map[string]string `yaml:",omitempty" json:",omitempty"`
	Max        map[string]string `yaml:",omitempty" json:",omitempty"`
}

The resource limits to set on the queue. The definition allows for an unlimited number of types to be used. The mapping to "known" resources is not handled here. - guaranteed resources - max resources

type SchedulerConfig 

type SchedulerConfig struct {
	Partitions []PartitionConfig
	Checksum   string `yaml:",omitempty" json:",omitempty"`
}

The configuration can contain multiple partitions. Each partition contains the queue definition for a logical set of scheduler resources.

func LoadSchedulerConfigFromByteArray 

func LoadSchedulerConfigFromByteArray(content []byte) (*SchedulerConfig, error)

func ParseAndValidateConfig 

func ParseAndValidateConfig(content []byte) (*SchedulerConfig, error)

type SchedulerConfigContext 

type SchedulerConfigContext struct {
	// contains filtered or unexported fields
}

scheduler config context provides thread-safe access for scheduler configurations 

var ConfigContext *SchedulerConfigContext

func (*SchedulerConfigContext) Get 

func (ctx *SchedulerConfigContext) Get(policyGroup string) *SchedulerConfig

func (*SchedulerConfigContext) Set 

func (ctx *SchedulerConfigContext) Set(policyGroup string, config *SchedulerConfig)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.