Documentation ¶
Index ¶
- type AWSConfig
- type ApplicationConfig
- type ApplicationConfiguration
- type Auth
- type ClusterConfig
- type ClusterConfiguration
- type ClusterEntity
- type ClusterResourceConfig
- type ClusterResourceConfiguration
- type Clusters
- type Configuration
- type DataSource
- type DataSourceValueFrom
- type DbConfig
- type DbConfigSection
- type Domain
- type DomainName
- type DomainsConfig
- type EventSchedulerConfig
- type ExecutionQueue
- type ExecutionQueues
- type GCPConfig
- type NamespaceMappingConfig
- type NamespaceMappingConfiguration
- type NotificationsConfig
- type NotificationsEmailerConfig
- type NotificationsProcessorConfig
- type NotificationsPublisherConfig
- type QualityOfServiceConfig
- type QualityOfServiceConfiguration
- type QualityOfServiceSpec
- type QueueConfig
- type QueueConfiguration
- type RegistrationValidationConfig
- type RegistrationValidationConfiguration
- type RemoteDataConfig
- type SchedulerConfig
- type SignedURL
- type TaskResourceConfiguration
- type TaskResourceSet
- type TaskTypeWhitelist
- type TemplateData
- type TierName
- type WhitelistConfiguration
- type WhitelistScope
- type WorkflowConfig
- type WorkflowConfigs
- type WorkflowExecutorConfig
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AWSConfig ¶ added in v0.2.7
type AWSConfig struct {
Region string `json:"region"`
}
This section holds common config for AWS
type ApplicationConfig ¶
type ApplicationConfig struct { // The RoleName key inserted as an annotation (https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) // in Flyte Workflow CRDs created in the CreateExecution flow. The corresponding role value is defined in the // launch plan that is used to create the execution. RoleNameKey string `json:"roleNameKey"` // Top-level name applied to all metrics emitted by the application. MetricsScope string `json:"metricsScope"` // Determines which port the profiling server used for admin monitoring and application debugging uses. ProfilerPort int `json:"profilerPort"` // This defines the nested path on the configured external storage provider where workflow closures are remotely // offloaded. MetadataStoragePrefix []string `json:"metadataStoragePrefix"` // Event version to be used for Flyte workflows EventVersion int `json:"eventVersion"` }
This configuration is the base configuration to start admin
type ApplicationConfiguration ¶
type ApplicationConfiguration interface { GetDbConfig() DbConfig GetTopLevelConfig() *ApplicationConfig GetSchedulerConfig() *SchedulerConfig GetRemoteDataConfig() *RemoteDataConfig GetNotificationsConfig() *NotificationsConfig GetDomainsConfig() *DomainsConfig }
Defines the interface to return top-level config structs necessary to start up a flyteadmin application.
type Auth ¶
type ClusterConfig ¶
type ClusterConfig struct { Name string `json:"name"` Endpoint string `json:"endpoint"` Auth Auth `json:"auth"` Enabled bool `json:"enabled"` }
Holds details about a cluster used for workflow execution.
type ClusterConfiguration ¶
type ClusterConfiguration interface { // Returns clusters defined in runtime configuration files. GetClusterConfigs() []ClusterConfig // Returns label cluster map for routing GetLabelClusterMap() map[string][]ClusterEntity }
Provides values set in runtime configuration files. These files can be changed without requiring a full server restart.
type ClusterEntity ¶ added in v0.2.3
type ClusterResourceConfig ¶
type ClusterResourceConfig struct { TemplatePath string `json:"templatePath"` // TemplateData maps template keys e.g. my_super_secret_password to a data source // which is then substituted in cluster resource templated config files wherever // {{ my_super_secret_password }} appears. TemplateData TemplateData `json:"templateData"` RefreshInterval config.Duration `json:"refreshInterval"` // Like TemplateData above, this also specifies template values as defaults to be substituted for specific domains // or for all domains. // For example: /* defaultData: production: foo: value: "bar" foofoo: valueFrom: env: SHELL staging: foo: value: "baz" */ CustomData map[DomainName]TemplateData `json:"customData"` }
type ClusterResourceConfiguration ¶
type ClusterResourceConfiguration interface { GetTemplatePath() string GetTemplateData() map[string]DataSource GetRefreshInterval() time.Duration GetCustomTemplateData() map[DomainName]TemplateData }
type Clusters ¶
type Clusters struct { ClusterConfigs []ClusterConfig `json:"clusterConfigs"` LabelClusterMap map[string][]ClusterEntity `json:"labelClusterMap"` }
type Configuration ¶
type Configuration interface { ApplicationConfiguration() ApplicationConfiguration QueueConfiguration() QueueConfiguration ClusterConfiguration() ClusterConfiguration TaskResourceConfiguration() TaskResourceConfiguration WhitelistConfiguration() WhitelistConfiguration RegistrationValidationConfiguration() RegistrationValidationConfiguration ClusterResourceConfiguration() ClusterResourceConfiguration NamespaceMappingConfiguration() NamespaceMappingConfiguration QualityOfServiceConfiguration() QualityOfServiceConfiguration }
Interface for getting parsed values from a configuration file
type DataSource ¶
type DataSource struct { Value string `json:"value"` ValueFrom DataSourceValueFrom `json:"valueFrom"` }
type DataSourceValueFrom ¶
type DbConfig ¶
type DbConfig struct { Host string `json:"host"` Port int `json:"port"` DbName string `json:"dbname"` User string `json:"username"` Password string `json:"password"` ExtraOptions string `json:"options"` Debug bool `json:"debug"` }
This represents a configuration used for initiating database connections much like DbConfigSection, however the password is *resolved* in this struct and therefore it is used as the value the runtime provider returns to callers requesting the database config.
type DbConfigSection ¶
type DbConfigSection struct { // The host name of the database server Host string `json:"host"` // The port name of the database server Port int `json:"port"` // The database name DbName string `json:"dbname"` // The database user who is connecting to the server. User string `json:"username"` // Either Password or PasswordPath must be set. // The Password resolves to the database password. Password string `json:"password"` PasswordPath string `json:"passwordPath"` // See http://gorm.io/docs/connecting_to_the_database.html for available options passed, in addition to the above. ExtraOptions string `json:"options"` // Whether or not to start the database connection with debug mode enabled. Debug bool `json:"debug"` }
This configuration section is used to for initiating the database connection with the store that holds registered entities (e.g. workflows, tasks, launch plans...) This struct specifically maps to the flyteadmin config yaml structure.
type Domain ¶
type Domain struct { // Unique identifier for a domain. ID string `json:"id"` // Human readable name for a domain. Name string `json:"name"` }
Domains are always globally set in the application config, whereas individual projects can be individually registered.
type DomainName ¶ added in v0.1.4
type DomainName = string
type DomainsConfig ¶
type DomainsConfig = []Domain
type EventSchedulerConfig ¶
type EventSchedulerConfig struct { // Defines the cloud provider that backs the scheduler. In the absence of a specification the no-op, 'local' // scheme is used. Scheme string `json:"scheme"` // Some cloud providers require a region to be set. Region string `json:"region"` // The role assumed to register and activate schedules. ScheduleRole string `json:"scheduleRole"` // The name of the queue for which scheduled events should enqueue. TargetName string `json:"targetName"` // Optional: The application-wide prefix to be applied for schedule names. ScheduleNamePrefix string `json:"scheduleNamePrefix"` }
This section holds configuration for the event scheduler used to schedule workflow executions.
type ExecutionQueue ¶
Holds details about a queue used for task execution. Matching attributes determine which workflows' tasks will run where.
func (ExecutionQueue) GetAttributes ¶ added in v0.2.0
func (q ExecutionQueue) GetAttributes() []string
type ExecutionQueues ¶
type ExecutionQueues []ExecutionQueue
type GCPConfig ¶ added in v0.2.7
type GCPConfig struct {
ProjectID string `json:"projectId"`
}
This section holds common config for GCP
type NamespaceMappingConfig ¶ added in v0.1.3
type NamespaceMappingConfig struct {
Mapping string `json:"mapping"`
}
type NamespaceMappingConfiguration ¶ added in v0.1.3
type NamespaceMappingConfiguration interface {
GetNamespaceMappingConfig() common.NamespaceMapping
}
type NotificationsConfig ¶
type NotificationsConfig struct { // Defines the cloud provider that backs the scheduler. In the absence of a specification the no-op, 'local' // scheme is used. Type string `json:"type"` // Deprecated: Please use AWSConfig instead. Region string `json:"region"` AWSConfig AWSConfig `json:"aws"` GCPConfig GCPConfig `json:"gcp"` NotificationsPublisherConfig NotificationsPublisherConfig `json:"publisher"` NotificationsProcessorConfig NotificationsProcessorConfig `json:"processor"` NotificationsEmailerConfig NotificationsEmailerConfig `json:"emailer"` // Number of times to attempt recreating a notifications processor client should there be any disruptions. ReconnectAttempts int `json:"reconnectAttempts"` // Specifies the time interval to wait before attempting to reconnect the notifications processor client. ReconnectDelaySeconds int `json:"reconnectDelaySeconds"` }
Configuration specific to notifications handling
type NotificationsEmailerConfig ¶
type NotificationsEmailerConfig struct { // The optionally templatized subject used in notification emails. Subject string `json:"subject"` // The optionally templatized sender used in notification emails. Sender string `json:"sender"` // The optionally templatized body the sender used in notification emails. Body string `json:"body"` }
This section handles the configuration of notifications emails.
type NotificationsProcessorConfig ¶
type NotificationsProcessorConfig struct { // The name of the queue onto which workflow notifications will enqueue. QueueName string `json:"queueName"` // The account id (according to whichever cloud provider scheme is used) that has permission to read from the above // queue. AccountID string `json:"accountId"` }
This section handles configuration for processing workflow events.
type NotificationsPublisherConfig ¶
type NotificationsPublisherConfig struct { // The topic which notifications use, e.g. AWS SNS topics. TopicName string `json:"topicName"` }
This section handles configuration for the workflow notifications pipeline.
type QualityOfServiceConfig ¶ added in v0.3.0
type QualityOfServiceConfig struct { TierExecutionValues map[TierName]QualityOfServiceSpec `json:"tierExecutionValues"` DefaultTiers map[DomainName]TierName `json:"defaultTiers"` }
type QualityOfServiceConfiguration ¶ added in v0.3.0
type QualityOfServiceConfiguration interface { GetTierExecutionValues() map[core.QualityOfService_Tier]core.QualityOfServiceSpec GetDefaultTiers() map[DomainName]core.QualityOfService_Tier }
type QualityOfServiceSpec ¶ added in v0.3.0
type QueueConfig ¶
type QueueConfig struct { ExecutionQueues ExecutionQueues `json:"executionQueues"` WorkflowConfigs WorkflowConfigs `json:"workflowConfigs"` }
type QueueConfiguration ¶
type QueueConfiguration interface { // Returns executions queues defined in runtime configuration files. GetExecutionQueues() []ExecutionQueue // Returns workflow configurations defined in runtime configuration files. GetWorkflowConfigs() []WorkflowConfig }
Provides values set in runtime configuration files. These files can be changed without requiring a full server restart.
type RegistrationValidationConfiguration ¶
type RegistrationValidationConfiguration interface { GetWorkflowNodeLimit() int GetMaxLabelEntries() int GetMaxAnnotationEntries() int GetWorkflowSizeLimit() string }
Provides validation limits used at entity registration
type RemoteDataConfig ¶
type RemoteDataConfig struct { // Defines the cloud provider that backs the scheduler. In the absence of a specification the no-op, 'local' // scheme is used. Scheme string `json:"scheme"` // Some cloud providers require a region to be set. Region string `json:"region"` SignedURL SignedURL `json:"signedUrls"` // Specifies the max size in bytes for which execution data such as inputs and outputs will be populated in line. MaxSizeInBytes int64 `json:"maxSizeInBytes"` }
This configuration handles all requests to get remote data such as execution inputs & outputs.
type SchedulerConfig ¶
type SchedulerConfig struct { EventSchedulerConfig EventSchedulerConfig `json:"eventScheduler"` WorkflowExecutorConfig WorkflowExecutorConfig `json:"workflowExecutor"` // Specifies the number of times to attempt recreating a workflow executor client should there be any disruptions. ReconnectAttempts int `json:"reconnectAttempts"` // Specifies the time interval to wait before attempting to reconnect the workflow executor client. ReconnectDelaySeconds int `json:"reconnectDelaySeconds"` }
This configuration is the base configuration for all scheduler-related set-up.
type SignedURL ¶
type SignedURL struct { // The amount of time for which a signed URL is valid. DurationMinutes int `json:"durationMinutes"` // The principal that signs the URL. This is only applicable to GCS URL. SigningPrincipal string `json:"signingPrincipal"` }
Configuration specific to setting up signed urls.
type TaskResourceConfiguration ¶
type TaskResourceConfiguration interface { GetDefaults() TaskResourceSet GetLimits() TaskResourceSet }
Provides default values for task resource limits and defaults.
type TaskResourceSet ¶
type TaskTypeWhitelist ¶
type TaskTypeWhitelist = map[string][]WhitelistScope
Defines specific task types whitelisted for support.
type TemplateData ¶ added in v0.1.4
type TemplateData = map[string]DataSource
type WhitelistConfiguration ¶
type WhitelistConfiguration interface { // Returns whitelisted task types defined in runtime configuration files. GetTaskTypeWhitelist() TaskTypeWhitelist }
type WhitelistScope ¶
type WorkflowConfig ¶
Defines the specific resource attributes (tags) a workflow requires to run.
type WorkflowConfigs ¶
type WorkflowConfigs []WorkflowConfig
type WorkflowExecutorConfig ¶
type WorkflowExecutorConfig struct { // Defines the cloud provider that backs the scheduler. In the absence of a specification the no-op, 'local' // scheme is used. Scheme string `json:"scheme"` // Some cloud providers require a region to be set. Region string `json:"region"` // The name of the queue onto which scheduled events will enqueue. ScheduleQueueName string `json:"scheduleQueueName"` // The account id (according to whichever cloud provider scheme is used) that has permission to read from the above // queue. AccountID string `json:"accountId"` }
This section holds configuration for the executor that processes workflow scheduled events fired.