compiler

Documentation

Overview

Package compiler is the backend compiler package for Kubeflow Pipelines v2.

KFP pipeline DSL in python are first compiled by KFP SDK (the frontend compiler) to pipeline spec in JSON format. KFP SDK / frontend compiler is not part of this package. Then, the backend compiler (this package) compiles pipeline spec into argo workflow spec, so that it can be run.

Index

• Constants
• func Accept(job *pipelinespec.PipelineJob, kubernetesSpec *pipelinespec.SinglePlatformSpec, ...) error
• func GetDeploymentConfig(spec *pipelinespec.PipelineSpec) (*pipelinespec.PipelineDeploymentConfig, error)
• func GetPipelineSpec(job *pipelinespec.PipelineJob) (*pipelinespec.PipelineSpec, error)
• type Visitor

Constants

const (
	RootComponentName = "root"
)

Functions

func Accept

func Accept(job *pipelinespec.PipelineJob, kubernetesSpec *pipelinespec.SinglePlatformSpec, v Visitor) error

Accept a pipeline spec and a visitor, iterate through components and call corresponding visitor methods for each component.

Iteration rules: * Components are visited in "bottom-up" order -- leaf container components are visited first, then DAG components. When a DAG component is visited, it's guaranteed that all the components used in it have already been visited. * Each component is visited exactly once.

func GetDeploymentConfig

func GetDeploymentConfig(spec *pipelinespec.PipelineSpec) (*pipelinespec.PipelineDeploymentConfig, error)

func GetPipelineSpec

func GetPipelineSpec(job *pipelinespec.PipelineJob) (*pipelinespec.PipelineSpec, error)

Types

type Visitor

type Visitor interface {
	Container(name string, component *pipelinespec.ComponentSpec, container *pipelinespec.PipelineDeploymentConfig_PipelineContainerSpec) error
	Importer(name string, component *pipelinespec.ComponentSpec, importer *pipelinespec.PipelineDeploymentConfig_ImporterSpec) error
	Resolver(name string, component *pipelinespec.ComponentSpec, resolver *pipelinespec.PipelineDeploymentConfig_ResolverSpec) error
	DAG(name string, component *pipelinespec.ComponentSpec, dag *pipelinespec.DagSpec) error
	AddKubernetesSpec(name string, kubernetesSpec *structpb.Struct) error
}

Visitor interface is called when each component is visited. The specific method called depends on the component's type.