pythondeployer

README

Arcaflow engine deployer python

This library is an implementation of the arcaflow deployer interface that uses the podman CLI.

Usage

Deployer configuration (config.yaml)

(full configuration example)

deployer:
  type: python
  # Optional Fields
  pythonPath: /usr/bin/python3.9
  workdir: /tmp
  modulePullPolicy: Always | IfNotPresent
  overrideModuleCompatibility: true | false

• pythonPath (optional, default /usr/bin/python)
  • Path to the python interpreter binary
• workdir (optional, default /tmp)
  • folder where the virtual environments of every single plugin are stored. Setting modulePullPolicy as IfNotPresent the workdir will work as a cache and will speed up the workflow runs.
• modulePullPolicy (optional, default IfNotPresent)
  • IfNotPresent: will check in the workdir path if the requested module At the requested version has been already pulled
  • Always: will always pull the module, if already present, will delete the previous version and will pull it again.
• overrideModuleCompatibility: (optional, default false) if enabled the module compatibility checks will be disabled

Worfklows (workflow.yaml)

The main difference in the workflow syntax is that instead of passing a container image as plugin (like the podman, docker and kubernetes deployer) must be passed a python module either in the Git or in the Pypi format as previously mentioned.

Module Name Format:

<module_name>@git+<repo_url>[@git_commit_sha]

Example Git source workflow

steps:
  kill_pod:
    plugin: arcaflow-plugin-kill-pod@git+https://github.com/redhat-chaos/arcaflow-plugin-kill-pod.git@a34551a4aa68d822ba54f338148ca6e6a28c493b
    step: kill-pods
    input:
    ...

Plugin Compatibility

Considering that some plugins are built on top external binaries that would be difficult to mantain and distribute and for that reason are usually distributed as containers we decided to not support all of them. To declare a module as compatible with the deployer a file named .python_deployer_compat must be placed in the plugin root folder and committed to the plugin repository.

Note: it is possible to bypass the compatibility checks for development setting overrideModuleCompatibility to true, but keep in mind that engine could behave in an unexpected way or even crash, please be careful!

Documentation

Index

• Variables
• func NewFactory() deployer.ConnectorFactory[*config.Config]
• type CliPlugin
  • func (p *CliPlugin) Close() error
  • func (p *CliPlugin) ID() string
  • func (p *CliPlugin) Read(b []byte) (n int, err error)
  • func (p *CliPlugin) Write(b []byte) (n int, err error)
• type Connector
  • func (c *Connector) Deploy(ctx context.Context, image string) (deployer.Plugin, error)

Variables

var Schema = schema.NewTypedScopeSchema[*config.Config](
	schema.NewStructMappedObjectSchema[*config.Config](
		"Config",
		map[string]*schema.PropertySchema{
			"pythonPath": schema.NewPropertySchema(
				schema.NewStringSchema(nil, nil, regexp.MustCompile("^.*$")),
				schema.NewDisplayValue(schema.PointerTo("Python path"),
					schema.PointerTo("Provides the path of python executable"), nil),
				false,
				nil,
				nil,
				nil,
				schema.PointerTo(util.JSONEncode(pythonGetDefaultPath())),
				nil,
			),
			"workdir": schema.NewPropertySchema(
				schema.NewStringSchema(nil, nil, nil),
				schema.NewDisplayValue(schema.PointerTo("Workdir Path"),
					schema.PointerTo("Provides the directory where the modules virtual environments will be stored"), nil),
				false,
				nil,
				nil,
				nil,
				nil,
				nil,
			),
			"modulePullPolicy": schema.NewPropertySchema(
				schema.NewStringEnumSchema(map[string]*schema.DisplayValue{
					string(config.ModulePullPolicyAlways):       {NameValue: schema.PointerTo("Always")},
					string(config.ModulePullPolicyIfNotPresent): {NameValue: schema.PointerTo("If not present")},
				}),
				schema.NewDisplayValue(schema.PointerTo("Module pull policy"), schema.PointerTo("When to pull the python module."), nil),
				false,
				nil,
				nil,
				nil,
				schema.PointerTo(util.JSONEncode(string(config.ModulePullPolicyIfNotPresent))),
				nil,
			),

			"overrideModuleCompatibility": schema.NewPropertySchema(
				schema.NewBoolSchema(),
				schema.NewDisplayValue(schema.PointerTo("Module Compatibility"), schema.PointerTo("Overrides the module compatibility check"), nil),
				false,
				nil,
				nil,
				nil,
				schema.PointerTo("false"),
				nil,
			),
		},
	),
)

Schema describes the deployment options of the Docker deployment mechanism.

Functions

func NewFactory

func NewFactory() deployer.ConnectorFactory[*config.Config]

NewFactory creates a new factory for the Docker deployer.

Types

type CliPlugin

type CliPlugin struct {
	// contains filtered or unexported fields
}

func (*CliPlugin) Close

func (p *CliPlugin) Close() error

func (*CliPlugin) ID

func (p *CliPlugin) ID() string

func (*CliPlugin) Read

func (p *CliPlugin) Read(b []byte) (n int, err error)

func (*CliPlugin) Write

func (p *CliPlugin) Write(b []byte) (n int, err error)

type Connector

type Connector struct {
	// contains filtered or unexported fields
}

func (*Connector) Deploy

func (c *Connector) Deploy(ctx context.Context, image string) (deployer.Plugin, error)