python

Documentation

Overview

Package python contains data structures required for python external transforms in a multilanguage pipeline.

Index

• Constants
• func NewExternalTransform[A, K any](constructor string) *pythonExternalTransform[A, K]
• type CallableSource

Constants

const (

	// ExpansionServiceModule is the module containing the python expansion service for python external transforms.
	ExpansionServiceModule = "apache_beam.runners.portability.expansion_service_main"
)

Functions

func NewExternalTransform

func NewExternalTransform[A, K any](constructor string) *pythonExternalTransform[A, K]

NewExternalTransform creates a new instance for python external transform. It accepts two types: A: used for normal arguments K: used for keyword arguments

Types

type CallableSource

type CallableSource string

CallableSource is a wrapper object storing a Python function definition that can be evaluated to Python callables in Python SDK.

The snippet of Python code can be a valid Python expression such as

   lambda x: x * x
	  str.upper

a fully qualified name such as

math.sin

or a complete multi-line function or class definition such as

   def foo(x):
	   ...
   class Foo:
	   ...

Any lines preceding the function definition are first evaluated to provide context in which to define the function which can be useful to declare imports or any other needed values, e.g.

import math

def helper(x):
    return x * x

def func(y):
    return helper(y) + y

in which case `func` would get applied to each element.