README
¶
interpolate - Argument Replacer
Pipedream has a dedicated argument interpolation mechanism in addition to the standard shell interpolation of environment variables (such as $EXAMPLE_ENV_VAR). It pulls values from the map of arguments provided to a pipe and uses the form @{argument-name} for value replacement and @?{argument-name} to check for the presence of an argument.
In addition, any occurrences of @!! will be replaced with the input to the pipe.
The interpolate middleware determines how the argument interpolation takes place. Nested arguments, i.e. maps cannot be interpolated, but strings, integers and arrays of strings are supported. String arrays will be turned into a single string, separated by new line (\n) characters.
If any substitutions have been made by the interpolation middleware, a full run (through the entire middleware stack) will be executed for the respective pipe with the new arguments. This is to ensure both that each middleware has a chance to parse the new arguments and that the full input is available for interpolation, if required.
Arguments
The interpolate middleware takes four arguments:
Enable
The enable argument is a boolean value switch that turns the interpolation on (default) or off.
private:
some-pipe:
interpolate:
enable: false
arg1: bar
# this value will remain as is, without interpolation
arg2: "foo: @{arg1}"
some-other-pipe:
# the interpolation is enabled by default
arg1: bar
# this value will change to `foo: bar`
arg2: "foo: @{arg1}"
Note that because interpolation is deep by default, inline arguments for nested child pipes will be interpolated at the time of invocation of the parent pipe. This may or may not be what you want. To delay the interpolation until the invocation of a child, it is often appropriate to use a separate definition for the child. Unlike the inline arguments, definition arguments are interpolated at the time of child invocation.
Inline Invocation
private:
some-pipe:
arg1: "fizz"
pipe:
- child-1:
arg1: bar
# this value will become "foo: fizz", as it is interpolated at parent invocation time
# using the parent's arguments
arg2: "foo: @{arg1}"
arg3: bar
# in this case, interpolation at parent invocation time cause a warning:
# "unable to find value for argument: `arg3`"
# subsequent interpolation at child invocation time will change the value to "foo: bar"
arg4: "foo: @{arg3}"
- child-2:
arg1: bar
child-2:
# this value will become "foo: fizz", as it is interpolated at invocation time
# of the child, using the child's arguments
arg2: "foo: @{arg1}"
IgnoreWarnings
If an expression of the form @{some-argument-key} is encountered, but no suitable argument is found, a warning will be issued. This can be suppressed by setting the ignoreWarnings argument to true.
private:
some-pipe:
# a warning will be issued here:
# "unable to find value for argument: `arg3`"
arg1: "@{not-present}"
# this will not cause a warning
# (presence of the argument is being checked and thus not expected)
arg2: "@?{also-not-present}"
some-pipe:
interpolate:
ignoreWarnings: true
# no warning will be issued, as suppression is enabled
arg1: "@{not-present}"
Quote
The Quote argument determines whether substituted values should be quoted.
It has four valid values:
single(default): Wrap each replacement in single quotes', escaping any existing single quotes using\'.double: Wrap each replacement in double quotes", escaping any existing double quotes using\".backticks: Wrap each replacement in backticks\``, escaping any existing backticks using\``.none: Do not quote the replacements at all.
private:
single-quoting-pipe:
interpolate:
quote: single
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "'\"value\" \\'with\\' `quotes`'"
arg2: "@{arg1}"
double-quoting-pipe:
interpolate:
quote: double
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "\"\\\"value\\\" 'with' `quotes`\""
arg2: "@{arg1}"
backtick-quoting-pipe:
interpolate:
quote: backtick
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "`\"value\" 'with' \\`quotes\\``'"
arg2: "@{arg1}"
not-quoting-pipe:
interpolate:
quote: none
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "\"value\" 'with' `quotes`"
arg2: "@{arg1}"
EscapeQuotes
The escapeQuote argument can be used to escape quote characters in addition to the escaping performed during quoting (if any).
It has five valid values:
single(default): Replace each single quote character'with\'.double: Replace each double quote character"with\".backticks: Replace each backtick character\`` with\``.all: Perform all of the above.none(default): Do not replace any quotes.
private:
single-quote-escaping-pipe:
interpolate:
escapeQuotes: single
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "'\"value\" \\\\'with\\\\' `quotes`'"
# (single quotes escaped once, then again when the entire string is single quoted)
arg2: "@{arg1}"
single-quote-escaping-pipe-without-quoting:
interpolate:
escapeQuotes: single
quote: none
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "\"value\" \\'with\\' `quotes`"
arg2: "@{arg1}"
double-quote-escaping-pipe:
interpolate:
escapeQuotes: double
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "'\\\"value\\\" \\'with\\' `quotes`'"
arg2: "@{arg1}"
double-quote-escaping-pipe-without-quoting:
interpolate:
escapeQuotes: double
quote: none
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "\\\"value\\\" 'with' `quotes`"
arg2: "@{arg1}"
backtick-escaping-pipe:
interpolate:
escapeQuotes: backtick
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "'\"value\" \\'with\\' \\`quotes\\`'"
arg2: "@{arg1}"
backtick-escaping-pipe-without-quoting:
interpolate:
escapeQuotes: backtick
quote: none
arg1: "\"value\" 'with' `quotes`"
# will be interpolated to "\"value\" 'with' \\`quotes\\`"
arg2: "@{arg1}"
Nested interpolation
Nested argument interpolation is supported up to a nesting level of 5 (this behavior may change in a future version of pipedream). It is also legal to create an interpolation expression using interpolation itself, as in this example:
private:
some-pipe:
interpolate:
quote: "none"
arg1: "value"
# simple substitution
arg2: "@{arg1}"
# creating an interpolation expression using interpolation itself is legal
arg4: "@{"
arg5: "arg2"
arg6: "}"
# this will evaluate to "value"
# after one step of interpolation, we have
# arg2: "value"
# arg3: @{arg2}
# which is then interpolated to
# arg2: "value"
# arg3: "value"
arg3: "@{arg4}@{arg5}@{arg6}"
Documentation
¶
Overview ¶
Package interpolate provides a middleware to substitute arguments or inputs into other arguments
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Middleware ¶ added in v0.32.2
type Middleware struct{}
Middleware is a argument replacer
func NewMiddleware ¶ added in v0.32.2
func NewMiddleware() Middleware
NewMiddleware creates a new middleware instance
func (Middleware) Apply ¶ added in v0.32.2
func (interpolateMiddleware Middleware) Apply( run *pipeline.Run, next func(*pipeline.Run), executionContext *middleware.ExecutionContext, )
Apply is where the middleware's logic resides
It adapts the run based on its slice of the run's arguments. It may also trigger side effects such as executing shell commands or full runs of other pipelines. When done, this function should call next in order to continue unwinding the stack.
func (Middleware) String ¶ added in v0.32.2
func (interpolateMiddleware Middleware) String() string
String is a human-readable description
Source Files