Documentation
¶
Overview ¶
Package runners defines the common "--runner" flag. We have a single definition for it to avoid re-definition errors, between it and test packages or other integration harnesses.
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var Runner = flag.String("runner", "", "Pipeline runner.")
Runner is a flag to specify which Beam runner should be used to execute the pipeline.
Functions ¶
This section is empty.
Types ¶
This section is empty.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow.
|
Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow. |
|
dataflowlib
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.
|
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow. |
|
Package direct contains the direct runner for running single-bundle pipelines in the current process.
|
Package direct contains the direct runner for running single-bundle pipelines in the current process. |
|
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan.
|
Package dot is a Beam runner that "runs" a pipeline by producing a DOT graph of the execution plan. |
|
Package flink contains the Flink runner.
|
Package flink contains the Flink runner. |
|
Package prism contains a local runner for running pipelines in the current process.
|
Package prism contains a local runner for running pipelines in the current process. |
|
internal
Package internal is where the less separable parts of the runner are put together in order to execute pipelines, and validate that beam features are implemented, and configured appropriately for the variant a pipeline is using.
|
Package internal is where the less separable parts of the runner are put together in order to execute pipelines, and validate that beam features are implemented, and configured appropriately for the variant a pipeline is using. |
|
internal/config
Package config defines and handles the parsing and provision of configurations for the runner.
|
Package config defines and handles the parsing and provision of configurations for the runner. |
|
internal/engine
Package engine handles the operational components of a runner, to track elements, watermarks, timers, triggers etc
|
Package engine handles the operational components of a runner, to track elements, watermarks, timers, triggers etc |
|
internal/jobservices
Package jobservices handles services necessary WRT handling jobs from SDKs.
|
Package jobservices handles services necessary WRT handling jobs from SDKs. |
|
internal/urns
Package urns handles extracting urns from all the protos.
|
Package urns handles extracting urns from all the protos. |
|
internal/web
Package web serves a web UI for Prism.
|
Package web serves a web UI for Prism. |
|
internal/worker
Package worker handles interactions with SDK side workers, representing the worker services, communicating with those services, and SDK environments.
|
Package worker handles interactions with SDK side workers, representing the worker services, communicating with those services, and SDK environments. |
|
Package samza contains the Samza runner.
|
Package samza contains the Samza runner. |
|
Package spark contains the Spark runner.
|
Package spark contains the Spark runner. |
|
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner.
|
Package universal contains a general-purpose runner that can submit jobs to any portable Beam runner. |
|
extworker
Package extworker provides an external worker service and related utilities.
|
Package extworker provides an external worker service and related utilities. |
|
runnerlib
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner.
|
Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner. |
|
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution.
|
Package vet is a Beam runner that "runs" a pipeline by producing generated code to avoid symbol table lookups and reflection in pipeline execution. |
|
testpipeline
Package testpipeline exports small test pipelines for testing the vet runner.
|
Package testpipeline exports small test pipelines for testing the vet runner. |
Click to show internal directories.
Click to hide internal directories.