dataflow

package
v3.0.0-...-7ba4d6b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2024 License: Apache-2.0, BSD-3-Clause, MIT Imports: 24 Imported by: 0

Documentation

Overview

Package dataflow contains the Dataflow runner for submitting pipelines to Google Cloud Dataflow.

This package infers Pipeline Options from flags automatically on job submission, for display in the Dataflow UI. Use the DontUseFlagAsPipelineOption function to prevent using a given flag as a PipelineOption.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DontUseFlagAsPipelineOption 

func DontUseFlagAsPipelineOption(s string)

DontUseFlagAsPipelineOption prevents a set flag from appearing as a PipelineOption in the Dataflow UI. Useful for sensitive, noisy, or irrelevant configuration.

func Execute 

func Execute(ctx context.Context, p *beam.Pipeline) (beam.PipelineResult, error)

Execute runs the given pipeline on Google Cloud Dataflow. It uses the default application credentials to submit the job.

Types

This section is empty.

Source Files

View all Source files

Directories

Path Synopsis
Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.
 Package dataflowlib translates a Beam pipeline model to the Dataflow API job model, for submission to Google Cloud Dataflow.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.