Documentation ¶
Overview ¶
Package bigtableio contains cross-language functionality for using Google Cloud BigQuery (https://cloud.google.com/bigquery). These transforms only work on runners that support cross-language transforms.
Setup ¶
Transforms specified here are cross-language transforms implemented in a different SDK (listed below). During pipeline construction, the Go SDK will need to connect to an expansion service containing information on these transforms in their native SDK. If an expansion service address is not provided, an appropriate expansion service will be automatically started; however this is slower than having a persistent expansion service running.
To use a persistent expansion service, it must be run as a separate process accessible during pipeline construction. The address of that process must be passed to the transforms in this package.
The version of the expansion service should match the version of the Beam SDK being used. For numbered releases of Beam, these expansions services are released to the Maven repository as modules. For development versions of Beam, it is recommended to build and run it from source using Gradle.
Current supported SDKs, including expansion service modules and reference documentation:
Java:
- Vendored Module: beam-sdks-java-io-google-cloud-platform-expansion-service
- Run via Gradle: ./gradlew :sdks:java:io:google-cloud-platform:expansion-service:runExpansionService
- Reference Class: org.apache.beam.sdk.io.gcp.bigtable.BigtableReadSchemaTransformProvider and org.apache.beam.sdk.io.gcp.bigtable.BigtableIO
Note On Documentation ¶
This cross-language implementation relies on the behavior of external SDKs. In order to keep documentation up-to-date and consistent, Bigtable functionality will not be described in detail in this package. Instead, references to relevant documentation in other SDKs is included where relevant.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Read ¶
func Read(s beam.Scope, projectId string, instanceId string, table string, opts ...ReadOption) beam.PCollection
Read reads rows from a Bigtable table.
Types ¶
type Cell ¶
type Cell = struct { Value []uint8 `beam:"value"` Timestamp_micros int64 `beam:"timestamp_micros"` }
Cell represents a single cell in a Bigtable row. TODO(https://github.com/apache/beam/issues/21784): Change back to a named struct once resolved.
type ReadOption ¶
type ReadOption func(*readConfig)
func ReadExpansionAddr ¶
func ReadExpansionAddr(addr string) ReadOption
ReadExpansionAddr specifies the address of a persistent expansion service to use for a Read transform. If this is not provided, or if an empty string is provided, the transform will automatically start an appropriate expansion service instead.