spannerio

Documentation

Overview

Package spannerio provides an API for reading and writing resouces to Google Spanner datastores.

Index

• func Query(s beam.Scope, db string, q string, t reflect.Type, options ...QueryOptionFn) beam.PCollection
• func Read(s beam.Scope, db string, table string, t reflect.Type) beam.PCollection
• func Write(s beam.Scope, db string, table string, col beam.PCollection, ...)
• type QueryOptionFn
  • func WithBatching(batching bool) QueryOptionFn
  • func WithMaxPartitions(maxPartitions int64) QueryOptionFn
  • func WithTimestampBound(timestampBound spanner.TimestampBound) QueryOptionFn
• type TestDto
• type WriteOptionsFn
  • func UseBatchSize(batchSize int) WriteOptionsFn

Functions

func Query

func Query(s beam.Scope, db string, q string, t reflect.Type, options ...QueryOptionFn) beam.PCollection

Query executes a spanner query. It returns a PCollection<t> for a given type T. T must be a struct with exported fields that have the "spanner" tag. By default, the transform uses spanners partitioned read ability to split the results into bundles. If the underlying query is not root-partitionable you can disable batching via UseBatching.

func Read

func Read(s beam.Scope, db string, table string, t reflect.Type) beam.PCollection

Read reads all rows from the given spanner table. It returns a PCollection<t> for a given type T. T must be a struct with exported fields that have the "spanner" tag. If the table has more rows than t, then Read is implicitly a projection.

func Write

func Write(s beam.Scope, db string, table string, col beam.PCollection, options ...WriteOptionsFn)

Write writes the elements of the given PCollection<T> to spanner. T is required to be the schema type.

Types

type QueryOptionFn

type QueryOptionFn func(*queryOptions) error

QueryOptionFn is a function that can be passed to Read or Query to configure options for reading or querying spanner.

func WithBatching

func WithBatching(batching bool) QueryOptionFn

WithBatching sets whether we will use a batched reader. Batching is set to true by default, disable it when the underlying query is not root-partitionable.

func WithMaxPartitions

func WithMaxPartitions(maxPartitions int64) QueryOptionFn

WithMaxPartitions sets the maximum number of Partitions to split the query into when batched reading.

func WithTimestampBound

func WithTimestampBound(timestampBound spanner.TimestampBound) QueryOptionFn

WithTimestampBound sets the TimestampBound to use when doing batched reads.

type TestDto

type TestDto struct {
	One string `spanner:"One"`
	Two int64  `spanner:"Two"`
}

type WriteOptionsFn

type WriteOptionsFn func(qo *writeOptions) error

WriteOptionsFn is a function that can be passed to Write to configure options for writing to spanner.

func UseBatchSize

func UseBatchSize(batchSize int) WriteOptionsFn

UseBatchSize explicitly sets the batch size per transaction for writes.