sproto

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 14, 2024 License: Apache-2.0 Imports: 16 Imported by: 0

README

Sproto: Streamlined CRUD Operations with Google Cloud Spanner and Protocol Buffers

Go Reference

The sproto package offers a collection of CRUD operations for interaction with Google Cloud Spanner, utilizing Protocol Buffers for serialization purposes.

Installation

Get the package

go get go.alis.build/sproto

Import the package

import "go.alis.build/sproto"

Usage

Create a new Sproto instance using New or NewClient

Using New
var spannerClient *spanner.Client
sproto := New(spannerClient)
defer sproto.Close()
Using NewClient
sproto, err := NewClient(context.Background(), "GOOGLE_PROJECT", "SPANNER_INSTANCE", "SPANNER_DATABASE")
if err != nil {
    log.Fatalf("failed to create client: %v", err)
}
defer sproto.Close()

If the Sproto instance was created using NewClient you can use Client() to get the underlying spanner.Client instance.

Examples

QueryProtos

Given the following Protocol Buffer message:

syntax = "proto3";

package com.example;

message User {
    string id = 1;
    string name = 2;
    Address address = 3;
    
    message Address {
        string street = 1;
    }
}

And the following table schema:

CREATE TABLE table_name (
    user_id STRING(MAX) NOT NULL,
    secondary_id STRING(MAX) NOT NULL,
    user com.example.User
) PRIMARY KEY (user_id, secondary_id);

You can read a User using ReadProto:

_, err := sproto.ReadProto(ctx, "table_name", spanner.Key{"123","456"},"user", &com.example.User{}, nil)

You can query for User using QueryProtos:

_, err := sproto.QueryProtos(ctx, "table_name", []string{"user"}, []proto.Message{&com.example.User{}}, 
            &spanner.Statement{
                    SQL: "user.name = @name",
                    Params: map[string]interface{}{
                        "name": "John Doe",
                    },
            },
            nil,
)

Documentation

Overview

Package sproto provides a collection of utilities for working with Google Cloud Spanner, utilizing Protocol Buffers for serialization purposes.

Index

Constants

This section is empty.

Variables

View Source 
var ErrInvalidFieldMask = errors.New("invalid field mask")

Functions

This section is empty.

Types

type ErrInvalidNextToken 

type ErrInvalidNextToken struct {
	// contains filtered or unexported fields
}

func (ErrInvalidNextToken) Error 

func (e ErrInvalidNextToken) Error() string

type ErrMismatchedTypes 

type ErrMismatchedTypes struct {
	Expected reflect.Type
	Actual   reflect.Type
}

func (ErrMismatchedTypes) Error 

func (e ErrMismatchedTypes) Error() string

type ErrNegativePageSize 

type ErrNegativePageSize struct{}

func (ErrNegativePageSize) Error 

func (e ErrNegativePageSize) Error() string

type ErrNotFound 

type ErrNotFound struct {
	RowKey string // unavailable locations
}

ErrNotFound is returned when the desired resource is not found in Spanner.

func (ErrNotFound) Error 

func (e ErrNotFound) Error() string

type ReadOptions 

type ReadOptions struct {
	// SortColumns is a map of column names and their respective sort order.
	SortColumns map[string]SortOrder
	// Limit is the maximum number of rows to read.
	Limit int32
	// Offset is the number of rows to skip before reading.
	Offset int32
}

ReadOptions represents the options for reading rows from a table.

type SortOrder 

type SortOrder int64

SortOrder represents the order of sorting. 

const (
	// SortOrderAsc sorts values in ascending order.
	SortOrderAsc SortOrder = iota
	// SortOrderDesc sorts values in descending order.
	SortOrderDesc
)

func (SortOrder) String 

func (s SortOrder) String() string

String returns the string representation of the SortOrder.

type Sproto 

type Sproto struct {
	// contains filtered or unexported fields
}

Sproto provides methods to easily read and write proto messages with Google Cloud Spanner(https://cloud.google.com/spanner/docs/).

It also provides methods to easily perform CRUD operations on tables in Google Cloud Spanner.

func New 

func New(client *spanner.Client) *Sproto

New creates a new Sproto instance with the provided spanner.Client instance.

func NewClient 

func NewClient(ctx context.Context, googleProject, spannerInstance, databaseName string) (*Sproto, error)

NewClient creates a new Sproto instance with the provided Google Cloud Spanner configuration.

func (*Sproto) BatchDeleteRows 

func (s *Sproto) BatchDeleteRows(ctx context.Context, tableName string, rowKeys []spanner.Key) error

BatchDeleteRows deletes multiple rows from the specified table using the provided row keys.

func (*Sproto) BatchInsertRows 

func (s *Sproto) BatchInsertRows(ctx context.Context, tableName string, rows []map[string]interface{}) error

BatchInsertRows inserts multiple rows into the specified table using the provided column values.

The primary key value(s) must be included in the rows.

The rows are represented as a slice of maps where each map represents a row. Each map contains column names and their respective values. The value types must match the column types in the table schema.

func (*Sproto) BatchReadProtos added in v0.1.0 

func (s *Sproto) BatchReadProtos(ctx context.Context, tableName string, rowKeys []spanner.Key, columnName string, message proto.Message, readMask *fieldmaskpb.FieldMask) ([]proto.Message, error)

BatchReadProtos reads multiple proto messages from the specified table using the provided row keys and column name.

The row keys are tuples of the rows' primary keys values and are used to identify the rows to read. The order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column name is used to specify the column where the proto messages are stored. The column must be of type PROTO.

The method returns a slice of proto messages.

func (*Sproto) BatchReadRows 

func (s *Sproto) BatchReadRows(ctx context.Context, tableName string, rowKeys []spanner.Key, columns []string, opts *spanner.ReadOptions) ([]map[string]interface{}, error)

BatchReadRows reads multiple rows from the specified table using the provided row keys and column names.

The row keys are tuples of the row's primary keys values and are used to identify the rows to read. If the primary key is composite, the order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column names are used to specify which columns to read. The order of the columns does not matter.

The method returns a slice of maps where each map represents a row. The maps contain column names and their respective values. Note that the order of the rows in the result is not guaranteed to match the order of the row keys provided.

func (*Sproto) BatchUpdateRows 

func (s *Sproto) BatchUpdateRows(ctx context.Context, tableName string, rows []map[string]interface{}) error

BatchUpdateRows updates multiple rows in the specified table using the provided column values. If a row does not exist, the operation returns an error

The primary key value(s) must be included in each row.

The rows are represented as a slice of maps where each map represents a row. Each map contains column names and their respective values. The value types must match the column types in the table schema.

func (*Sproto) BatchUpsertRows 

func (s *Sproto) BatchUpsertRows(ctx context.Context, tableName string, rows []map[string]interface{}) error

BatchUpsertRows performs upserts for multiple rows into the specified table using the provided column values. If a row already exists, it will be updated with the new values. If a row does not exist, it will be inserted with the provided values.

The primary key value(s) must be included in each row.

The rows are represented as a slice of maps where each map represents a row. Each map contains column names and their respective values. The value types must match the column types in the table schema.

func (*Sproto) BatchWriteMutations 

func (s *Sproto) BatchWriteMutations(ctx context.Context, mutations []*spanner.Mutation) error

BatchWriteMutations writes the provided mutations to the database. This method provides a convenient way to write custom mutations to the database.

func (*Sproto) BatchWriteProtos 

func (s *Sproto) BatchWriteProtos(ctx context.Context, tableName string, rowKeys []spanner.Key, columnNames []string, messages []proto.Message) error

BatchWriteProtos writes multiple proto messages to the provided table.

The row keys are tuples of the rows' primary keys values and are used to identify the rows to write. The row keys must match the length of the messages and are a 1-to-1 mapping. Index i of the row keys corresponds to index i of the messages. The order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column names are used to specify the columns where the proto messages will be stored. The column names must match the length of the messages and are a 1-to-1 mapping. Index i of the column names corresponds to index i of the messages.

The proto messages will be serialized to bytes and stored in the specified columns. The columns must be of type PROTO.

func (*Sproto) Client 

func (s *Sproto) Client() *spanner.Client

Client returns the underlying spanner.Client instance. This client can be used to perform custom queries and mutations

func (*Sproto) Close 

func (s *Sproto) Close()

Close closes the underlying spanner.Client instance.

func (*Sproto) DeleteRow 

func (s *Sproto) DeleteRow(ctx context.Context, tableName string, rowKey spanner.Key) error

DeleteRow deletes a row from the specified table using the provided row key.

func (*Sproto) InsertRow 

func (s *Sproto) InsertRow(ctx context.Context, tableName string, row map[string]interface{}) error

InsertRow inserts a row into the specified table using the provided column values.

The primary key value(s) must be included in the row.

The row is represented as a map where the key is the column name and the value is the column value. The value types must match the column types in the table schema.

func (*Sproto) ListProtos added in v0.1.0 

func (s *Sproto) ListProtos(ctx context.Context, tableName string, columnName string, message proto.Message, opts *spanner.ReadOptions) ([]proto.Message, error)

ListProtos lists all proto messages from the specified table using the provided column name.

The column name is used to specify the column where the proto messages are stored. The column must be of type PROTO.

The method returns a slice of proto messages.

func (*Sproto) ListRows 

func (s *Sproto) ListRows(ctx context.Context, tableName string, columns []string, opts *spanner.ReadOptions) ([]map[string]interface{}, error)

ListRows reads all rows from the specified table using the provided column names.

The column names are used to specify which columns to read. The order of the columns does not matter.

The method returns a slice of maps where each map represents a row. The maps contain column names and their respective values.

func (*Sproto) PurgeRows 

func (s *Sproto) PurgeRows(ctx context.Context, tableName string) error

PurgeRows deletes all rows from the specified table.

func (*Sproto) QueryProtos added in v0.1.0 

func (s *Sproto) QueryProtos(ctx context.Context, tableName string, columnNames []string, messages []proto.Message, filter *spanner.Statement, opts *ReadOptions) (map[string][]proto.Message, error)

QueryProtos reads multiple protos from the specified table using the provided column names and filtering condition.

The column names are used to specify the columns where the proto messages are stored. Each column name must have a corresponding proto message in the messages slice at the same index. Specified columns must be of type PROTO.

The filter is a SQL statement that is used to filter the rows to read. The statement should not include the WHERE keyword. The filter can include placeholders for parameters. The parameters are provided as a map where the key is the parameter name and the value is the parameter value. An example of a filter statement with parameters is "proto_column.name = @name" where "name" is the parameter name. Keep in mind that GoogleSQL uses parameters(@) whereas PostgreSQL uses placeholders($).

Opts can be used to specify sorting, limiting and offsetting conditions.

The method returns a map of column names and their respective values where the key is the column name and the value is a slice of the proto messages.

func (*Sproto) QueryRows 

func (s *Sproto) QueryRows(ctx context.Context, tableName string, columns []string, filter *spanner.Statement, opts *ReadOptions) ([]map[string]interface{}, error)

QueryRows reads multiple rows from the specified table using the provided column names and filtering condition.

The column names are used to specify which columns to read. The order of the columns does not matter.

The filter is a SQL statement that is used to filter the rows to read. The statement should not include the WHERE keyword. The filter can include placeholders for parameters. The parameters are provided as a map where the key is the parameter name and the value is the parameter value. An example of a filter statement with parameters is "name = @name" where "name" is the parameter name. Keep in mind that GoogleSQL uses parameters(@) whereas PostgreSQL uses placeholders($).

Opts can be used to specify sorting, limiting and offsetting conditions.

The method returns a slice of maps where each map represents a row. The maps contain column names and their respective values.

func (*Sproto) ReadProto 

func (s *Sproto) ReadProto(ctx context.Context, tableName string, rowKey spanner.Key, columnName string, message proto.Message, readMask *fieldmaskpb.FieldMask) error

ReadProto reads a proto message from the specified table using the provided row key and column name.

The row key is a tuple of the row's primary keys values and is used to identify the row to read. If the primary key is composite, the order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column name is used to specify the column where the proto message is stored.

func (*Sproto) ReadRow 

func (s *Sproto) ReadRow(ctx context.Context, tableName string, rowKey spanner.Key, columns []string, opts *spanner.ReadOptions) (map[string]interface{}, error)

ReadRow reads a row from the specified table using the provided row key and column names.

The row key is a tuple of the row's primary keys values and is used to identify the row to read. If the primary key is composite, the order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column names are used to specify which columns to read. The order of the columns does not matter.

The method returns a map of column names and their respective values.

func (*Sproto) StreamProtos added in v0.1.0 

func (s *Sproto) StreamProtos(ctx context.Context, tableName string, columnName string, message proto.Message, opts *spanner.ReadOptions) *StreamResponse[proto.Message]

StreamProtos streams proto messages from the specified table using the provided column name.

The column name is used to specify the column where the proto messages are stored. The column must be of type PROTO.

The method returns a StreamResponse[proto.Message] which can be used to iterate over the proto messages. Call Next() on the StreamResponse to get the next item from the stream. Remember to check for io.EOF to determine when the stream is closed.

func (*Sproto) StreamQueryProtos added in v0.1.0 

func (s *Sproto) StreamQueryProtos(ctx context.Context, tableName string, columnNames []string, messages []proto.Message, filter *spanner.Statement, opts *ReadOptions) *StreamResponse[map[string]proto.Message]

StreamQueryProtos streams proto messages from the specified table using the provided column names and filtering condition.

The column names are used to specify the columns where the proto messages are stored. Each column name must have a corresponding proto message in the messages slice at the same index. Specified columns must be of type PROTO.

The filter is a SQL statement that is used to filter the rows to read. The statement should not include the WHERE keyword. The filter can include placeholders for parameters. The parameters are provided as a map where the key is the parameter name and the value is the parameter value. An example of a filter statement with parameters is "proto_column.name = @name" where "name" is the parameter name. Keep in mind that GoogleSQL uses parameters(@) whereas PostgreSQL uses placeholders($).

Opts can be used to specify sorting, limiting and offsetting conditions.

The method returns a StreamResponse[map[string]proto.Message] which can be used to iterate over the proto messages. Call Next() on the StreamResponse to get the next item from the stream. Remember to check for io.EOF to determine when the stream is closed.

func (*Sproto) StreamRows 

func (s *Sproto) StreamRows(ctx context.Context, tableName string, columns []string, filter *spanner.Statement, opts *ReadOptions) (*StreamResponse[map[string]interface{}], error)

StreamRows reads multiple rows from the specified table using the provided column names and filtering condition.

The column names are used to specify which columns to read. The order of the columns does not matter.

The filter is a SQL statement that is used to filter the rows to read. The statement should not include the WHERE keyword. The filter can include placeholders for parameters. The parameters are provided as a map where the key is the parameter name and the value is the parameter value. An example of a filter statement with parameters is "name = @name" where "name" is the parameter name. Keep in mind that GoogleSQL uses parameters(@) whereas PostgreSQL uses placeholders($).

Opts can be used to specify sorting, limiting and offsetting conditions.

The method returns a StreamResponse that can be used to get the items from the stream. Call Next() on the StreamResponse to get the next item from the stream. Remember to check for io.EOF to determine when the stream is closed.

func (*Sproto) UpdateProto 

func (s *Sproto) UpdateProto(ctx context.Context, tableName string, rowKey spanner.Key, columnName string, message proto.Message, updateMask *fieldmaskpb.FieldMask) error

UpdateProto updates a proto message in the specified table using the provided row key and column name.

The row key is a tuple of the row's primary keys values and is used to identify the row to update. If the primary key is composite, the order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column name is used to specify the column where the proto message will be stored. This is still required even if it is included in the row key.

func (*Sproto) UpdateRow 

func (s *Sproto) UpdateRow(ctx context.Context, tableName string, row map[string]interface{}) error

UpdateRow updates a row in the specified table using the provided column values. If the row does not exist, the operation returns an error

The primary key value(s) must be included in the row.

The row is represented as a map where the key is the column name and the value is the column value. The value types must match the column types in the table schema.

func (*Sproto) UpsertRow 

func (s *Sproto) UpsertRow(ctx context.Context, tableName string, row map[string]interface{}) error

UpsertRow performs an upsert into the specified table using the provided column values. If the row already exists, it will be updated with the new values. If the row does not exist, it will be inserted with the provided values.

The primary key value(s) must be included in the row.

The row is represented as a map where the key is the column name and the value is the column value. The value types must match the column types in the table schema.

func (*Sproto) WriteProto 

func (s *Sproto) WriteProto(ctx context.Context, tableName string, rowKey spanner.Key, columnName string, message proto.Message) error

WriteProto writes a provided proto message to the provided table.

The row key is a tuple of the row's primary keys values and is used to identify the row to write. The order of the keys must match the order of the primary key columns in the table schema. For example if the primary key is (id, name), the row key must be spanner.Key{{id}, {name}} where {id} and {name} are the primary key values.

The column name is used to specify the column where the proto message will be stored. This is still required even if it is included in the row key.

The proto message will be stored as is in the specified column. The column's type must match the full message name including the proto package. See https://cloud.google.com/spanner/docs/reference/standard-sql/protocol-buffers

type StreamResponse 

type StreamResponse[T interface{}] struct {
	// contains filtered or unexported fields
}

StreamResponse is a response for a stream Call Next to get the next item from the stream

func NewStreamResponse 

func NewStreamResponse[T interface{}]() *StreamResponse[T]

NewStreamResponse creates a new StreamResponse

func (*StreamResponse[T]) Next 

func (r *StreamResponse[T]) Next() (*T, error)

Next gets the next item from the stream. It returns io.EOF when the stream is closed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.