Documentation ¶
Overview ¶
Package bigtable is an API to Google Cloud Bigtable.
See https://cloud.google.com/bigtable/docs/ for general product documentation.
Setup and Credentials ¶
Use NewClient or NewAdminClient to create a client that can be used to access the data or admin APIs respectively. Both require credentials that have permission to access the Cloud Bigtable API.
If your program is run on Google App Engine or Google Compute Engine, using the Application Default Credentials (https://developers.google.com/accounts/docs/application-default-credentials) is the simplest option. Those credentials will be used by default when NewClient or NewAdminClient are called.
To use alternate credentials, pass them to NewClient or NewAdminClient using cloud.WithTokenSource. For instance, you can use service account credentials by visiting https://cloud.google.com/console/project/MYPROJECT/apiui/credential, creating a new OAuth "Client ID", storing the JSON key somewhere accessible, and writing
jsonKey, err := ioutil.ReadFile(pathToKeyFile) ... config, err := google.JWTConfigFromJSON(jsonKey, bigtable.Scope) // or bigtable.AdminScope, etc. ... client, err := bigtable.NewClient(ctx, project, zone, cluster, cloud.WithTokenSource(config.TokenSource(ctx))) ...
Here, `google` means the golang.org/x/oauth2/google package and `cloud` means the google.golang.org/cloud package.
Reading ¶
The principal way to read from a Bigtable is to use the ReadRows method on *Table. A RowRange specifies a contiguous portion of a table. A Filter may be provided through RowFilter to limit or transform the data that is returned.
tbl := client.Open("mytable") ... // Read all the rows starting with "com.google.", // but only fetch the columns in the "links" family. rr := bigtable.PrefixRange("com.google.") err := tbl.ReadRows(ctx, rr, func(r Row) bool { // do something with r return true // keep going }, bigtable.RowFilter(bigtable.FamilyFilter("links"))) ...
To read a single row, use the ReadRow helper method.
r, err := tbl.ReadRow(ctx, "com.google.cloud") // "com.google.cloud" is the entire row key ...
Writing ¶
This API exposes two distinct forms of writing to a Bigtable: a Mutation and a ReadModifyWrite. The former expresses idempotent operations. The latter expresses non-idempotent operations and returns the new values of updated cells. These operations are performed by creating a Mutation or ReadModifyWrite (with NewMutation or NewReadModifyWrite), building up one or more operations on that, and then using the Apply or ApplyReadModifyWrite methods on a Table.
For instance, to set a couple of cells in a table,
tbl := client.Open("mytable") mut := bigtable.NewMutation() mut.Set("links", "maps.google.com", bigtable.Now(), []byte("1")) mut.Set("links", "golang.org", bigtable.Now(), []byte("1")) err := tbl.Apply(ctx, "com.google.cloud", mut) ...
To increment an encoded value in one cell,
tbl := client.Open("mytable") rmw := bigtable.NewReadModifyWrite() rmw.Increment("links", "golang.org", 12) // add 12 to the cell in column "links:golang.org" r, err := tbl.ApplyReadModifyWrite(ctx, "com.google.cloud", rmw) ...
Index ¶
- Constants
- type AdminClient
- func (ac *AdminClient) Close()
- func (ac *AdminClient) CreateColumnFamily(ctx context.Context, table, family string) error
- func (ac *AdminClient) CreateTable(ctx context.Context, table string) error
- func (ac *AdminClient) DeleteColumnFamily(ctx context.Context, table, family string) error
- func (ac *AdminClient) DeleteTable(ctx context.Context, table string) error
- func (ac *AdminClient) SetGCPolicy(ctx context.Context, table, family string, policy GCPolicy) error
- func (ac *AdminClient) TableInfo(ctx context.Context, table string) (*TableInfo, error)
- func (ac *AdminClient) Tables(ctx context.Context) ([]string, error)
- type ApplyOption
- type Client
- type ClusterAdminClient
- type ClusterInfo
- type Filter
- func ChainFilters(sub ...Filter) Filter
- func ColumnFilter(pattern string) Filter
- func FamilyFilter(pattern string) Filter
- func InterleaveFilters(sub ...Filter) Filter
- func LatestNFilter(n int) Filter
- func RowKeyFilter(pattern string) Filter
- func StripValueFilter() Filter
- func ValueFilter(pattern string) Filter
- type GCPolicy
- type Mutation
- func (m *Mutation) DeleteCellsInColumn(family, column string)
- func (m *Mutation) DeleteCellsInFamily(family string)
- func (m *Mutation) DeleteRow()
- func (m *Mutation) DeleteTimestampRange(family, column string, start, end Timestamp)
- func (m *Mutation) Set(family, column string, ts Timestamp, value []byte)
- type ReadItem
- type ReadModifyWrite
- type ReadOption
- type Row
- type RowRange
- type Table
- func (t *Table) Apply(ctx context.Context, row string, m *Mutation, opts ...ApplyOption) error
- func (t *Table) ApplyReadModifyWrite(ctx context.Context, row string, m *ReadModifyWrite) (Row, error)
- func (t *Table) ReadRow(ctx context.Context, row string, opts ...ReadOption) (Row, error)
- func (t *Table) ReadRows(ctx context.Context, arg RowRange, f func(Row) bool, opts ...ReadOption) error
- type TableInfo
- type Timestamp
Constants ¶
const ( // Scope is the OAuth scope for Cloud Bigtable data operations. Scope = "https://www.googleapis.com/auth/bigtable.data" // ReadonlyScope is the OAuth scope for Cloud Bigtable read-only data operations. ReadonlyScope = "https://www.googleapis.com/auth/bigtable.readonly" // AdminScope is the OAuth scope for Cloud Bigtable table admin operations. AdminScope = "https://www.googleapis.com/auth/bigtable.admin.table" // ClusterAdminScope is the OAuth scope for Cloud Bigtable cluster admin operations. ClusterAdminScope = "https://www.googleapis.com/auth/bigtable.admin.cluster" )
Scope constants for authentication credentials. These should be used when using credential creation functions such as oauth.NewServiceAccountFromFile.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AdminClient ¶
type AdminClient struct {
// contains filtered or unexported fields
}
AdminClient is a client type for performing admin operations within a specific cluster.
func NewAdminClient ¶
func NewAdminClient(ctx context.Context, project, zone, cluster string, opts ...cloud.ClientOption) (*AdminClient, error)
NewAdminClient creates a new AdminClient for a given project, zone and cluster.
func (*AdminClient) CreateColumnFamily ¶
func (ac *AdminClient) CreateColumnFamily(ctx context.Context, table, family string) error
CreateColumnFamily creates a new column family in a table.
func (*AdminClient) CreateTable ¶
func (ac *AdminClient) CreateTable(ctx context.Context, table string) error
CreateTable creates a new table in the cluster. This method may return before the table's creation is complete.
func (*AdminClient) DeleteColumnFamily ¶
func (ac *AdminClient) DeleteColumnFamily(ctx context.Context, table, family string) error
DeleteColumnFamily deletes a column family in a table and all of its data.
func (*AdminClient) DeleteTable ¶
func (ac *AdminClient) DeleteTable(ctx context.Context, table string) error
DeleteTable deletes a table and all of its data.
func (*AdminClient) SetGCPolicy ¶
func (ac *AdminClient) SetGCPolicy(ctx context.Context, table, family string, policy GCPolicy) error
SetGCPolicy specifies which cells in a column family should be garbage collected. GC executes opportunistically in the background; table reads may return data matching the GC policy.
type ApplyOption ¶
type ApplyOption interface {
// contains filtered or unexported methods
}
An ApplyOption is an optional argument to Apply.
func GetCondMutationResult ¶
func GetCondMutationResult(matched *bool) ApplyOption
GetCondMutationResult returns an ApplyOption that reports whether the conditional mutation's condition matched.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is a client for reading and writing data to tables in a cluster.
A Client is safe to use concurrently, except for its Close method.
type ClusterAdminClient ¶
type ClusterAdminClient struct {
// contains filtered or unexported fields
}
ClusterAdminClient is a client type for performing admin operations on clusters. These operations can be substantially more dangerous than those provided by AdminClient.
func NewClusterAdminClient ¶
func NewClusterAdminClient(ctx context.Context, project string, opts ...cloud.ClientOption) (*ClusterAdminClient, error)
NewClusterAdminClient creates a new ClusterAdminClient for a given project.
func (*ClusterAdminClient) Close ¶
func (cac *ClusterAdminClient) Close()
Close closes the ClusterAdminClient.
func (*ClusterAdminClient) Clusters ¶
func (cac *ClusterAdminClient) Clusters(ctx context.Context) ([]*ClusterInfo, error)
Clusters returns a list of clusters in the project.
type ClusterInfo ¶
type ClusterInfo struct { Name string // name of the cluster Zone string // GCP zone of the cluster (e.g. "us-central1-a") DisplayName string // display name for UIs ServeNodes int // number of allocated serve nodes }
ClusterInfo represents information about a cluster.
type Filter ¶
type Filter interface { String() string // contains filtered or unexported methods }
A Filter represents a row filter.
func ChainFilters ¶
ChainFilters returns a filter that applies a sequence of filters.
func ColumnFilter ¶
ColumnFilter returns a filter that matches cells whose column name matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.
func FamilyFilter ¶
FamilyFilter returns a filter that matches cells whose family name matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.
func InterleaveFilters ¶
InterleaveFilters returns a filter that applies a set of filters in parallel and interleaves the results.
func LatestNFilter ¶
LatestNFilter returns a filter that matches the most recent N cells in each column.
func RowKeyFilter ¶
RowKeyFilter returns a filter that matches cells from rows whose key matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.
func StripValueFilter ¶
func StripValueFilter() Filter
StripValueFilter returns a filter that replaces each value with the empty string.
func ValueFilter ¶
ValueFilter returns a filter that matches cells whose value matches the provided RE2 pattern. See https://github.com/google/re2/wiki/Syntax for the accepted syntax.
type GCPolicy ¶
type GCPolicy interface { String() string // contains filtered or unexported methods }
A GCPolicy represents a rule that determines which cells are eligible for garbage collection.
func IntersectionPolicy ¶
IntersectionPolicy returns a GC policy that only applies when all its sub-policies apply.
func MaxAgePolicy ¶
MaxAgePolicy returns a GC policy that applies to all cells older than the given age.
func MaxVersionsPolicy ¶
MaxVersionsPolicy returns a GC policy that applies to all versions of a cell except for the most recent n.
func UnionPolicy ¶
UnionPolicy returns a GC policy that applies when any of its sub-policies apply.
type Mutation ¶
type Mutation struct {
// contains filtered or unexported fields
}
Mutation represents a set of changes for a single row of a table.
func NewCondMutation ¶
NewCondMutation returns a conditional mutation. The given row filter determines which mutation is applied: If the filter matches any cell in the row, mtrue is applied; otherwise, mfalse is applied. Either given mutation may be nil.
func (*Mutation) DeleteCellsInColumn ¶
DeleteCellsInColumn will delete all the cells whose columns are family:column.
func (*Mutation) DeleteCellsInFamily ¶
DeleteCellsInFamily will delete all the cells whose columns are family:*.
func (*Mutation) DeleteTimestampRange ¶
DeleteTimestampRange deletes all cells whose columns are family:column and whose timestamps are in the half-open interval [start, end). If end is zero, it will be interpreted as infinity.
type ReadItem ¶
A ReadItem is returned by Read. A ReadItem contains data from a specific row and column.
type ReadModifyWrite ¶
type ReadModifyWrite struct {
// contains filtered or unexported fields
}
ReadModifyWrite represents a set of operations on a single row of a table. It is like Mutation but for non-idempotent changes. When applied, these operations operate on the latest values of the row's cells, and result in a new value being written to the relevant cell with a timestamp that is max(existing timestamp, current server time).
The application of a ReadModifyWrite is atomic; concurrent ReadModifyWrites will be executed serially by the server.
func NewReadModifyWrite ¶
func NewReadModifyWrite() *ReadModifyWrite
NewReadModifyWrite returns a new ReadModifyWrite.
func (*ReadModifyWrite) AppendValue ¶
func (m *ReadModifyWrite) AppendValue(family, column string, v []byte)
AppendValue appends a value to a specific cell's value. If the cell is unset, it will be treated as an empty value.
func (*ReadModifyWrite) Increment ¶
func (m *ReadModifyWrite) Increment(family, column string, delta int64)
Increment interprets the value in a specific cell as a 64-bit big-endian signed integer, and adds a value to it. If the cell is unset, it will be treated as zero. If the cell is set and is not an 8-byte value, the entire ApplyReadModifyWrite operation will fail.
type ReadOption ¶
type ReadOption interface {
// contains filtered or unexported methods
}
A ReadOption is an optional argument to ReadRows.
func LimitRows ¶
func LimitRows(limit int64) ReadOption
LimitRows returns a ReadOption that will limit the number of rows to be read.
func RowFilter ¶
func RowFilter(f Filter) ReadOption
RowFilter returns a ReadOption that applies f to the contents of read rows.
type Row ¶
A Row is returned by ReadRow. The map is keyed by column family (the prefix of the column name before the colon). The values are the returned ReadItems for that column family in the order returned by Read.
type RowRange ¶
type RowRange struct {
// contains filtered or unexported fields
}
A RowRange is used to describe the rows to be read. A RowRange is a half-open interval [Start, Limit) encompassing all the rows with keys at least as large as Start, and less than Limit. (Bigtable string comparison is the same as Go's.) A RowRange can be unbounded, encompassing all keys at least as large as Start.
func InfiniteRange ¶
InfiniteRange returns the RowRange consisting of all keys at least as large as start.
func PrefixRange ¶
PrefixRange returns a RowRange consisting of all keys starting with the prefix.
type Table ¶
type Table struct {
// contains filtered or unexported fields
}
A Table refers to a table.
A Table is safe to use concurrently.
func (*Table) ApplyReadModifyWrite ¶
func (t *Table) ApplyReadModifyWrite(ctx context.Context, row string, m *ReadModifyWrite) (Row, error)
ApplyReadModifyWrite applies a ReadModifyWrite to a specific row. It returns the newly written cells.
func (*Table) ReadRow ¶
ReadRow is a convenience implementation of a single-row reader. A missing row will return a zero-length map and a nil error.
func (*Table) ReadRows ¶
func (t *Table) ReadRows(ctx context.Context, arg RowRange, f func(Row) bool, opts ...ReadOption) error
ReadRows reads rows from a table. f is called for each row. If f returns false, the stream is shut down and ReadRows returns. f owns its argument, and f is called serially.
By default, the yielded rows will contain all values in all cells. Use RowFilter to limit the cells returned.
type TableInfo ¶
type TableInfo struct {
Families []string
}
TableInfo represents information about a table.
type Timestamp ¶
type Timestamp int64
Timestamp is in units of microseconds since 1 January 1970.
const ServerTime Timestamp = -1
ServerTime is a specific Timestamp that may be passed to (*Mutation).Set. It indicates that the server's timestamp should be used.
Directories ¶
Path | Synopsis |
---|---|
Package bttest contains test helpers for working with the bigtable package.
|
Package bttest contains test helpers for working with the bigtable package. |
cmd
|
|
cbt
Cbt is a tool for doing basic interactions with Cloud Bigtable.
|
Cbt is a tool for doing basic interactions with Cloud Bigtable. |
loadtest
Loadtest does some load testing through the Go client library for Cloud Bigtable.
|
Loadtest does some load testing through the Go client library for Cloud Bigtable. |
internal
|
|
cbtrc
Package cbtrc encapsulates common code for reading .cbtrc files.
|
Package cbtrc encapsulates common code for reading .cbtrc files. |
cluster_data_proto
Package google_bigtable_admin_cluster_v1 is a generated protocol buffer package.
|
Package google_bigtable_admin_cluster_v1 is a generated protocol buffer package. |
cluster_service_proto
Package google_bigtable_admin_cluster_v1 is a generated protocol buffer package.
|
Package google_bigtable_admin_cluster_v1 is a generated protocol buffer package. |
data_proto
Package google_bigtable_v1 is a generated protocol buffer package.
|
Package google_bigtable_v1 is a generated protocol buffer package. |
duration_proto
Package google_protobuf is a generated protocol buffer package.
|
Package google_protobuf is a generated protocol buffer package. |
empty
Package google_protobuf is a generated protocol buffer package.
|
Package google_protobuf is a generated protocol buffer package. |
service_proto
Package google_bigtable_v1 is a generated protocol buffer package.
|
Package google_bigtable_v1 is a generated protocol buffer package. |
table_data_proto
Package google_bigtable_admin_table_v1 is a generated protocol buffer package.
|
Package google_bigtable_admin_table_v1 is a generated protocol buffer package. |
table_service_proto
Package google_bigtable_admin_table_v1 is a generated protocol buffer package.
|
Package google_bigtable_admin_table_v1 is a generated protocol buffer package. |