firestore

Documentation

Index

• Constants
• func AlphaNumID() string
• func EnsureNotEmulator()
• func FixTimestamp(t time.Time) time.Time
• func QuerySnapshotChannel(ctx context.Context, q firestore.Query) <-chan *firestore.QuerySnapshot
• type Client
  • func NewClient(ctx context.Context, project, app, instance string, ts oauth2.TokenSource) (*Client, error)
  • func NewClientForTesting(ctx context.Context, t sktest.TestingT) (*Client, util.CleanupFunc)
  • func (c *Client) BatchWrite(ctx context.Context, total, batchSize int, maxWriteTime time.Duration, ...) error
  • func (c *Client) Collection(path string) *firestore.CollectionRef
  • func (c *Client) Collections(ctx context.Context) *firestore.CollectionIterator
  • func (c *Client) CountReadQueryAndRows(path string, rowCount int)
  • func (c *Client) CountWriteQueryAndRows(path string, rowCount int)
  • func (c *Client) Create(ctx context.Context, ref *firestore.DocumentRef, data interface{}, ...) (*firestore.WriteResult, error)
  • func (c *Client) Delete(ctx context.Context, ref *firestore.DocumentRef, attempts int, ...) (*firestore.WriteResult, error)
  • func (c *Client) Doc(path string) *firestore.DocumentRef
  • func (c *Client) Get(ctx context.Context, ref *firestore.DocumentRef, attempts int, ...) (*firestore.DocumentSnapshot, error)
  • func (c *Client) GetAllDescendantDocuments(ctx context.Context, ref *firestore.DocumentRef, attempts int, ...) ([]*firestore.DocumentRef, error)
  • func (c *Client) IterDocs(ctx context.Context, name, detail string, query firestore.Query, attempts int, ...) error
  • func (c *Client) IterDocsInParallel(ctx context.Context, name, detail string, queries []firestore.Query, ...) error
  • func (c *Client) RecurseDocs(ctx context.Context, name string, ref *firestore.DocumentRef, attempts int, ...) error
  • func (c *Client) RecursiveDelete(ctx context.Context, ref *firestore.DocumentRef, attempts int, ...) error
  • func (c *Client) RunTransaction(ctx context.Context, name, detail string, attempts int, timeout time.Duration, ...) error
  • func (c *Client) Set(ctx context.Context, ref *firestore.DocumentRef, data interface{}, ...) (*firestore.WriteResult, error)
  • func (c *Client) Update(ctx context.Context, ref *firestore.DocumentRef, attempts int, ...) (*firestore.WriteResult, error)
• type DocumentRefSlice
  • func (s DocumentRefSlice) Len() int
  • func (s DocumentRefSlice) Less(i, j int) bool
  • func (s DocumentRefSlice) Swap(i, j int)

Constants

const (
	// Firestore has a timestamp resolution of one microsecond.
	TS_RESOLUTION = time.Microsecond

	// List all apps here as constants.
	APP_TASK_SCHEDULER = "task-scheduler"

	// Base wait time between attempts.
	BACKOFF_WAIT = 5 * time.Second

	// Project ID. At the moment only the skia-firestore project has
	// Firestore enabled.
	FIRESTORE_PROJECT = "skia-firestore"

	// List all instances here as constants.
	INSTANCE_PROD = "prod"
	INSTANCE_TEST = "test"

	// Maximum number of docs in a single transaction.
	MAX_TRANSACTION_DOCS = 500

	// IterDocs won't iterate for longer than this amount of time at once,
	// otherwise we risk server timeouts. Instead, it will stop and resume
	// iteration.
	MAX_ITER_TIME = 50 * time.Second

	// Length of IDs returned by AlphaNumID.
	ID_LEN = 20

	EmulatorEnvVar = "FIRESTORE_EMULATOR_HOST"
)

Functions

func AlphaNumID

func AlphaNumID() string

AlphaNumID generates a fixed-length alphanumeric document ID using crypto/rand. Panics if crypto/rand fails to generate random bytes, eg. it cannot read from /dev/urandom.

Motivation: the Go client library for Firestore generates URL-safe base64- encoded IDs, whic may not be desirable for all use cases (eg. passing an ID which may contain a '-' on the command line would require some escaping).

func EnsureNotEmulator

func EnsureNotEmulator()

EnsureNotEmulator will panic if it detects the Firestore Emulator is configured. Trying to authenticate to the emulator results in errors like: "Failed to initialize Cloud Datastore: dialing: options.WithoutAuthentication is incompatible with any option that provides credentials"

func FixTimestamp

func FixTimestamp(t time.Time) time.Time

FixTimestamp adjusts the given timestamp for storage in Firestore. Firestore only supports microsecond precision, and we always want to store UTC.

func QuerySnapshotChannel

func QuerySnapshotChannel(ctx context.Context, q firestore.Query) <-chan *firestore.QuerySnapshot

QuerySnapshotChannel is a helper for firestore.QuerySnapshotIterator which passes each QuerySnapshot along the returned channel. QuerySnapshotChannel returns the channel immediately but spins up a goroutine which runs indefinitely or until an error occurs, in which case the channel is closed and an error is logged.

A couple of notes:

1. QuerySnapshotIterator immediately produces a QuerySnapshot containing all of the current results for the query, then blocks until those results change. QuerySnapshot.Changes contains the changes since the last snapshot (and will therefore be empty on the first snapshot), while its Documents field is an iterator which will obtain all of the updated results.
2. If the consumer of the QuerySnapshotChannel is slower than the QuerySnapshotIterator, the most recent snapshot will get stuck waiting to be passed along the channel and thus may be out of date by the time the consumer sees it. If your consumer is slow, consider adding a buffered channel as an intermediate, or a goroutine to collect batches of snapshots to be processed.

Types

type Client

type Client struct {
	*firestore.Client
	ParentDoc *firestore.DocumentRef
	// contains filtered or unexported fields
}

Client is a Cloud Firestore client which enforces separation of app/instance data via separate collections and documents. All references to collections and documents are automatically prefixed with the app name as the top-level collection and instance name as the parent document.

func NewClient

func NewClient(ctx context.Context, project, app, instance string, ts oauth2.TokenSource) (*Client, error)

NewClient returns a Cloud Firestore client which enforces separation of app/ instance data via separate collections and documents. All references to collections and documents are automatically prefixed with the app name as the top-level collection and instance name as the parent document.

func NewClientForTesting

func NewClientForTesting(ctx context.Context, t sktest.TestingT) (*Client, util.CleanupFunc)

NewClientForTesting returns a Client and ensures that it will connect to the Firestore emulator. The Client's instance name will be randomized to ensure concurrent tests don't interfere with each other. It also returns a CleanupFunc that closes the Client.

This function doesn't call unittest.RequiresFirestoreEmulator(t). See //go/firestore/testutils for a version of NewClientForTesting that does.

func (*Client) BatchWrite

func (c *Client) BatchWrite(ctx context.Context, total, batchSize int, maxWriteTime time.Duration, startBatch *firestore.WriteBatch, fn func(b *firestore.WriteBatch, i int) error) error

BatchWrite executes breaks a large amount of writes into batches of the given size and commits them, using retries that backoff exponentially up to the maxWriteTime. If a single batch fails, even with backoff, an error is returned, without attempting any further batches and without rolling back the previous successful batches (at present, rollback of previous batches is impossible to do correctly in the general case). Callers should also call CountWriteQueryAndRows to update counts of the affected collection(s).

func (*Client) Collection

func (c *Client) Collection(path string) *firestore.CollectionRef

See documentation for firestore.Client.

func (*Client) Collections

func (c *Client) Collections(ctx context.Context) *firestore.CollectionIterator

See documentation for firestore.Client.

func (*Client) CountReadQueryAndRows

func (c *Client) CountReadQueryAndRows(path string, rowCount int)

CountReadQueryAndRows increments the metric counters for a path. The "read queries" metric will be incremented by one and the "rows read" by the amount given. This should be done when a client does some amount of reading from firestore w/o using the helpers in this package (e.g. GetAll).

func (*Client) CountWriteQueryAndRows

func (c *Client) CountWriteQueryAndRows(path string, rowCount int)

CountWriteQueryAndRows increments the metric counters for a path. The "write queries" metric will be incremented by one and the "rows written" by the amount given. This should be done when a client does some amount of writing to firestore w/o using the helpers in this package (or uses BatchWrite).

func (*Client) Create

func (c *Client) Create(ctx context.Context, ref *firestore.DocumentRef, data interface{}, attempts int, timeout time.Duration) (*firestore.WriteResult, error)

See documentation for firestore.DocumentRef.Create(). Uses the given maximum number of attempts and the given per-attempt timeout.

func (*Client) Delete

func (c *Client) Delete(ctx context.Context, ref *firestore.DocumentRef, attempts int, timeout time.Duration, preconds ...firestore.Precondition) (*firestore.WriteResult, error)

See documentation for firestore.DocumentRef.Delete(). Uses the given maximum number of attempts and the given per-attempt timeout.

func (*Client) Doc

func (c *Client) Doc(path string) *firestore.DocumentRef

See documentation for firestore.Client.

func (*Client) Get

func (c *Client) Get(ctx context.Context, ref *firestore.DocumentRef, attempts int, timeout time.Duration) (*firestore.DocumentSnapshot, error)

Get retrieves the given document, using the given timeout and maximum number of attempts. Returns (nil, nil) if the document does not exist. Uses the given maximum number of attempts and the given per-attempt timeout.

func (*Client) GetAllDescendantDocuments

func (c *Client) GetAllDescendantDocuments(ctx context.Context, ref *firestore.DocumentRef, attempts int, timeout time.Duration) ([]*firestore.DocumentRef, error)

GetAllDescendantDocuments returns a slice of DocumentRefs for every descendant of the given Document. This includes missing documents, ie. those which do not exist but have sub-documents. This function does nothing to account for documents which may be added or modified while it is running.

func (*Client) IterDocs

func (c *Client) IterDocs(ctx context.Context, name, detail string, query firestore.Query, attempts int, timeout time.Duration, callback func(*firestore.DocumentSnapshot) error) error

IterDocs is a convenience function which executes the given query and calls the given callback function for each document. Uses the given maximum number of attempts and the given per-attempt timeout. IterDocs automatically stops iterating after enough time has passed and re-issues the query, continuing where it left off. This is to avoid server-side timeouts resulting from iterating a large number of results. Note that this behavior may result in individual results coming from inconsistent snapshots.

func (*Client) IterDocsInParallel

func (c *Client) IterDocsInParallel(ctx context.Context, name, detail string, queries []firestore.Query, attempts int, timeout time.Duration, callback func(int, *firestore.DocumentSnapshot) error) error

IterDocsInParallel is a convenience function which executes the given queries in multiple goroutines and calls the given callback function for each document. Uses the maximum number of attempts and the given per-attempt timeout for each goroutine. Each callback includes the goroutine index. IterDocsInParallel automatically stops iterating after enough time has passed and re-issues the query, continuing where it left off. This is to avoid server-side timeouts resulting from iterating a large number of results. Note that this behavior may result in individual results coming from inconsistent snapshots.

func (*Client) RecurseDocs

func (c *Client) RecurseDocs(ctx context.Context, name string, ref *firestore.DocumentRef, attempts int, timeout time.Duration, fn func(*firestore.DocumentRef) error) error

RecurseDocs runs the given func for every descendent of the given document. This includes missing documents, ie. those which do not exist but have sub- documents. The func is run for leaf documents before their parents. This function does nothing to account for documents which may be added or modified while it is running.

func (*Client) RecursiveDelete

func (c *Client) RecursiveDelete(ctx context.Context, ref *firestore.DocumentRef, attempts int, timeout time.Duration) error

RecursiveDelete deletes the given document and all of its descendant documents and collections. The given maximum number of attempts and the given per-attempt timeout apply for each delete operation, as opposed to the whole series of operations. This function does nothing to account for documents which may be added or modified while it is running.

func (*Client) RunTransaction

func (c *Client) RunTransaction(ctx context.Context, name, detail string, attempts int, timeout time.Duration, fn func(context.Context, *firestore.Transaction) error) error

RunTransaction runs the given function in a transaction. Uses the given maximum number of attempts and the given per-attempt timeout.

func (*Client) Set

func (c *Client) Set(ctx context.Context, ref *firestore.DocumentRef, data interface{}, attempts int, timeout time.Duration, opts ...firestore.SetOption) (*firestore.WriteResult, error)

See documentation for firestore.DocumentRef.Set(). Uses the given maximum number of attempts and the given per-attempt timeout.

func (*Client) Update

func (c *Client) Update(ctx context.Context, ref *firestore.DocumentRef, attempts int, timeout time.Duration, updates []firestore.Update, preconds ...firestore.Precondition) (*firestore.WriteResult, error)

See documentation for firestore.DocumentRef.Update(). Uses the given maximum number of attempts and the given per-attempt timeout.

type DocumentRefSlice

type DocumentRefSlice []*firestore.DocumentRef

DocumentRefSlice is a slice of DocumentRefs, used for sorting.

func (DocumentRefSlice) Len

func (s DocumentRefSlice) Len() int

func (DocumentRefSlice) Less

func (s DocumentRefSlice) Less(i, j int) bool

func (DocumentRefSlice) Swap

func (s DocumentRefSlice) Swap(i, j int)