docstore

package
v0.43.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 15, 2019 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package docstore provides a portable implementation of a document store. TODO(jba): link to an explanation of document stores (https://en.wikipedia.org/wiki/Document-oriented_database?) TODO(jba): expand package doc to batch other Go CDK APIs.

Representing Times

Docstore can store and retrieve values of type time.Time, with two caveats. First, the timezone may not be preserved. Second, Docstore guarantees only that time.Time values are represented to millisecond precision. Many providers will do better, but if you need to be sure that times are stored with nanosecond precision, convert the time.Time to another type before storing, and re-create when you retrieve it. For instance, you could call the time's UnixNano method to get an int64, and get the original time back (in the local timezone) with the time.Unix function.

Revisions

Docstore gives every document a revision when it is created. Docstore uses the field name "DocstoreRevision" (stored in the constant docstore.RevisionField) to hold the revision. Whenever a document is modified, its revision changes. Revisions can be used for optimistic locking: whenever a Put, Replace, Update or Delete action is given a document with a revision, then an error for which gcerrors.Code returns FailedPrecondition or NotFound is returned if the stored document's revision does not match the given document's. Thus a Get followed by one of those write actions will fail if the document was changed between the Get and the write.

Since different providers use different types for revisions, the type of the revision field is unspecified. When defining a struct for storing docstore data, define the field to be of type interface{}. For example, 

type User { Name string; DocstoreRevision interface{} }

If a struct doesn't have a DocstoreRevision field, then the logic described above won't apply to documents read and written with that struct. All writes with the struct will succeed even if the document was changed since the last Get.

Queries

Docstore supports querying within a collection. Call the Query method on Collection to obtain a Query value, then build your query by calling Query methods like Where, Limit and so on. Finally, call the Get method on the query to execute it. The result is an iterator, whose use is described below. 

iter := coll.Query().Where("size", ">", 10).Limit(5).Get(ctx)

The Where methods defines a filter condition, much like a WHERE clause in SQL. Conditions are of the form "field op value", where field is any document field path (including dot-separated paths), op is one of "=", ">", "<", ">=" or "<=", and value can be a string or number.

You can make multiple Where calls. In some cases, parts of a Where clause may be processed on the client rather than natively by the provider, which may have performance implications for large result sets. See the provider-specific package doc for details.

The Limit method specifies the maximum number of results to return.

Docstore provides no guarantees about the ordering of results. There is no way to ask for a particular ordering.

Use the DocumentIterator returned from Query.Get by repeatedly calling its Next method until it returns io.EOF. Always call Stop when you are finished with an iterator. 

 iter := coll.Query().Where("size", ">", 10).Limit(5).Get(ctx)
 defer iter.Stop()
 for {
     m := map[string]interface{}{}
     err := iter.Next(ctx, m)
     if err == io.EOF {
         break
     }
     if err != nil {
         return err
     }
     fmt.Println(m)
}
Example (Actions) 
package main

import (
	"context"

	"github.com/eliben/gocdkx/internal/docstore"
)

func main() {
	ctx := context.Background()

	type Player struct {
		Name  string // Name is the key
		Score int
	}

	fred := &Player{Name: "Fred", Score: 18}
	got := &Player{Name: "Fred"}
	coll := docstore.NewCollection(nil)
	errs := coll.Actions().
		Put(fred).
		Get(got).
		Do(ctx)
	_ = errs
}

Output:

Index

Examples

Constants

View Source 
const RevisionField = "DocstoreRevision"

RevisionField is the name of the document field used for document revision information, to implement optimistic locking. See the Revisions section of the package documentation.

Variables

View Source 
var NewCollection = newCollection

NewCollection is intended for use by provider implementations.

Functions

This section is empty.

Types

type Action 

type Action struct {
	// contains filtered or unexported fields
}

An Action is a read or write on a single document. Use the methods of ActionList to create and execute Actions.

type ActionList 

type ActionList struct {
	// contains filtered or unexported fields
}

An ActionList is a group of actions that affect a single collection.

By default, the actions in the list are performed in order from the point of view of the client. However, the actions may not be performed atomically, and there is no guarantee that a Get following a write will see the value just written (for example, if the provider is eventually consistent). Execution stops with the first failed action.

If the Unordered method is called on an ActionList, then the actions may be executed in any order, perhaps concurrently. All actions will be executed, even if some fail.

func (*ActionList) BeforeDo 

func (l *ActionList) BeforeDo(f func(asFunc func(interface{}) bool) error) *ActionList

BeforeDo takes a callback function that will be called before the ActionList is executed by the underlying provider's action functionality. The callback takes a parameter, asFunc, that converts its argument to provider-specific types. See https://godoc.org/github.com/eliben/gocdkx#hdr-As for background information.

func (*ActionList) Create 

func (l *ActionList) Create(doc Document) *ActionList

Create adds an action that creates a new document. The document must not already exist; an error for which gcerrors.Code returns AlreadyExists is returned if it does. If the document doesn't have key fields, it will be given key fields with unique values. TODO(jba): treat zero values for struct fields as not present?

func (*ActionList) Delete 

func (l *ActionList) Delete(doc Document) *ActionList

Delete adds an action that deletes a document. Only the key fields and RevisionField of doc are used. See the Revisions section of the package documentation for how revisions are handled. If doc has no revision and the document doesn't exist, nothing happens and no error is returned.

func (*ActionList) Do 

func (l *ActionList) Do(ctx context.Context) error

Do executes the action list.

If Do returns a non-nil error, it will be of type ActionListError. If any action fails, the returned error will contain the position in the ActionList of each failed action (but see the discussion of unordered mode, below). As a special case, none of the actions will be executed if any is invalid (for example, a Put whose document is missing its key field).

In ordered mode (when the Unordered method was not called on the list), execution will stop after the first action that fails.

In unordered mode, all the actions will be executed. Docstore tries to execute the actions as efficiently as possible. Sometimes this makes it impossible to attribute failures to specific actions; in such cases, the returned ActionListError will have entries whose Index field is negative.

func (*ActionList) Get 

func (l *ActionList) Get(doc Document, fps ...FieldPath) *ActionList

Get adds an action that retrieves a document. Only the key fields of doc are used. If fps is omitted, doc will contain all the fields of the retrieved document. If fps is present, only the given field paths are retrieved, in addition to the revision field. It is undefined whether other fields of doc at the time of the call are removed, unchanged, or zeroed, so for portable behavior doc should contain only the key fields.

func (*ActionList) Put 

func (l *ActionList) Put(doc Document) *ActionList

Put adds an action that adds or replaces a document. The key fields must be set. The document may or may not already exist. See the Revisions section of the package documentation for how revisions are handled.

func (*ActionList) Replace 

func (l *ActionList) Replace(doc Document) *ActionList

Replace adds an action that replaces a document. The key fields must be set. The document must already exist; an error for which gcerrors.Code returns NotFound is returned if it does not. See the Revisions section of the package documentation for how revisions are handled.

func (*ActionList) Unordered 

func (l *ActionList) Unordered() *ActionList

After Unordered is called, Do may execute the actions in any order. All actions will be executed, even if some fail.

func (*ActionList) Update 

func (l *ActionList) Update(doc Document, mods Mods) *ActionList

Update atomically applies Mods to doc, which must exist. Only the key and revision fields of doc are used.

A modification will create a field if it doesn't exist.

No field path in mods can be a prefix of another. (It makes no sense to, say, set foo but increment foo.bar.)

See the Revisions section of the package documentation for how revisions are handled.

It is undefined whether updating a sub-field of a non-map field will succeed. For instance, if the current document is {a: 1} and Update is called with the mod "a.b": 2, then either Update will fail, or it will succeed with the result {a: {b: 2}}.

Update does not modify its doc argument. To obtain the new value of the document, call Get after calling Update.

type ActionListError 

type ActionListError []struct {
	Index int
	Err   error
}

An ActionListError is returned by ActionList.Do. It contains all the errors encountered while executing the ActionList, and the positions of the corresponding actions.

func (ActionListError) Error 

func (e ActionListError) Error() string

func (ActionListError) Unwrap 

func (e ActionListError) Unwrap() error

Unwrap returns the error in e, if there is exactly one. If there is more than one error, Unwrap returns nil, since there is no way to determine which should be returned.

type Collection 

type Collection struct {
	// contains filtered or unexported fields
}

A Collection is a set of documents. TODO(jba): make the docstring look more like blob.Bucket.

func OpenCollection 

func OpenCollection(ctx context.Context, urlstr string) (*Collection, error)

OpenCollection opens the collection identified by the URL given. See the URLOpener documentation in provider-specific subpackages for details on supported URL formats, and https://github.com/eliben/gocdkx/concepts/urls/ for more information.

func (*Collection) Actions 

func (c *Collection) Actions() *ActionList

Actions returns an ActionList that can be used to perform actions on the collection's documents.

func (*Collection) As 

func (c *Collection) As(i interface{}) bool

As converts i to provider-specific types. See https://godoc.org/github.com/eliben/gocdkx#hdr-As for background information, the "As" examples in this package for examples, and the provider-specific package documentation for the specific types supported for that provider.

func (*Collection) Create 

func (c *Collection) Create(ctx context.Context, doc Document) error

Create is a convenience for building and running a single-element action list. See ActionList.Create.

func (*Collection) Delete 

func (c *Collection) Delete(ctx context.Context, doc Document) error

Delete is a convenience for building and running a single-element action list. See ActionList.Delete.

func (*Collection) Get 

func (c *Collection) Get(ctx context.Context, doc Document, fps ...FieldPath) error

Get is a convenience for building and running a single-element action list. See ActionList.Get.

func (*Collection) Put 

func (c *Collection) Put(ctx context.Context, doc Document) error

Put is a convenience for building and running a single-element action list. See ActionList.Put.

func (*Collection) Query 

func (c *Collection) Query() *Query

Query creates a new Query over the collection.

func (*Collection) Replace 

func (c *Collection) Replace(ctx context.Context, doc Document) error

Replace is a convenience for building and running a single-element action list. See ActionList.Replace.

func (*Collection) Update 

func (c *Collection) Update(ctx context.Context, doc Document, mods Mods) error

Update is a convenience for building and running a single-element action list. See ActionList.Update.

type CollectionURLOpener 

type CollectionURLOpener interface {
	OpenCollectionURL(ctx context.Context, u *url.URL) (*Collection, error)
}

CollectionURLOpener opens a collection of documents based on a URL. The opener must not modify the URL argument. It must be safe to call from multiple goroutines.

This interface is generally implemented by types in driver packages.

type Document 

type Document = interface{}

A Document is a set of field-value pairs. One or more fields, called the key fields, must uniquely identify the document in the collection. You specify the key fields when you open a provider collection. A field name must be a valid UTF-8 string that does not contain a '.'.

A Document can be represented as a map[string]int or a pointer to a struct. For structs, the exported fields are the document fields.

type DocumentIterator 

type DocumentIterator struct {
	// contains filtered or unexported fields
}

DocumentIterator iterates over documents.

Always call Stop on the iterator.

func (*DocumentIterator) As 

func (it *DocumentIterator) As(i interface{}) bool

As converts i to provider-specific types. See https://godoc.org/github.com/eliben/gocdkx#hdr-As for background information, the "As" examples in this package for examples, and the provider-specific package documentation for the specific types supported for that provider.

func (*DocumentIterator) Next 

func (it *DocumentIterator) Next(ctx context.Context, dst Document) error

Next stores the next document in dst. It returns io.EOF if there are no more documents. Once Next returns an error, it will always return the same error.

func (*DocumentIterator) Stop 

func (it *DocumentIterator) Stop()

Stop stops the iterator. Calling Next on a stopped iterator will return io.EOF, or the error that Next previously returned.

type FieldPath 

type FieldPath string

A FieldPath is a dot-separated sequence of UTF-8 field names. Examples: 

room
room.size
room.size.width

A FieldPath can be used select top-level fields or elements of sub-documents. There is no way to select a single list element.

type Mods 

type Mods map[FieldPath]interface{}

Mods is a map from field paths to modifications. At present, a modification is one of: - nil, to delete the field - any other value, to set the field to that value TODO(jba): add other kinds of modification See ActionList.Update.

type Query 

type Query struct {
	// contains filtered or unexported fields
}

Query represents a query over a collection.

func (*Query) BeforeQuery 

func (q *Query) BeforeQuery(f func(asFunc func(interface{}) bool) error) *Query

BeforeQuery takes a callback function that will be called before the Query is executed to the underlying provider's query functionality. The callback takes a parameter, asFunc, that converts its argument to provider-specific types. See https://godoc.org/github.com/eliben/gocdkx#hdr-As for background information.

func (*Query) Delete 

func (q *Query) Delete(ctx context.Context) error

Delete deletes all the documents specified by the query. It is an error if the query has a limit.

func (*Query) Get 

func (q *Query) Get(ctx context.Context, fps ...FieldPath) *DocumentIterator

Get returns an iterator for retrieving the documents specified by the query. If field paths are provided, only those paths are set in the resulting documents.

Call Stop on the iterator when finished.

func (*Query) Limit 

func (q *Query) Limit(n int) *Query

Limit will limit the results to at most n documents.

func (*Query) Plan 

func (q *Query) Plan(fps ...FieldPath) (string, error)

Plan describes how the query would be executed if its Get method were called with the given field paths. Plan uses only information available to the client, so it cannot know whether a service uses indexes or scans internally.

func (*Query) Where 

func (q *Query) Where(fieldpath, op string, value interface{}) *Query

Where expresses a condition on the query. Valid ops are: "=", ">", "<", ">=", "<=".

type URLMux 

type URLMux struct {
	// contains filtered or unexported fields
}

URLMux is a URL opener multiplexer. It matches the scheme of the URLs against a set of registered schemes and calls the opener that matches the URL's scheme. See https://github.com/eliben/gocdkx/concepts/urls/ for more information.

The zero value is a multiplexer with no registered scheme.

func DefaultURLMux 

func DefaultURLMux() *URLMux

DefaultURLMux returns the URLMux used by OpenCollection.

Driver packages can use this to register their CollectionURLOpener on the mux.

func (*URLMux) CollectionSchemes 

func (mux *URLMux) CollectionSchemes() []string

CollectionSchemes returns a sorted slice of the registered Collection schemes.

func (*URLMux) OpenCollection 

func (mux *URLMux) OpenCollection(ctx context.Context, urlstr string) (*Collection, error)

OpenCollection calls OpenCollectionURL with the URL parsed from urlstr. OpenCollection is safe to call from multiple goroutines.

func (*URLMux) OpenCollectionURL 

func (mux *URLMux) OpenCollectionURL(ctx context.Context, u *url.URL) (*Collection, error)

OpenCollectionURL dispatches the URL to the opener that is registered with the URL's scheme. OpenCollectionURL is safe to call from multiple goroutines.

func (*URLMux) RegisterCollection 

func (mux *URLMux) RegisterCollection(scheme string, opener CollectionURLOpener)

RegisterCollection registers the opener with the given scheme. If an opener already exists for the scheme, RegisterCollection panics.

func (*URLMux) ValidCollectionScheme 

func (mux *URLMux) ValidCollectionScheme(scheme string) bool

ValidCollectionScheme returns true iff scheme has been registered for Collections.

Source Files

View all Source files

Directories

Path Synopsis
Package driver defines a set of interfaces that the docstore package uses to interact with the underlying services.
 Package driver defines a set of interfaces that the docstore package uses to interact with the underlying services.
Package drivertest provides a conformance test for implementations of driver.
 Package drivertest provides a conformance test for implementations of driver.
Package dynamodocstore provides a docstore implementation backed by AWS DynamoDB.
 Package dynamodocstore provides a docstore implementation backed by AWS DynamoDB.
Package firedocstore provides an implementation of the docstore API for Google Cloud Firestore.
 Package firedocstore provides an implementation of the docstore API for Google Cloud Firestore.
internal
fields
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
 Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
Package memdocstore provides an in-memory implementation of the docstore API.
 Package memdocstore provides an in-memory implementation of the docstore API.
Package mongodocstore provides an implementation of the docstore API for MongoDB.
 Package mongodocstore provides an implementation of the docstore API for MongoDB.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.