goon

package module
v0.0.0-...-a8d9fba Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 27, 2020 License: ISC Imports: 19 Imported by: 0

README

goon

An autocaching interface to the app engine datastore for Go. Designed to be similar to the python NDB package.

Documentation: http://godoc.org/github.com/mjibson/goon

Documentation

Overview

Package goon provides an autocaching interface to the app engine datastore similar to the python NDB package.

Goon differs from the datastore package in various ways: it remembers the appengine Context, which need only be specified once at creation time; kinds need not be specified as they are computed, by default, from a type's name; keys are inferred from specially-tagged fields on types, removing the need to pass key objects around.

In general, the difference is that Goon's API is identical to the datastore API, it's just shorter.

Keys in Goon are stored in the structs themselves. Below is an example struct with a field to specify the id (see the Key Specifications section below for full documentation). 

type User struct {
	Id    string `datastore:"-" goon:"id"`
	Name  string
}

Thus, to get a User with id 2: 

userid := 2
g := goon.NewGoon(r)
u := &User{Id: userid}
g.Get(u)

Key Specifications

For both the Key and KeyError functions, src must be a S or *S for some struct type S. The key is extracted based on various fields of S. If a field of type int64 or string has a struct tag named goon with value "id", it is used as the key's id. If a field of type *datastore.Key has a struct tag named goon with value "parent", it is used as the key's parent. If a field of type string has a struct tag named goon with value "kind", it is used as the key's kind. The "kind" field supports an optional second parameter which is the default kind name. If no kind field exists, the struct's name is used. These fields should all have their datastore field marked as "-".

Example, with kind User: 

type User struct {
	Id    string `datastore:"-" goon:"id"`
	Read  time.Time
}

Example, with kind U if _kind is the empty string: 

type User struct {
	_kind string `goon:"kind,U"`
	Id    string `datastore:"-" goon:"id"`
	Read  time.Time
}

To override kind of a single entity to UserKind: 

u := User{_kind: "UserKind"}

An example with both parent and kind: 

type UserData struct {
	Id     string         `datastore:"-" goon:"id"`
	_kind  string         `goon:"kind,UD"`
	Parent *datastore.Key `datastore:"-" goon:"parent"`
	Data   []byte
}

Features

Datastore interaction with: Get, GetMulti, Put, PutMulti, Delete, DeleteMulti, Queries.

All key-based operations backed by memory and memcache.

Per-request, in-memory cache: fetch the same key twice, the second request is served from local memory.

Intelligent multi support: running GetMulti correctly fetches from memory, then memcache, then the datastore; each tier only sends keys off to the next one if they were missing.

Memcache control variance: long memcache requests are cancelled.

Transactions use a separate context, but locally cache any results on success.

Automatic kind naming: struct names are inferred by reflection, removing the need to manually specify key kinds.

Simpler API than appengine/datastore.

API comparison between goon and datastore

put with incomplete key

datastore: 

type Group struct {
	Name string
}
c := appengine.NewContext(r)
g := &Group{Name: "name"}
k := datastore.NewIncompleteKey(c, "Group", nil)
err := datastore.Put(c, k, g)

goon: 

type Group struct {
	Id   int64 `datastore:"-" goon:"id"`
	Name string
}
n := goon.NewGoon(r)
g := &Group{Name: "name"}
err := n.Put(g)

get with known key

datastore: 

type Group struct {
	Name string
}
c := appengine.NewContext(r)
g := &Group{}
k := datastore.NewKey(c, "Group", "", 1, nil)
err := datastore.Get(c, k, g)

goon: 

type Group struct {
	Id   int64 `datastore:"-" goon:"id"`
	Name string
}
n := goon.NewGoon(r)
g := &Group{Id: 1}
err := n.Get(g)

Memcache Control Variance

Memcache is generally fast. When it is slow, goon will timeout the memcache requests and proceed to use the datastore directly. The memcache put and get timeout variables determine how long to wait for various kinds of requests. The default settings were determined experimentally and should provide reasonable defaults for most applications.

See: http://talks.golang.org/2013/highperf.slide#23

PropertyLoadSaver support

Structs that implement the PropertyLoadSaver interface are guaranteed to call the Save() method once and only once per Put/PutMulti call and never elsewhere. Similarly the Load() method is guaranteed to be called once and only once per Get/GetMulti/GetAll/Next call and never elsewhere.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// LogErrors issues appengine.Context.Errorf on any error.
	LogErrors = true
	// LogTimeoutErrors issues appengine.Context.Warningf on memcache timeout errors.
	LogTimeoutErrors = false

	// MemcachePutTimeoutThreshold is the number of bytes after which the large
	// timeout setting is added to the small timeout setting. This repeats.
	// Which means that with a 20 KiB threshold and a 100 KiB memcache payload,
	// the final timeout is MemcachePutTimeoutSmall + 5*MemcachePutTimeoutLarge
	MemcachePutTimeoutThreshold = 20 * 1024 // 20 KiB
	// MemcachePutTimeoutSmall is the minimum time to wait during memcache
	// Put operations before aborting them and using the datastore.
	MemcachePutTimeoutSmall = 5 * time.Millisecond
	// MemcachePutTimeoutLarge is the amount of extra time to wait for larger
	// memcache Put requests. See also MemcachePutTimeoutThreshold.
	MemcachePutTimeoutLarge = 1 * time.Millisecond
	// MemcacheGetTimeout is the amount of time to wait for all memcache Get
	// requests, per key fetched. Because we can't really know how big entities
	// we are requesting, this setting should be for the maximum size entity.
	// The final timeout is limited to the number of maximum sized entities
	// an RPC result can contain, so the timeout won't grow insanely large
	// if you're fetching a ton of small entities.
	MemcacheGetTimeout = 31250 * time.Microsecond // 31.25 milliseconds

	// IgnoreFieldMismatch decides whether *datastore.ErrFieldMismatch errors
	// should be silently ignored. This allows you to easily remove fields from structs.
	IgnoreFieldMismatch = true
)
View Source 
var MemcacheKey = func(k *datastore.Key) string {
	return k.Encode()
}

MemcacheKey returns the string form of the provided datastore key.

Functions

func DefaultKindName 

func DefaultKindName(src interface{}) string

DefaultKindName is the default implementation to determine the Kind an Entity has. Returns the basic Type of the src (no package name included).

func NotFound 

func NotFound(err error, idx int) bool

NotFound returns true if err is an appengine.MultiError and err[idx] is a datastore.ErrNoSuchEntity.

Types

type Goon 

type Goon struct {
	Context context.Context

	// KindNameResolver is used to determine what Kind to give an Entity.
	// Defaults to DefaultKindName
	KindNameResolver KindNameResolver
	// contains filtered or unexported fields
}

Goon holds the app engine context and the request memory cache.

func FromContext 

func FromContext(c context.Context) *Goon

FromContext creates a new Goon object from the given appengine Context. Useful with profiling packages like appstats.

func NewGoon 

func NewGoon(r *http.Request) *Goon

NewGoon creates a new Goon object from the given request.

func (*Goon) Count 

func (g *Goon) Count(q *datastore.Query) (int, error)

Count returns the number of results for the query.

func (*Goon) Delete 

func (g *Goon) Delete(src interface{}) error

Delete deletes the provided entity. Takes either *S or *datastore.Key.

func (*Goon) DeleteMulti 

func (g *Goon) DeleteMulti(src interface{}) error

DeleteMulti is a batch version of Delete. Takes either []*S or []*datastore.Key.

func (*Goon) FlushLocalCache 

func (g *Goon) FlushLocalCache()

FlushLocalCache clears the local memory cache.

func (*Goon) Get 

func (g *Goon) Get(dst interface{}) error

Get loads the entity based on dst's key into dst If there is no such entity for the key, Get returns datastore.ErrNoSuchEntity.

func (*Goon) GetAll 

func (g *Goon) GetAll(q *datastore.Query, dst interface{}) ([]*datastore.Key, error)

GetAll runs the query and returns all the keys that match the query, as well as appending the values to dst, setting the goon key fields of dst, and caching the returned data in local memory.

For "keys-only" queries dst can be nil, however if it is not, then GetAll appends zero value structs to dst, only setting the goon key fields. No data is cached with "keys-only" queries.

See: https://developers.google.com/appengine/docs/go/datastore/reference#Query.GetAll

func (*Goon) GetMulti 

func (g *Goon) GetMulti(dst interface{}) error

GetMulti is a batch version of Get.

dst must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.

func (*Goon) Key 

func (g *Goon) Key(src interface{}) *datastore.Key

Key is the same as KeyError, except nil is returned on error or if the key is incomplete.

func (*Goon) KeyError 

func (g *Goon) KeyError(src interface{}) (*datastore.Key, error)

KeyError returns the key of src based on its properties.

func (*Goon) Kind 

func (g *Goon) Kind(src interface{}) string

Kind returns src's datastore Kind or "" on error.

func (*Goon) Put 

func (g *Goon) Put(src interface{}) (*datastore.Key, error)

Put saves the entity src into the datastore based on src's key k. If k is an incomplete key, the returned key will be a unique key generated by the datastore.

func (*Goon) PutMulti 

func (g *Goon) PutMulti(src interface{}) ([]*datastore.Key, error)

PutMulti is a batch version of Put.

src must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.

func (*Goon) Run 

func (g *Goon) Run(q *datastore.Query) *Iterator

Run runs the query.

func (*Goon) RunInTransaction 

func (g *Goon) RunInTransaction(f func(tg *Goon) error, opts *datastore.TransactionOptions) error

RunInTransaction runs f in a transaction. It calls f with a transaction context tg that f should use for all App Engine operations. Neither cache nor memcache are used or set during a transaction.

Otherwise similar to appengine/datastore.RunInTransaction: https://developers.google.com/appengine/docs/go/datastore/reference#RunInTransaction

type Iterator 

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator is the result of running a query.

func (*Iterator) Cursor 

func (t *Iterator) Cursor() (datastore.Cursor, error)

Cursor returns a cursor for the iterator's current location.

func (*Iterator) Next 

func (t *Iterator) Next(dst interface{}) (*datastore.Key, error)

Next returns the entity of the next result. When there are no more results, datastore.Done is returned as the error.

If the query is not keys only and dst is non-nil, it also loads the entity stored for that key into the struct pointer dst, with the same semantics and possible errors as for the Get function. This result is cached in memory.

If the query is keys only and dst is non-nil, dst will be given the right id.

Refer to appengine/datastore.Iterator.Next: https://developers.google.com/appengine/docs/go/datastore/reference#Iterator.Next

type KindNameResolver 

type KindNameResolver func(src interface{}) string

KindNameResolver takes an Entity and returns what the Kind should be for Datastore.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.