README
¶
goon
An autocaching interface to the app engine datastore for Go. Designed to be similar to the python NDB package.
The different flavors of App Engine
You must choose the goon major version based on which App Engine library you are using.
| App Engine Go library | Include in your project | Correct goon version |
|---|---|---|
google.golang.org/appengine/v2 |
github.com/mjibson/goon/v2 |
goon v2.0.0 |
google.golang.org/appengine |
github.com/mjibson/goon |
goon v1.1.0 |
appengine |
github.com/mjibson/goon |
goon v0.9.0 |
cloud.google.com/go |
N/A | Not supported (issue #74) |
Documentation
Documentation
¶
Overview ¶
Package goon provides an autocaching interface to the app engine datastore similar to the python NDB package.
Goon differs from the datastore package in various ways: it remembers the appengine Context, which need only be specified once at creation time; kinds need not be specified as they are computed, by default, from a type's name; keys are inferred from specially-tagged fields on types, removing the need to pass key objects around.
In general, the difference is that Goon's API is identical to the datastore API, it's just shorter.
Keys in Goon are stored in the structs themselves. Below is an example struct with a field to specify the id (see the Key Specifications section below for full documentation).
type User struct {
Id string `datastore:"-" goon:"id"`
Name string
}
Thus, to get a User with id 2:
userid := 2
g := goon.NewGoon(r)
u := &User{Id: userid}
g.Get(u)
Key Specifications ¶
For both the Key and KeyError functions, src must be a S or *S for some struct type S. The key is extracted based on various fields of S. If a field of type int64 or string has a struct tag named goon with value "id", it is used as the key's id. If a field of type *datastore.Key has a struct tag named goon with value "parent", it is used as the key's parent. If a field of type string has a struct tag named goon with value "kind", it is used as the key's kind. The "kind" field supports an optional second parameter which is the default kind name. If no kind field exists, the struct's name is used. These fields should all have their datastore field marked as "-".
Example, with kind User:
type User struct {
Id string `datastore:"-" goon:"id"`
Read time.Time
}
Example, with kind U if _kind is the empty string:
type User struct {
_kind string `goon:"kind,U"`
Id string `datastore:"-" goon:"id"`
Read time.Time
}
To override kind of a single entity to UserKind:
u := User{_kind: "UserKind"}
An example with both parent and kind:
type UserData struct {
Id string `datastore:"-" goon:"id"`
_kind string `goon:"kind,UD"`
Parent *datastore.Key `datastore:"-" goon:"parent"`
Data []byte
}
Features ¶
Datastore interaction with: Get, GetMulti, Put, PutMulti, Delete, DeleteMulti, Queries.
All key-based operations backed by memory and memcache.
Per-request, in-memory cache: fetch the same key twice, the second request is served from local memory.
Intelligent multi support: running GetMulti correctly fetches from memory, then memcache, then the datastore; each tier only sends keys off to the next one if they were missing.
Memcache control variance: long memcache requests are cancelled.
Transactions use a separate context, but locally cache any results on success.
Automatic kind naming: struct names are inferred by reflection, removing the need to manually specify key kinds.
Simpler API than appengine/datastore.
API comparison between goon and datastore ¶
put with incomplete key
datastore:
type Group struct {
Name string
}
c := appengine.NewContext(r)
g := &Group{Name: "name"}
k := datastore.NewIncompleteKey(c, "Group", nil)
err := datastore.Put(c, k, g)
goon:
type Group struct {
Id int64 `datastore:"-" goon:"id"`
Name string
}
n := goon.NewGoon(r)
g := &Group{Name: "name"}
err := n.Put(g)
get with known key
datastore:
type Group struct {
Name string
}
c := appengine.NewContext(r)
g := &Group{}
k := datastore.NewKey(c, "Group", "", 1, nil)
err := datastore.Get(c, k, g)
goon:
type Group struct {
Id int64 `datastore:"-" goon:"id"`
Name string
}
n := goon.NewGoon(r)
g := &Group{Id: 1}
err := n.Get(g)
Memcache Control Variance ¶
Memcache is generally fast. When it is slow, goon will timeout the memcache requests and proceed to use the datastore directly. The memcache put and get timeout variables determine how long to wait for various kinds of requests. The default settings were determined experimentally and should provide reasonable defaults for most applications.
See: http://talks.golang.org/2013/highperf.slide#23
PropertyLoadSaver support ¶
Structs that implement the PropertyLoadSaver interface are guaranteed to call the Save() method once and only once per Put/PutMulti call and never elsewhere. Similarly the Load() method is guaranteed to be called once and only once per Get/GetMulti/GetAll/Next call and never elsewhere.
Index ¶
- Variables
- func DefaultKindName(src interface{}) string
- func NotFound(err error, idx int) bool
- type Goon
- func (g *Goon) ClearCache(src interface{}, mem, local bool) error
- func (g *Goon) ClearCacheMulti(src interface{}, mem, local bool) error
- func (g *Goon) Count(q *datastore.Query) (int, error)
- func (g *Goon) Delete(src interface{}) error
- func (g *Goon) DeleteMulti(src interface{}) error
- func (g *Goon) FlushLocalCache()
- func (g *Goon) Get(dst interface{}) error
- func (g *Goon) GetAll(q *datastore.Query, dst interface{}) ([]*datastore.Key, error)
- func (g *Goon) GetMulti(dst interface{}) error
- func (g *Goon) Key(src interface{}) *datastore.Key
- func (g *Goon) KeyError(src interface{}) (*datastore.Key, error)
- func (g *Goon) Kind(src interface{}) string
- func (g *Goon) Put(src interface{}) (*datastore.Key, error)
- func (g *Goon) PutMulti(src interface{}) ([]*datastore.Key, error)
- func (g *Goon) Run(q *datastore.Query) *Iterator
- func (g *Goon) RunInTransaction(f func(tg *Goon) error, opts *datastore.TransactionOptions) error
- type Iterator
- type KindNameResolver
Constants ¶
This section is empty.
Variables ¶
var ( // LogErrors issues appengine.Context.Errorf on any error. LogErrors = true // LogTimeoutErrors issues appengine.Context.Warningf on memcache timeout errors. LogTimeoutErrors = false // MemcachePutTimeoutThreshold is the number of bytes after which the large // timeout setting is added to the small timeout setting. This repeats. // Which means that with a 20 KiB threshold and a 100 KiB memcache payload, // the final timeout is MemcachePutTimeoutSmall + 5*MemcachePutTimeoutLarge MemcachePutTimeoutThreshold = 20 * 1024 // 20 KiB // MemcachePutTimeoutSmall is the minimum time to wait during memcache // Put operations before aborting them and using the datastore. MemcachePutTimeoutSmall = 5 * time.Millisecond // MemcachePutTimeoutLarge is the amount of extra time to wait for larger // memcache Put requests. See also MemcachePutTimeoutThreshold. MemcachePutTimeoutLarge = 1 * time.Millisecond // MemcacheGetTimeout is the amount of time to wait for all memcache Get // requests, per key fetched. Because we can't really know how big entities // we are requesting, this setting should be for the maximum size entity. // The final timeout is limited to the number of maximum sized entities // an RPC result can contain, so the timeout won't grow insanely large // if you're fetching a ton of small entities. MemcacheGetTimeout = 31250 * time.Microsecond // 31.25 milliseconds // IgnoreFieldMismatch decides whether *datastore.ErrFieldMismatch errors // should be silently ignored. This allows you to easily remove fields from structs. IgnoreFieldMismatch = true )
var MemcacheKey = func(k *datastore.Key) string {
return k.Encode()
}
MemcacheKey returns the string form of the provided datastore key.
Functions ¶
func DefaultKindName ¶
func DefaultKindName(src interface{}) string
DefaultKindName is the default implementation to determine the Kind an Entity has. Returns the basic Type of the src (no package name included).
Types ¶
type Goon ¶
type Goon struct {
Context context.Context
// KindNameResolver is used to determine what Kind to give an Entity.
// Defaults to DefaultKindName
KindNameResolver KindNameResolver
// contains filtered or unexported fields
}
Goon holds the app engine context and the request memory cache.
func FromContext ¶
FromContext creates a new Goon object from the given appengine Context. Useful with profiling packages like appstats.
func (*Goon) ClearCache ¶
ClearCache removes the provided entity from cache. Takes either *S or *datastore.Key. The 'mem' and 'local' booleans dictate the type of caches to clear.
func (*Goon) ClearCacheMulti ¶
ClearCacheMulti removes the provided entities from cache. Takes either []*S or []*datastore.Key. The 'mem' and 'local' booleans dictate the type of caches to clear.
func (*Goon) DeleteMulti ¶
DeleteMulti is a batch version of Delete. Takes either []*S or []*datastore.Key.
func (*Goon) FlushLocalCache ¶
func (g *Goon) FlushLocalCache()
FlushLocalCache clears the local memory cache.
func (*Goon) Get ¶
Get loads the entity based on dst's key into dst If there is no such entity for the key, Get returns datastore.ErrNoSuchEntity.
func (*Goon) GetAll ¶
GetAll runs the query and returns all the keys that match the query, as well as appending the values to dst, setting the goon key fields of dst, and caching the returned data in local memory.
For "keys-only" queries dst can be nil, however if it is not, then GetAll appends zero value structs to dst, only setting the goon key fields.
No data is cached with projection or "keys-only" queries.
See: https://developers.google.com/appengine/docs/go/datastore/reference#Query.GetAll
func (*Goon) GetMulti ¶
GetMulti is a batch version of Get.
dst must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.
func (*Goon) Key ¶
Key is the same as KeyError, except nil is returned on error or if the key is incomplete.
func (*Goon) Put ¶
Put saves the entity src into the datastore based on src's key k. If k is an incomplete key, the returned key will be a unique key generated by the datastore.
func (*Goon) PutMulti ¶
PutMulti is a batch version of Put.
src must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.
func (*Goon) RunInTransaction ¶
RunInTransaction runs f in a transaction. It calls f with a transaction context tg that f should use for all App Engine operations. Neither cache nor memcache are used or set during a transaction.
Otherwise similar to appengine/datastore.RunInTransaction: https://developers.google.com/appengine/docs/go/datastore/reference#RunInTransaction
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator is the result of running a query.
func (*Iterator) Next ¶
Next returns the entity of the next result. When there are no more results, datastore.Done is returned as the error.
If the query is not keys only and dst is non-nil, it also loads the entity stored for that key into the struct pointer dst, with the same semantics and possible errors as for the Get function.
This result is cached in memory, unless it's a projection query.
If the query is keys only and dst is non-nil, dst will be given the right id.
Refer to appengine/datastore.Iterator.Next: https://developers.google.com/appengine/docs/go/datastore/reference#Iterator.Next
type KindNameResolver ¶
type KindNameResolver func(src interface{}) string
KindNameResolver takes an Entity and returns what the Kind should be for Datastore.
Source Files