godscache

package module
v0.0.0-...-dc0ce86 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 8, 2019 License: MIT Imports: 12 Imported by: 0

README

Go DS Cache

Build Status Coverage Status Go Report Card Awesome Go Gitter chat Freenode chat GoDoc

An unofficial Google Cloud Platform Go Datastore wrapper that adds caching using memcached.

For App Engine Flexible, Compute Engine, Kubernetes Engine, and more.

Documentation is here: https://godoc.org/github.com/defcronyke/godscache

Documentation

Overview

Package godscache is a wrapper around the official Google Cloud Datastore Go client library "cloud.google.com/go/datastore", which adds caching to datastore requests using memcached. Its name is a play on words, and it's actually called Go DS Cache. Note that it is wrapping the newer datastore library, which is intended for use with App Engine Flexible Environment, Compute Engine, or Kubernetes Engine, and is not for use with App Engine Standard.

If you're looking for something similar for App Engine Standard, check out the highly recommended nds library: https://godoc.org/github.com/qedus/nds

For the official Google library documentation, go here: https://godoc.org/cloud.google.com/go/datastore

Godscache follows the Google Cloud Datastore client library API very closely, to act as a drop-in replacement for the official library from Google.

Things which aren't implemented in this library can be used anyway, they just won't be cached. For example, the Client struct holds a Parent member which is the raw Google Datastore client, so you can use that instead to make your requests if you need to use some feature that's not implemented in godscache, or if you want to bypass the cache for some reason.

To use godscache, you will need a Google Cloud project with an initialized Datastore on it, and a memcached instance to connect to. You can connect to as many memcached instances as you want.

Index

Examples

Constants

View Source 
const (
	// MemcacheServerKey is the key to use to add the memcache servers to the context.
	MemcacheServerKey = ctxKeyMemcacheServers("memcachedServers")
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// The raw Datastore client, which can be used directly if you want to bypass caching.
	Parent *datastore.Client

	// The Google Cloud Platform project ID.
	ProjectID string

	// The memcached IP:PORT addresses.
	MemcacheServers []string

	// The memcache client, which you can use directly if you want to access the cache.
	MemcacheClient *memcache.Client
}

Client is the main struct for godscache. It holds a regular datastore client in the Parent field, as well as the memcache client.

func NewClient 

func NewClient(ctx context.Context, projectID string, opts ...option.ClientOption) (*Client, error)

NewClient is a constructor for making a new godscache client. Start here. It makes a datastore client and stores it in the Parent field, and it makes a memcache client. Set the context with the MemcacheServerKey, with a value of []string{"ip_address1:port", "ip_addressN:port"}, to specify which memcache servers to connect to. Alternately you can set the environment variable GODSCACHE_MEMCACHED_SERVERS="ip_address1:port,ip_addressN:port" instead to specify the memcached servers. The context value will take priority over the environment variables if both are present.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Provide the memcached server addresses in the context, or in the
// GODSCACHE_MEMCACHED_SERVERS environment variable.
//
// memcacheServers := []string{"ip_address1:port", "ip_addressN:port"}
// ctx = context.WithValue(ctx, MemcacheServerKey, memcacheServers)

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleNewClient: failed creating new godscache client: %v", err)
	return
}

fmt.Printf("godscache.ExampleNewClient: client instantiated with %v memcache server(s).\n", len(c.MemcacheServers))

Output:

godscache.ExampleNewClient: client instantiated with 1 memcache server(s).

func (*Client) Delete 

func (c *Client) Delete(ctx context.Context, key *datastore.Key) error

Delete data from the datastore and cache.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_Delete: failed creating new godscache client: %v", err)
	return
}

// Create a new incomplete key for a given datastore kind. This key will be complete
// and usable for queries after running Put() below.
key := datastore.IncompleteKey("exampleClient_Delete", nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_Delete",
}

// Put data into the datastore and cache, and save to key the complete key received from
// the datastore.
key, err = c.Put(ctx, key, val)
if err != nil {
	log.Printf("godscache.ExampleClient_Delete: failed putting data into datastore and cache: %v", err)
	return
}

// Delete test data from datastore and cache.
err = c.Delete(ctx, key)
if err != nil {
	log.Printf("godscache.ExampleClient_Delete: failed deleting data from datastore and cache: %v", err)
	return
}

// Create a variable which we will save the value to after Get() returns.
var result TestDbData

// Try to get the deleted data from datastore or cache. This will result in an error.
err = c.Get(ctx, key, &result)
if err != nil {
	fmt.Printf("godscache.ExampleClient_Delete: failed getting result from datastore or cache: %v\n", err)
	return
}

Output:

godscache.ExampleClient_Delete: failed getting result from datastore or cache: datastore: no such entity

func (*Client) DeleteMulti 

func (c *Client) DeleteMulti(ctx context.Context, keys []*datastore.Key) error

DeleteMulti deletes multiple pieces of data from the datastore and cache all at once.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_DeleteMulti: failed creating new godscache client: %v", err)
	return
}

// Set the entity kind to be used for the queries below.
kind := "exampleClient_DeleteMulti"

// Make new slices to hold keys and values.
keys := make([]*datastore.Key, 0, 2)
vals := make([]*TestDbData, 0, 2)

// Create a new incomplete key for a given datastore kind.
key := datastore.IncompleteKey(kind, nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_DeleteMulti 1",
}

// Add key and value to keys and values slices.
keys = append(keys, key)
vals = append(vals, val)

// Create another incomplete key for a given datastore kind.
key = datastore.IncompleteKey(kind, nil)

// Create another piece of test data to put into datastore and cache.
val = &TestDbData{
	TestString: "ExampleClient_DeleteMulti 2",
}

// Add key and value to keys and values slices.
keys = append(keys, key)
vals = append(vals, val)

// Put data into the datastore and cache, and save to keys the complete keys received from
// the datastore.
keys, err = c.PutMulti(ctx, keys, vals)
if err != nil {
	log.Printf("godscache.ExampleClient_DeleteMulti: failed putting data into datastore and cache: %v", err)
	return
}

// Delete test data from datastore and cache.
err = c.DeleteMulti(ctx, keys)
if err != nil {
	log.Printf("godscache.ExampleClient_DeleteMulti: failed deleting data from datastore and cache: %v", err)
	return
}

// Create a variable which we will save the values to after GetMulti() returns.
results := make([]*TestDbData, len(keys))

// Try to get the deleted data from datastore or cache. This will result in an error.
err = c.GetMulti(ctx, keys, results)
if err != nil {
	fmt.Printf("godscache.ExampleClient_DeleteMulti: failed getting results from datastore or cache: %v\n", err)
	return
}

Output:

godscache.ExampleClient_DeleteMulti: failed getting results from datastore or cache: godscache.Client.GetMulti: failed getting multiple values from datastore: datastore: no such entity (and 1 other error)

func (*Client) Get 

func (c *Client) Get(ctx context.Context, key *datastore.Key, dst interface{}) error

Get data from the datastore or cache. The dst value must be a Struct pointer.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_Get: failed creating new godscache client: %v", err)
	return
}

// Create a new incomplete key for a given datastore kind. This key will be complete
// and usable for queries after running Put() below.
key := datastore.IncompleteKey("exampleClient_Get", nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_Get",
}

// Put data into the datastore and cache, and save to key the complete key received from
// the datastore.
key, err = c.Put(ctx, key, val)
if err != nil {
	log.Printf("godscache.ExampleClient_Get: failed putting data into datastore and cache: %v", err)
	return
}

// Create a variable which we will save the value to after Get() returns.
var result TestDbData

// Get the value from datastore or cache and save it in result.
err = c.Get(ctx, key, &result)
if err != nil {
	log.Printf("godscache.ExampleClient_Get: failed getting result from datastore or cache: %v", err)
	return
}

// Delete test data from datastore and cache.
err = c.Delete(ctx, key)
if err != nil {
	log.Printf("godscache.ExampleClient_Get: failed deleting data from datastore and cache: %v", err)
	return
}

fmt.Printf("godscache.ExampleClient_Get: result: %+v\n", result)

Output:

godscache.ExampleClient_Get: result: {TestString:ExampleClient_Get}

func (*Client) GetMulti 

func (c *Client) GetMulti(ctx context.Context, keys []*datastore.Key, dst interface{}) error

GetMulti is for getting multiple values from the datastore or cache. The dst value must be a slice of structs or struct pointers, and not a datastore.PropertyList. It must also be the same length as the keys slice.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_GetMulti: failed creating new godscache client: %v", err)
	return
}

// Allocate a slice to hold two datastore keys.
keys := make([]*datastore.Key, 0, 2)

// Create a new incomplete key for a given datastore kind. This key will be complete
// and usable for queries after running Put() below.
key := datastore.IncompleteKey("exampleClient_GetMulti", nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_GetMulti 1",
}

// Put data into the datastore and cache, and save to key the complete key received from
// the datastore.
key, err = c.Put(ctx, key, val)
if err != nil {
	log.Printf("godscache.ExampleClient_GetMulti: failed putting data into datastore and cache: %v", err)
	return
}

// Add complete key to the keys slice, for use with GetMulti() below.
keys = append(keys, key)

// Create a second incomplete key for a given datastore kind. This key will be complete
// and usable for queries after running Put() below.
key = datastore.IncompleteKey("exampleClient_GetMulti", nil)

// Create test data to put into datastore and cache.
val = &TestDbData{
	TestString: "ExampleClient_GetMulti 2",
}

// Put data into the datastore and cache, and save to key the complete key received from
// the datastore.
key, err = c.Put(ctx, key, val)
if err != nil {
	log.Printf("godscache.ExampleClient_GetMulti: failed putting data into datastore and cache: %v", err)
	return
}

// Add second complete key to the keys slice, for use with GetMulti() below.
keys = append(keys, key)

// Create a variable which we will save the results to after GetMulti() returns.
// The results slice needs to be the same length as the keys slice.
results := make([]*TestDbData, len(keys))

// Get the value from datastore or cache and save it in results.
err = c.GetMulti(ctx, keys, results)
if err != nil {
	log.Printf("godscache.ExampleClient_GetMulti: failed getting results from datastore or cache: %v", err)
	return
}

// Go through all the keys, and delete them from datastore and cache.
for _, key := range keys {
	// Delete test data from datastore and cache.
	err = c.Delete(ctx, key)
	if err != nil {
		log.Printf("godscache.ExampleClient_GetMulti: failed deleting data from datastore and cache: %v", err)
		return
	}
}

fmt.Printf("godscache.ExampleClient_GetMulti: result1: %+v\n", results[0])

Output:

godscache.ExampleClient_GetMulti: result1: &{TestString:ExampleClient_GetMulti 1}

func (*Client) Put 

func (c *Client) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error)

Put data into the datastore and into the cache. The src value must be a Struct pointer.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_Put: failed creating new godscache client: %v", err)
	return
}

// Create a new incomplete key for a given datastore kind. This key will be complete
// and usable for queries after running Put() below.
key := datastore.IncompleteKey("exampleClient_Put", nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_Put",
}

// Put data into the datastore and cache, and save to key the complete key received from
// the datastore.
key, err = c.Put(ctx, key, val)
if err != nil {
	log.Printf("godscache.ExampleClient_Put: failed putting data into datastore and cache: %v", err)
	return
}

// Delete test data from datastore and cache.
err = c.Delete(ctx, key)
if err != nil {
	log.Printf("godscache.ExampleClient_Put: failed deleting data from datastore and cache: %v", err)
	return
}

fmt.Printf("godscache.ExampleClient_Put: is the datastore key incomplete?: %v\n", key.Incomplete())

Output:

godscache.ExampleClient_Put: is the datastore key incomplete?: false

func (*Client) PutMulti 

func (c *Client) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error)

PutMulti adds multiple pieces of data to the datastore and cache all at once. It returns a slice of complete keys.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_PutMulti: failed creating new godscache client: %v", err)
	return
}

// Set the kind to use for the queries.
kind := "exampleClient_PutMulti"

// Initialize slices for keys and values, to be used with PutMulti() below.
keys := make([]*datastore.Key, 0, 2)
vals := make([]*TestDbData, 0, 2)

// Create a new incomplete key for a given datastore kind.
key := datastore.IncompleteKey(kind, nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_PutMulti 1",
}

// Add key and data to keys and vals slices.
keys = append(keys, key)
vals = append(vals, val)

// Create another incomplete key for a given datastore kind.
key = datastore.IncompleteKey(kind, nil)

// Create test data to put into datastore and cache.
val = &TestDbData{
	TestString: "ExampleClient_PutMulti 2",
}

// Add key and data to keys and vals slices.
keys = append(keys, key)
vals = append(vals, val)

// Put data into the datastore and cache, and save to keys the complete keys received from
// the datastore.
keys, err = c.PutMulti(ctx, keys, vals)
if err != nil {
	log.Printf("godscache.ExampleClient_PutMulti: failed putting data into datastore and cache: %v", err)
	return
}

// Delete test data from datastore and cache.
err = c.DeleteMulti(ctx, keys)
if err != nil {
	log.Printf("godscache.ExampleClient_PutMulti: failed deleting data from datastore and cache: %v", err)
	return
}

fmt.Printf("godscache.ExampleClient_PutMulti: is the first datastore key incomplete?: %v\n", keys[0].Incomplete())

Output:

godscache.ExampleClient_PutMulti: is the first datastore key incomplete?: false

func (*Client) Run 

func (c *Client) Run(ctx context.Context, q *datastore.Query) *datastore.Iterator

Run a datastore query. To utilize this with caching, you should perform a KeysOnly() query, and then use Get() on the keys.

Example 
// Make a new context for running the queries.
ctx := context.Background()

// Instantiate a new godscache client. You could also just supply the project ID string
// directly here instead of calling os.Getenv("GODSCACHE_PROJECT_ID").
c, err := NewClient(ctx, os.Getenv("GODSCACHE_PROJECT_ID"))
if err != nil {
	log.Printf("godscache.ExampleClient_Run: failed creating new godscache client: %v", err)
	return
}

// Create a new incomplete key for a given datastore kind. This key will be complete
// and usable for queries after running Put() below.
key := datastore.IncompleteKey("exampleClient_Run", nil)

// Create test data to put into datastore and cache.
val := &TestDbData{
	TestString: "ExampleClient_Run",
}

// Put data into the datastore and cache, and save to key the complete key received from
// the datastore.
_, err = c.Put(ctx, key, val)
if err != nil {
	log.Printf("godscache.ExampleClient_Run: failed putting data into datastore and cache: %v", err)
	return
}

// Make a new KeysOnly query, so we can look up the data with Get() so that it will
// get cached.
q := datastore.NewQuery("exampleClient_Run").KeysOnly()

// Run the query and iterate over the results, passing nil to Next() because we only want
// the keys.
for t := c.Run(ctx, q); ; {
	// Get the next key.
	key, err := t.Next(nil)
	if err == iterator.Done {
		break
	}
	if err != nil {
		log.Printf("godscache.ExampleClient_Run: failed getting query result: %v", err)
		return
	}

	// Create a variable to hold the current result.
	var result TestDbData

	// Use the key to look up the result in a way which can be cached.
	err = c.Get(ctx, key, &result)
	if err != nil {
		log.Printf("godscache.ExampleClient_Run: failed getting data from datastore or cache: %v", err)
		return
	}

	fmt.Printf("godscache.ExampleClient_Run: result: %+v\n", result)

	// Delete test data from datastore and cache.
	err = c.Delete(ctx, key)
	if err != nil {
		log.Printf("godscache.ExampleClient_Run: failed deleting data from datastore and cache: %v", err)
		return
	}

	// For this example, we only care about the first result.
	break
}

Output:

godscache.ExampleClient_Run: result: {TestString:ExampleClient_Run}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.