zoom

README

Zoom

A blazing-fast datastore and querying engine for Go built on Redis.

Requires Redis version >= 2.8.9 and Go version >= 1.2. The latest version of both is recommended.

Full documentation is available on godoc.org.

Table of Contents

• Development Status
• When is Zoom a Good Fit?
• Installation
• Initialization
• Models
• Transactions
• Queries
• More Information
• Testing & Benchmarking
• Contributing
• Example Usage
• License

Development Status

Zoom has been around for more than a year. It is well-tested and going forward the API will be relatively stable. We are closing in on Version 1.0.0-alpha.

At this time, Zoom can be considered safe for use in low-traffic production applications. However, as with any relatively new package, it is possible that there are some undiscovered bugs. Therefore we would recommend writing good tests, reporting any bugs you may find, and avoiding using Zoom for mission-critical or high-traffic applications.

Zoom follows semantic versioning, but offers no guarantees of backwards compatibility until version 1.0. We recommend using a dependency manager such as godep or glide to lock in a specific version of Zoom. You can also keep an eye on the Releases page to see a full changelog for each release. In addition, starting with version 0.9.0, migration guides will be provided for any non-trivial breaking changes, making it easier to stay up to date with the latest version.

When is Zoom a Good Fit?

Zoom might be a good fit if:

1. You are building a low-latency application. Because Zoom is built on top of Redis and all data is stored in memory, it is typically much faster than datastores/ORMs based on traditional SQL databases. Latency will be the most noticeable difference, although throughput may also be improved.
2. You want more out of Redis. Zoom offers a number of features that you don't get by using a Redis driver directly. For example, Zoom supports a larger number of types out of the box (including custom types, slices, maps, complex types, and embedded structs), provides tools for making multi-command transactions easier, and of course, provides the ability to run queries.
3. You want an easy-to-use datastore. Zoom has a simple API and is arguably easier to use than some ORMs. For example, it doesn't require database migrations and instead builds up a schema based on your struct types. Zoom also does not typically require any knowledge of Redis in order to use effectively. Just connect it to a database and you're good to go!

Zoom might not be a good fit if:

1. You are working with a lot of data. Zoom stores all data in memory at all times, and does not yet support sharding or Redis Cluster. Memory could be a hard constraint for larger applications. Keep in mind that it is possible (if expensive) to run Redis on machines with up to 256GB of memory on cloud providers such as Amazon EC2.
2. You require the ability to run advanced queries. Zoom currently only provides support for basic queries and is not as powerful or flexible as something like SQL. For example, Zoom currently lacks the equivalent of the IN or OR SQL keywords. See the documentation for a full list of the types of queries supported.

Installation

Zoom is powered by Redis and needs to connect to a Redis database. You can install Redis on the same machine that Zoom runs on, connect to a remote database, or even use a Redis-as-a-service provider such as Redis To Go, RedisLabs, Google Cloud Redis, or Amazon Elasticache.

If you need to install Redis, see the installation instructions on the official Redis website.

To install Zoom itself, run go get github.com/albrow/zoom to pull down the current master branch, or install with the dependency manager of your choice to lock in a specific version.

Initialization

First, add github.com/albrow/zoom to your import statement:

import (
	 // ...
	 github.com/albrow/zoom
)

Then, you must create a new pool with zoom.NewPool. A pool represents a pool of connections to the database. Since you may need access to the pool in different parts of your application, it is sometimes a good idea to declare a top-level variable and then initialize it in the main or init function. You must also call pool.Close when your application exits, so it's a good idea to use defer.

var pool *zoom.Pool

func main() {
	pool = zoom.NewPool(nil)
	defer func() {
		if err := pool.Close(); err != nil {
			// handle error
		}
	}()
	// ...
}

The NewPool function takes a zoom.PoolOptions as an argument. Here's a list of options and their defaults:

type PoolOptions struct {
	// Address to connect to. Default: "localhost:6379"
	Address string
	// Network to use. Default: "tcp"
	Network string
	// Database id to use (using SELECT). Default: 0
	Database int
	// Password for a password-protected redis database. If not empty,
	// every connection will use the AUTH command during initialization
	// to authenticate with the database. Default: ""
	Password string
}

If you pass in nil to NewPool, Zoom will use all the default values. Any fields in the PoolOptions struct that are empty (e.g., an empty string or 0) will fall back to their default values, so you only need to provide a PoolOptions struct with the fields you want to change.

Models

What is a Model?

Models in Zoom are just structs which implement the zoom.Model interface:

type Model interface {
  ModelId() string
  SetModelId(string)
}

To clarify, all you have to do to implement the Model interface is add a getter and setter for a unique id property.

If you want, you can embed zoom.RandomId to give your model all the required methods. A struct with zoom.RandomId embedded will genrate a pseudo-random id for itself the first time the ModelId method is called iff it does not already have an id. The pseudo-randomly generated id consists of the current UTC unix time with second precision, an incremented atomic counter, a unique machine identifier, and an additional random string of characters. With ids generated this way collisions are extremely unlikely.

Future versions of Zoom may provide additional id implementations out of the box, e.g. one that assigns auto-incremented ids. You are also free to write your own id implementation as long as it satisfies the interface.

A struct definition serves as a sort of schema for your model. Here's an example of a model for a person:

type Person struct {
	 Name string
	 Age  int
	 zoom.RandomId
}

Because of the way Zoom uses reflection, all the fields you want to save need to be public. Almost any type of field is supported, including custom types, slices, maps, complex types, and embedded structs. The only things that are not supported are recursive data structures and functions.

Customizing Field Names

You can change the name used to store the field in Redis with the redis:"<name>" struct tag. So for example, if you wanted the fields to be stored as lowercase fields in redis, you could use the following struct definition:

type Person struct {
	 Name string    `redis:"name"`
	 Age  int       `redis:"age"`
	 zoom.RandomId
}

If you don't want a field to be saved in Redis at all, you can use the special struct tag redis:"-".

Creating Collections

You must create a Collection for each type of model you want to save. A Collection is simply a set of all models of a specific type and has methods for saving, finding, deleting, and querying those models. NewCollection examines the type of a model and uses reflection to build up an internal schema. You only need to call NewCollection once per type. Each pool keeps track of its own collections, so if you wish to share a model type between two or more pools, you will need to create a collection for each pool.

// Create a new collection for the Person type.
People, err := pool.NewCollection(&Person{}, nil)
if err != nil {
	 // handle error
}

The second argument to NewCollection is a CollectionOptions. It works similarly to PoolOptions. You can just pass nil to use all the default options. Additionally, any zero-valued fields in the struct indicate that the default value should be used for that field.

type CollectionOptions struct {
	// FallbackMarshalerUnmarshaler is used to marshal/unmarshal any type
	// into a slice of bytes which is suitable for storing in the database. If
	// Zoom does not know how to directly encode a certain type into bytes, it
	// will use the FallbackMarshalerUnmarshaler. By default, the value is
	// GobMarshalerUnmarshaler which uses the builtin gob package. Zoom also
	// provides JSONMarshalerUnmarshaler to support json encoding out of the box.
	// Default: GobMarshalerUnmarshaler.
	FallbackMarshalerUnmarshaler MarshalerUnmarshaler
	// Iff Index is true, any model in the collection that is saved will be added
	// to a set in redis which acts as an index. The default value is false. The
	// key for the set is exposed via the IndexKey method. Queries and the
	// FindAll, Count, and DeleteAll methods will not work for unindexed
	// collections. This may change in future versions. Default: false.
	Index bool
	// Name is a unique string identifier to use for the collection in redis. All
	// models in this collection that are saved in the database will use the
	// collection name as a prefix. If not provided, the default name will be the
	// name of the model type without the package prefix or pointer declarations.
	// So for example, the default name corresponding to *models.User would be
	// "User". If a custom name is provided, it cannot contain a colon.
	// Default: The name of the model type, excluding package prefix and pointer
	// declarations.
	Name string
}

There are a few important points to emphasize concerning collections:

1. The collection name cannot contain a colon.
2. Queries, as well as the FindAll, DeleteAll, and Count methods will not work if Index is false. This may change in future versions.

Convention is to name the Collection the plural of the corresponding model type (e.g. "People"), but it's just a variable so you can name it whatever you want. If you need to access a Collection in different parts of your application, it is sometimes a good idea to declare a top-level variable and then initialize it in the init function:

var (
	People *zoom.Collection
)

func init() {
	var err error
	// Assuming pool and Person are already defined.
	People, err = pool.NewCollection(&Person{}, nil)
	if err != nil {
		// handle error
	}
}

Saving Models

Continuing from the previous example, to persistently save a Person model to the database, we use the People.Save method. Recall that in this example, "People" is just the name we gave to the Collection which corresponds to the model type Person.

p := &Person{Name: "Alice", Age: 27}
if err := People.Save(p); err != nil {
	 // handle error
}

When you call Save, Zoom converts all the fields of the model into a format suitable for Redis and stores them as a Redis hash. There is a wiki page describing how zoom works under the hood in more detail.

Updating Models

Sometimes, it is preferable to only update certain fields of the model instead of saving them all again. It is more efficient and in some scenarios can allow safer simultaneous changes to the same model (as long as no two clients update the same field at the same time). In such cases, you can use UpdateFields.

if err := People.UpdateFields([]string{"Name"}, person); err != nil {
	// handle error
}

UpdateFields uses "last write wins" semantics, so if another caller updates the same field, your changes may be overwritten. That means it is not safe for "read before write" updates. See the section on Concurrent Updates for more information.

Finding a Single Model

To retrieve a model by id, use the Find method:

p := &Person{}
if err := People.Find("a_valid_person_id", p); err != nil {
	 // handle error
}

The second argument to Find must be a pointer to a struct which satisfies Model, and must have a type corresponding to the Collection. In this case, we passed in Person since that is the struct type that corresponds to our People collection. Find will mutate p by setting all its fields. Using Find in this way allows the caller to maintain type safety and avoid type casting. If Zoom couldn't find a model of type Person with the given id, it will return a ModelNotFoundError.

Finding Only Certain Fields

If you only want to find certain fields in the model instead of retrieving all of them, you can use FindFields, which works similarly to UpdateFields.

p := &Person{}
if err := People.FindFields("a_valid_person_id", []string{"Name"}, p); err != nil {
	// handle error
}
fmt.Println(p.Name, p.Age)
// Output:
// Alice 0

Fields that are not included in the given field names will not be mutated. In the above example, p.Age is 0 because p was just initialized and that's the zero value for the int type.

Finding All Models

To find all models of a given type, use the FindAll method:

people := []*Person{}
if err := People.FindAll(&people); err != nil {
	 // handle error
}

FindAll expects a pointer to a slice of some registered type that implements Model. It grows or shrinks the slice as needed, filling in all the fields of the elements inside of the slice. So the result of the call is that people will be a slice of all models in the People collection.

FindAll only works on indexed collections. To index a collection, you need to include Index: true in the CollectionOptions.

Deleting Models

To delete a model, use the Delete method:

// ok will be true iff a model with the given id existed and was deleted
if ok, err := People.Delete("a_valid_person_id"); err != nil {
	// handle err
}

Delete expects a valid id as an argument, and will attempt to delete the model with the given id. If there was no model with the given type and id, the first return value will be false.

You can also delete all models in a collection with the DeleteAll method:

numDeleted, err := People.DeleteAll()
if err != nil {
  // handle error
}

DeleteAll will return the number of models that were successfully deleted. DeleteAll only works on indexed collections. To index a collection, you need to include Index: true in the CollectionOptions.

Counting the Number of Models

You can get the number of models in a collection using the Count method:

count, err := People.Count()
if err != nil {
  // handle err
}

Count only works on indexed collections. To index a collection, you need to include Index: true in the CollectionOptions.

Transactions

Zoom exposes a transaction API which you can use to run multiple commands efficiently and atomically. Under the hood, Zoom uses a single Redis transaction to perform all the commands in a single round trip. Transactions feature delayed execution, so nothing touches the database until you call Exec. A transaction also remembers its errors to make error handling easier on the caller. The first error that occurs (if any) will be returned when you call Exec.

Here's an example of how to save two models and get the new number of models in the People collection in a single transaction.

numPeople := 0
t := pool.NewTransaction()
t.Save(People, &Person{Name: "Foo"})
t.Save(People, &Person{Name: "Bar"})
// Count expects a pointer to an integer, which it will change the value of
// when the transaction is executed. If you don't care about the number of
// models deleted, you can pass in nil.
t.Count(People, &numPeople)
if err := t.Exec(); err != nil {
  // handle error
}
// numPeople will now equal the number of *Person models in the database
fmt.Println(numPeople)
// Output:
// 2

You can also execute custom Redis commands or run lua scripts with the Command and Script methods. Both methods expect a ReplyHandler as an argument. A ReplyHandler is simply a function that will do something with the reply from Redis corresponding to the script or command that was run. ReplyHandler's are executed in order when you call Exec.

Queries

The Query Object

Zoom provides a useful abstraction for querying the database. You create queries by using the NewQuery constructor, where you must pass in the name corresponding to the type of model you want to query. For now, Zoom only supports queries on a single collection at a time.

You can add one or more query modifiers to the query, such as Order, Limit, and Filter. These methods return the query itself, so you can chain them together. The first error (if any) that occurs due to invalid arguments in the query modifiers will be remembered and returned when you attempt to run the query.

Finally, you run the query using a query finisher method, such as Run or Count. Queries feature delayed execution, so nothing touches the database until you execute the query with a finisher method.

Using Query Modifiers

You can chain a query object together with one or more different modifiers. Here's a list of all the available modifiers:

• Order
• Limit
• Offset
• Include
• Exclude
• Filter

You can run a query with one of the following query finishers:

• Run
• Ids
• Count
• RunOne

Here's an example of a more complicated query using several modifiers:

people := []*Person{}
q := People.NewQuery().Order("-Name").Filter("Age >=", 25).Limit(10)
if err := q.Run(&people); err != nil {
	// handle error
}

Full documentation on the different modifiers and finishers is available on godoc.org.

A Note About String Indexes

Because Redis does not allow you to use strings as scores for sorted sets, Zoom relies on a workaround to store string indexes. It uses a sorted set where all the scores are 0 and each member has the following format: value\x00id, where \x00 is the NULL character. With the string indexes stored this way, Zoom can issue the ZRANGEBYLEX command and related commands to filter models by their string values. As a consequence, here are some caveats to keep in mind:

• Strings are sorted by ASCII value, exactly as they appear in an ASCII table, not alphabetically. This can have surprising effects, for example 'Z' is considered less than 'a'.
• Indexed string values may not contain the NULL or DEL characters (the characters with ASCII codepoints of 0 and 127 respectively). Zoom uses NULL as a separator and DEL as a suffix for range queries.

More Information

Persistence

Zoom is as persistent as the underlying Redis database. If you intend to use Redis as a permanent datastore, it is recommended that you turn on both AOF and RDB persistence options and set fsync to everysec. This will give you good performance while making data loss highly unlikely.

If you want greater protections against data loss, you can set fsync to always. This will hinder performance but give you persistence guarantees very similar to SQL databases such as PostgreSQL.

Read more about Redis persistence

Atomicity

All methods and functions in Zoom that touch the database do so atomically. This is accomplished using Redis transactions and lua scripts when necessary. What this means is that Zoom will not put Redis into an inconsistent state (e.g. where indexes to not match the rest of the data).

However, it should be noted that there is a caveat with Redis atomicity guarantees. If Redis crashes in the middle of a transaction or script execution, it is possible that your AOF file can become corrupted. If this happens, Redis will refuse to start until the AOF file is fixed. It is relatively easy to fix the problem with the redis-check-aof tool, which will remove the partial transaction from the AOF file.

If you intend to issue custom Redis commands or run custom scripts, it is highly recommended that you also make everything atomic. If you do not, Zoom can no longer guarantee that its indexes are consistent. For example, if you change the value of a field which is indexed, you should also update the index for that field in the same transaction. The keys that Zoom uses for indexes and models are provided via the ModelKey, AllIndexKey, and FieldIndexKey methods.

Read more about:

• Redis persistence
• Redis scripts
• Redis transactions

Concurrent Updates

Currently, Zoom does not support concurrent "read before write" updates on models. The UpdateFields method introduced in version 0.12 offers some additional safety for concurrent updates, as long as no concurrent callers update the same fields (or if you are okay with updates overwriting previous changes). However, cases where you need to do a "read before write" update are still not safe by default. For example, consider the following code:

func likePost(postId string) error {
  // Find the Post with the given postId
  post := &Post{}
  if err := Posts.Find(postId); err != nil {
	 return err
  }
  // Increment the number of likes
  post.Likes += 1
  // Save the post
  if err := Posts.Save(post); err != nil {
	 return err
  }
}

The line post.Likes += 1 is a "read before write" operation. That's because the += operator implicitly reads the current value of post.Likes and then adds to it.

This can cause a bug if the function is called across multiple threads or multiple machines concurrently, because the Post model can change in between the time we retrieved it from the database with Find and saved it again with Save. Future versions of Zoom may provide optimistic locking or other means to avoid these kinds of errors. In the meantime, you could fix this code by using an HINCRBY command directly like so:

func likePost(postId string) error {
  // modelKey is the key of the main hash for the model, which
  // stores the struct fields as hash fields in Redis.
  modelKey, err := Posts.ModelKey(postId)
  if err != nil {
	 return err
  }
  conn := zoom.NewConn()
  defer conn.Close()
  if _, err := conn.Do("HINCRBY", modelKey, 1); err != nil {
	 return err
  }
}

You could also use a lua script, which have full transactional support in Zoom, for more complicated "read before write" updates.

Testing & Benchmarking

Running the Tests:

To run the tests, make sure you're in the root directory for Zoom and run:

go test

If everything passes, you should see something like:

ok    github.com/albrow/zoom  2.267s

If any of the tests fail, please open an issue and describe what happened.

By default, tests and benchmarks will run on localhost:6379 and use database #9. You can change the address, network, and database used with flags. So to run on a unix socket at /tmp/redis.sock and use database #3, you could use:

go test -network=unix -address=/tmp/redis.sock -database=3

Running the Benchmarks:

To run the benchmarks, make sure you're in the root directory for the project and run:

go test -run=none -bench .

The -run=none flag is optional, and just tells the test runner to skip the tests and run only the benchmarks (because no test function matches the pattern "none"). You can also use the same flags as above to change the network, address, and database used.

You should see some runtimes for various operations. If you see an error or if the build fails, please open an issue.

Here are the results from my laptop (2.3GHz quad-core i7, 8GB RAM) using a socket connection with Redis set to append-only mode:

BenchmarkConnection  2000000         656 ns/op
BenchmarkPing          50000       26627 ns/op
BenchmarkSet           50000       36452 ns/op
BenchmarkGet           50000       27864 ns/op
BenchmarkSave          20000       58962 ns/op
BenchmarkSave100        2000      960483 ns/op
BenchmarkFind          30000       43054 ns/op
BenchmarkFind100        3000      562743 ns/op
BenchmarkFindAll100     2000      665035 ns/op
BenchmarkFindAll10000     20    68657190 ns/op
BenchmarkDelete        20000       61379 ns/op
BenchmarkDelete100      2000     1031886 ns/op
BenchmarkDeleteAll100   2000      968367 ns/op
BenchmarkDeleteAll1000   100    11857145 ns/op
BenchmarkCount100      50000       28340 ns/op
BenchmarkCount10000    50000       29746 ns/op
BenchmarkQueryFilterInt1From1         10000      149719 ns/op
BenchmarkQueryFilterInt1From10        10000      148245 ns/op
BenchmarkQueryFilterInt10From100       5000      264959 ns/op
BenchmarkQueryFilterInt100From1000     1000     1654756 ns/op
BenchmarkQueryFilterString1From1      10000      152185 ns/op
BenchmarkQueryFilterString1From10     10000      154507 ns/op
BenchmarkQueryFilterString10From100    5000      287958 ns/op
BenchmarkQueryFilterString100From1000  1000     1862549 ns/op
BenchmarkQueryFilterBool1From1        10000      146349 ns/op
BenchmarkQueryFilterBool1From10       10000      147950 ns/op
BenchmarkQueryFilterBool10From100      5000      276740 ns/op
BenchmarkQueryFilterBool100From1000    1000     1641239 ns/op
BenchmarkQueryOrderInt100       2000      681141 ns/op
BenchmarkQueryOrderInt10000       20    72602768 ns/op
BenchmarkQueryOrderString100    1000     1662290 ns/op
BenchmarkQueryOrderString10000    10   118660109 ns/op
BenchmarkQueryOrderBool100      2000      681808 ns/op
BenchmarkQueryOrderBool10000      20    71249344 ns/op
BenchmarkComplexQuery          10000      142476 ns/op

The results of these benchmarks can vary widely from system to system, and so the benchmarks here are really only useful for comparing across versions of Zoom, and for identifying possible performance regressions or improvements during development. You should run your own benchmarks that are closer to your use case to get a real sense of how Zoom will perform for you. High performance is one of the top priorities for this project, because without that you are better off using an ORM designed for SQL databases.

Contributing

See CONTRIBUTING.md.

Example Usage

There is an example json/rest application which uses the latest version of Zoom. It is a simple example that doesn't use all of Zoom's features, but should be good enough for understanding how zoom can work in a real application.

License

Zoom is licensed under the MIT License. See the LICENSE file for more information.

Documentation

Overview

Package zoom is a blazing-fast datastore and querying engine for Go built on Redis. It supports models of any arbitrary struct type and provides basic querying functionality. It also supports atomic transactions, lua scripts, and running Redis commands directly if needed.

Version 0.13.1

For installation instructions, examples, and more information visit https://github.com/albrow/zoom.

Index

• func Interfaces(in interface{}) []interface{}
• type Action
• type ActionKind
• type Collection
  • func (c *Collection) Count() (int, error)
  • func (c *Collection) Delete(id string) (bool, error)
  • func (c *Collection) DeleteAll() (int, error)
  • func (c *Collection) FieldIndexKey(fieldName string) (string, error)
  • func (c *Collection) Find(id string, model Model) error
  • func (c *Collection) FindAll(models interface{}) error
  • func (c *Collection) FindFields(id string, fieldNames []string, model Model) error
  • func (c *Collection) IndexKey() string
  • func (c *Collection) ModelKey(id string) (string, error)
  • func (c *Collection) Name() string
  • func (collection *Collection) NewQuery() *Query
  • func (c *Collection) Save(model Model) error
  • func (c *Collection) UpdateFields(fieldNames []string, model Model) error
• type CollectionOptions
• type MarshalerUnmarshaler
• type Model
  • func Models(in interface{}) []Model
• type ModelNotFoundError
  • func (e ModelNotFoundError) Error() string
• type Pool
  • func NewPool(options *PoolOptions) *Pool
  • func (p *Pool) Close() error
  • func (p *Pool) NewCollection(model Model, options *CollectionOptions) (*Collection, error)
  • func (p *Pool) NewConn() redis.Conn
  • func (p *Pool) NewTransaction() *Transaction
• type PoolOptions
• type Query
  • func (q *Query) Count() (uint, error)
  • func (q *Query) Exclude(fields ...string) *Query
  • func (q *Query) Filter(filterString string, value interface{}) *Query
  • func (q *Query) Ids() ([]string, error)
  • func (q *Query) Include(fields ...string) *Query
  • func (q *Query) Limit(amount uint) *Query
  • func (q *Query) Offset(amount uint) *Query
  • func (q *Query) Order(fieldName string) *Query
  • func (q *Query) Run(models interface{}) error
  • func (q *Query) RunOne(model Model) error
  • func (q *Query) String() string
• type RandomId
  • func (r *RandomId) ModelId() string
  • func (r *RandomId) SetModelId(id string)
• type ReplyHandler
• type Transaction
  • func (t *Transaction) Command(name string, args redis.Args, handler ReplyHandler)
  • func (t *Transaction) Count(c *Collection, count *int)
  • func (t *Transaction) Delete(c *Collection, id string, deleted *bool)
  • func (t *Transaction) DeleteAll(c *Collection, count *int)
  • func (t *Transaction) Exec() error
  • func (t *Transaction) Find(c *Collection, id string, model Model)
  • func (t *Transaction) FindAll(c *Collection, models interface{})
  • func (t *Transaction) FindFields(c *Collection, id string, fieldNames []string, model Model)
  • func (t *Transaction) Save(c *Collection, model Model)
  • func (t *Transaction) Script(script *redis.Script, args redis.Args, handler ReplyHandler)
  • func (t *Transaction) UpdateFields(c *Collection, fieldNames []string, model Model)

Functions

func Interfaces

func Interfaces(in interface{}) []interface{}

Interfaces converts in to []interface{}. It will panic if the underlying type of in is not a slice.

Types

type Action

type Action struct {
	// contains filtered or unexported fields
}

Action is a single step in a transaction and must be either a command or a script with optional arguments and a reply handler.

type ActionKind

type ActionKind int

ActionKind is either a command or a script

const (
	CommandAction ActionKind = iota
	ScriptAction
)

type Collection

type Collection struct {
	// contains filtered or unexported fields
}

Collection represents a specific registered type of model. It has methods for saving, finding, and deleting models of a specific type. Use the NewCollection method to create a new collection.

func (*Collection) Count

func (c *Collection) Count() (int, error)

Count returns the number of models of the given type that exist in the database. It returns an error if there was a problem connecting to the database.

func (*Collection) Delete

func (c *Collection) Delete(id string) (bool, error)

Delete removes the model with the given type and id from the database. It will not return an error if the model corresponding to the given id was not found in the database. Instead, it will return a boolean representing whether or not the model was found and deleted, and will only return an error if there was a problem connecting to the database.

func (*Collection) DeleteAll

func (c *Collection) DeleteAll() (int, error)

DeleteAll deletes all the models of the given type in a single transaction. See http://redis.io/topics/transactions. It returns the number of models deleted and an error if there was a problem connecting to the database.

func (*Collection) FieldIndexKey

func (c *Collection) FieldIndexKey(fieldName string) (string, error)

FieldIndexKey returns the key for the sorted set used to index the field identified by fieldName. It returns an error if fieldName does not identify a field in the spec or if the field it identifies is not an indexed field.

func (*Collection) Find

func (c *Collection) Find(id string, model Model) error

Find retrieves a model with the given id from redis and scans its values into model. model should be a pointer to a struct of a registered type corresponding to the Collection. Find will mutate the struct, filling in its fields and overwriting any previous values. It returns an error if a model with the given id does not exist, if the given model was the wrong type, or if there was a problem connecting to the database.

func (*Collection) FindAll

func (c *Collection) FindAll(models interface{}) error

FindAll finds all the models of the given type. It executes the commands needed to retrieve the models in a single transaction. See http://redis.io/topics/transactions. models must be a pointer to a slice of models with a type corresponding to the Collection. FindAll will grow or shrink the models slice as needed and if any of the models in the models slice are nil, FindAll will use reflection to allocate memory for them. FindAll returns an error if models is the wrong type or if there was a problem connecting to the database.

func (*Collection) FindFields

func (c *Collection) FindFields(id string, fieldNames []string, model Model) error

FindFields is like Find but finds and sets only the specified fields. Any fields of the model which are not in the given fieldNames are not mutated. FindFields will return an error if any of the given fieldNames are not found in the model type.

func (*Collection) IndexKey

func (c *Collection) IndexKey() string

IndexKey returns the key that identifies a set in the database that stores all the ids for models in the given collection.

func (*Collection) ModelKey

func (c *Collection) ModelKey(id string) (string, error)

ModelKey returns the key that identifies a hash in the database which contains all the fields of the model corresponding to the given id. It returns an error iff id is empty.

func (*Collection) Name

func (c *Collection) Name() string

Name returns the name for the given collection. The name is a unique string identifier to use for the collection in redis. All models in this collection that are saved in the database will use the collection name as a prefix.

func (*Collection) NewQuery

func (collection *Collection) NewQuery() *Query

NewQuery is used to construct a query. The query returned can be chained together with one or more query modifiers (e.g. Filter or Order), and then executed using the Run, RunOne, Count, or Ids methods. If no query modifiers are used, running the query will return all models of the given type in uspecified order. Queries use delated execution, so nothing touches the database until you execute it.

func (*Collection) Save

func (c *Collection) Save(model Model) error

Save writes a model (a struct which satisfies the Model interface) to the redis database. Save returns an error if the type of model does not match the registered Collection. To make a struct satisfy the Model interface, you can embed zoom.RandomId, which will generate pseudo-random ids for each model.

func (*Collection) UpdateFields

func (c *Collection) UpdateFields(fieldNames []string, model Model) error

UpdateFields updates only the given fields of the model. UpdateFields uses "last write wins" semantics. If another caller updates the the same fields concurrently, your updates may be overwritten. It will return an error if the type of model does not match the registered Collection, or if any of the given fieldNames are not found in the registered Collection. If UpdateFields is called on a model that has not yet been saved, it will not return an error. Instead, only the given fields will be saved in the database.

type CollectionOptions

type CollectionOptions struct {
	// FallbackMarshalerUnmarshaler is used to marshal/unmarshal any type
	// into a slice of bytes which is suitable for storing in the database. If
	// Zoom does not know how to directly encode a certain type into bytes, it
	// will use the FallbackMarshalerUnmarshaler. By default, the value is
	// GobMarshalerUnmarshaler which uses the builtin gob package. Zoom also
	// provides JSONMarshalerUnmarshaler to support json encoding out of the box.
	// Default: GobMarshalerUnmarshaler.
	FallbackMarshalerUnmarshaler MarshalerUnmarshaler
	// Iff Index is true, any model in the collection that is saved will be added
	// to a set in redis which acts as an index. The default value is false. The
	// key for the set is exposed via the IndexKey method. Queries and the
	// FindAll, Count, and DeleteAll methods will not work for unindexed
	// collections. This may change in future versions. Default: false.
	Index bool
	// Name is a unique string identifier to use for the collection in redis. All
	// models in this collection that are saved in the database will use the
	// collection name as a prefix. If not provided, the default name will be the
	// name of the model type without the package prefix or pointer declarations.
	// So for example, the default name corresponding to *models.User would be
	// "User". If a custom name is provided, it cannot contain a colon.
	// Default: The name of the model type, excluding package prefix and pointer
	// declarations.
	Name string
}

CollectionOptions contains various options for a pool.

type MarshalerUnmarshaler

type MarshalerUnmarshaler interface {
	// Marshal returns a byte-encoded representation of v
	Marshal(v interface{}) ([]byte, error)
	// Unmarshal parses byte-encoded data and store the result in the value
	// pointed to by v.
	Unmarshal(data []byte, v interface{}) error
}

MarshalerUnmarshaler defines a handler for marshaling arbitrary data structures into a byte format and unmarshaling bytes into arbitrary data structures. Any struct which correctly implements the interface should have the property that what you put in using Marshal is identical to what you get out using Unmarshal.

var (
	// GobMarshalerUnmarshaler is an object that implements MarshalerUnmarshaler
	// and uses uses the builtin gob package. Note that not all types are
	// supported by the gob package. See https://golang.org/pkg/encoding/gob/
	GobMarshalerUnmarshaler MarshalerUnmarshaler = gobMarshalerUnmarshaler{}
	// JSONMarshalerUnmarshaler is an object that implements MarshalerUnmarshaler
	// and uses the builtin json package. Note that not all types are supported
	// by the json package. See https://golang.org/pkg/encoding/json/#Marshal
	JSONMarshalerUnmarshaler MarshalerUnmarshaler = jsonMarshalerUnmarshaler{}
)

type Model

type Model interface {
	ModelId() string
	SetModelId(string)
}

Model is an interface encapsulating anything that can be saved and retrieved by Zoom. The only requirement is that a Model must have a getter and a setter for a unique id property.

func Models

func Models(in interface{}) []Model

Models converts in to []Model. It will panic if the underlying type of in is not a slice of some concrete type which implements Model.

type ModelNotFoundError

type ModelNotFoundError struct {
	Msg string
}

ModelNotFoundError is returned from Find and Query methods if a model that fits the given criteria is not found.

func (ModelNotFoundError) Error

func (e ModelNotFoundError) Error() string

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool represents a pool of connections. Each pool connects to one database and manages its own set of registered models.

func NewPool

func NewPool(options *PoolOptions) *Pool

NewPool initializes and returns a pool with the given options. To use all the default options, you can pass in nil.

func (*Pool) Close

func (p *Pool) Close() error

Close closes the pool. It should be run whenever the pool is no longer needed. It is often used in conjunction with defer.

func (*Pool) NewCollection

func (p *Pool) NewCollection(model Model, options *CollectionOptions) (*Collection, error)

NewCollection registers and returns a new collection of the given model type. You must create a collection for each model type you want to save. The type of model must be unique, i.e., not already registered, and must be a pointer to a struct. To use the default options, pass in nil as the options argument.

func (*Pool) NewConn

func (p *Pool) NewConn() redis.Conn

NewConn gets a connection from the pool and returns it. It can be used for directly interacting with the database. See http://godoc.org/github.com/garyburd/redigo/redis for full documentation on the redis.Conn type.

func (*Pool) NewTransaction

func (p *Pool) NewTransaction() *Transaction

NewTransaction instantiates and returns a new transaction.

type PoolOptions

type PoolOptions struct {
	// Address to connect to. Default: "localhost:6379"
	Address string
	// Network to use. Default: "tcp"
	Network string
	// Database id to use (using SELECT). Default: 0
	Database int
	// Password for a password-protected redis database. If not empty,
	// every connection will use the AUTH command during initialization
	// to authenticate with the database. Default: ""
	Password string
}

PoolOptions contains various options for a pool.

type Query

type Query struct {
	// contains filtered or unexported fields
}

Query represents a query which will retrieve some models from the database. A Query may consist of one or more query modifiers (e.g. Filter or Order) and may be executed with a query finisher (e.g. Run or Ids).

func (*Query) Count

func (q *Query) Count() (uint, error)

Count counts the number of models that would be returned by the query without actually retreiving the models themselves. Count will also return the first error that occured during the lifetime of the query object (if any). Otherwise, the second return value will be nil.

func (*Query) Exclude

func (q *Query) Exclude(fields ...string) *Query

Exclude specifies one or more field names which will *not* be read from the database and scanned. Any other fields *will* be read and scanned into the resulting models when the query is run. You can only use one of Include or Exclude, not both on the same query. Exclude will set an error if you try to use it with Include on the same query. The error, same as any other error that occurs during the lifetime of the query, is not returned until the query is executed. When the query is executed the first error that occured during the lifetime of the query object (if any) will be returned.

func (*Query) Filter

func (q *Query) Filter(filterString string, value interface{}) *Query

Filter applies a filter to the query, which will cause the query to only return models with attributes matching the expression. filterString should be an expression which includes a fieldName, a space, and an operator in that order. Operators must be one of "=", "!=", ">", "<", ">=", or "<=". You can only use Filter on fields which are indexed, i.e. those which have the `zoom:"index"` struct tag. If multiple filters are applied to the same query, the query will only return models which have matches for ALL of the filters. I.e. applying multiple filters is logially equivalent to combining them with a AND or INTERSECT operator. Filter will set an error on the query if the arguments are improperly formated, if the field you are attempting to filter is not indexed, or if the type of value does not match the type of the field. The error, same as any other error that occurs during the lifetime of the query, is not returned until the query is executed. When the query is executed the first error that occured during the lifetime of the query object (if any) will be returned.

func (*Query) Ids

func (q *Query) Ids() ([]string, error)

Ids returns only the ids of the models without actually retreiving the models themselves. Ids will return the first error that occured during the lifetime of the query object (if any).

func (*Query) Include

func (q *Query) Include(fields ...string) *Query

Include specifies one or more field names which will be read from the database and scanned into the resulting models when the query is run. Field names which are not specified in Include will not be read or scanned. You can only use one of Include or Exclude, not both on the same query. Include will set an error if you try to use it with Exclude on the same query. The error, same as any other error that occurs during the lifetime of the query, is not returned until the query is executed. When the query is executed the first error that occured during the lifetime of the query object (if any) will be returned.

func (*Query) Limit

func (q *Query) Limit(amount uint) *Query

Limit specifies an upper limit on the number of records to return. If amount is 0, no limit will be applied. The default value is 0.

func (*Query) Offset

func (q *Query) Offset(amount uint) *Query

Offset specifies a starting index (inclusive) from which to start counting records that will be returned. The default value is 0.

func (*Query) Order

func (q *Query) Order(fieldName string) *Query

Order specifies a field by which to sort the models. fieldName should be a field in the struct type corresponding to the Collection used in the query constructor. By default, the records are sorted by ascending order by the given field. To sort by descending order, put a negative sign before the field name. Zoom can only sort by fields which have been indexed, i.e. those which have the `zoom:"index"` struct tag. However, in the future this may change. Only one order may be specified per query. However in the future, secondary orders may be allowed, and will take effect when two or more models have the same value for the primary order field. Order will set an error on the query if the fieldName is invalid, if another order has already been applied to the query, or if the fieldName specified does not correspond to an indexed field. The error, same as any other error that occurs during the lifetime of the query, is not returned until the query is executed. When the query is executed the first error that occured during the lifetime of the query object (if any) will be returned.

func (*Query) Run

func (q *Query) Run(models interface{}) error

Run executes the query and scans the results into models. The type of models should be a pointer to a slice of pointers to a registered Model. Run will return the first error that occured during the lifetime of the query object (if any). It will also return an error if models is the wrong type.

func (*Query) RunOne

func (q *Query) RunOne(model Model) error

RunOne is exactly like Run but finds only the first model that fits the query criteria and scans the values into model. If no model fits the criteria, an error will be returned.

func (*Query) String

func (q *Query) String() string

String satisfies fmt.Stringer and prints out the query in a format that matches the go code used to declare it.

type RandomId

type RandomId struct {
	Id string
}

RandomId can be embedded in any model struct in order to satisfy the Model interface. The first time the ModelId method is called on an embedded RandomId, it will generate a pseudo-random id which is highly likely to be unique.

func (*RandomId) ModelId

func (r *RandomId) ModelId() string

ModelId returns the id of the model, satisfying the Model interface. If r.Id is an empty string, it will generate a pseudo-random id which is highly likely to be unique.

func (*RandomId) SetModelId

func (r *RandomId) SetModelId(id string)

SetModelId sets the id of the model, satisfying the Model interface

type ReplyHandler

type ReplyHandler func(interface{}) error

ReplyHandler is a function which does something with the reply from a redis command or script.

type Transaction

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction is an abstraction layer around a redis transaction. Transactions consist of a set of actions which are either redis commands or lua scripts. Transactions feature delayed execution, so nothing toches the database until you call Exec.

func (*Transaction) Command

func (t *Transaction) Command(name string, args redis.Args, handler ReplyHandler)

Command adds a command action to the transaction with the given args. handler will be called with the reply from this specific command when the transaction is executed.

func (*Transaction) Count

func (t *Transaction) Count(c *Collection, count *int)

Count counts the number of models of the given type in the database in an existing transaction. It sets the value of count to the number of models. Any errors encountered will be added to the transaction and returned as an error when the transaction is executed.

func (*Transaction) Delete

func (t *Transaction) Delete(c *Collection, id string, deleted *bool)

Delete removes a model with the given type and id in an existing transaction. deleted will be set to true iff the model was successfully deleted when the transaction is executed. If the no model with the given type and id existed, the value of deleted will be set to false. Any errors encountered will be added to the transaction and returned as an error when the transaction is executed. You may pass in nil for deleted if you do not care whether or not the model was deleted.

func (*Transaction) DeleteAll

func (t *Transaction) DeleteAll(c *Collection, count *int)

DeleteAll delets all models for the given model type in an existing transaction. The value of count will be set to the number of models that were successfully deleted when the transaction is executed. Any errors encountered will be added to the transaction and returned as an error when the transaction is executed. You may pass in nil for count if you do not care about the number of models that were deleted.

func (*Transaction) Exec

func (t *Transaction) Exec() error

Exec executes the transaction, sequentially sending each action and calling all the action handlers with the corresponding replies.

func (*Transaction) Find

func (t *Transaction) Find(c *Collection, id string, model Model)

Find retrieves a model with the given id from redis and scans its values into model in an existing transaction. model should be a pointer to a struct of a registered type corresponding to the Collection. find will mutate the struct, filling in its fields and overwriting any previous values. Any errors encountered will be added to the transaction and returned as an error when the transaction is executed.

func (*Transaction) FindAll

func (t *Transaction) FindAll(c *Collection, models interface{})

FindAll finds all the models of the given type and scans the values of the models into models in an existing transaction. See http://redis.io/topics/transactions. models must be a pointer to a slice of models with a type corresponding to the Collection. findAll will grow the models slice as needed and if any of the models in the models slice are nil, FindAll will use reflection to allocate memory for them. Any errors encountered will be added to the transaction and returned as an error when the transaction is executed.

func (*Transaction) FindFields

func (t *Transaction) FindFields(c *Collection, id string, fieldNames []string, model Model)

FindFields is like Find but finds and sets only the specified fields. Any fields of the model which are not in the given fieldNames are not mutated. FindFields will return an error if any of the given fieldNames are not found in the model type.

func (*Transaction) Save

func (t *Transaction) Save(c *Collection, model Model)

Save writes a model (a struct which satisfies the Model interface) to the redis database inside an existing transaction. save will set the err property of the transaction if the type of model does not match the registered Collection, which will cause exec to fail immediately and return the error. To make a struct satisfy the Model interface, you can embed zoom.RandomId, which will generate pseudo-random ids for each model. Any errors encountered will be added to the transaction and returned as an error when the transaction is executed.

func (*Transaction) Script

func (t *Transaction) Script(script *redis.Script, args redis.Args, handler ReplyHandler)

Script adds a script action to the transaction with the given args. handler will be called with the reply from this specific script when the transaction is executed.

func (*Transaction) UpdateFields

func (t *Transaction) UpdateFields(c *Collection, fieldNames []string, model Model)

UpdateFields updates only the given fields of the model inside an existing transaction. UpdateFields will set the err property of the transaction if the type of model does not match the registered Collection, or if any of the given fieldNames are not found in the model type. In either case, the transaction will return the error when you call Exec. UpdateFields uses "last write wins" semantics. If another caller updates the the same fields concurrently, your updates may be overwritten. If UpdateFields is called on a model that has not yet been saved, it will not return an error. Instead, only the given fields will be saved in the database.