datastorage

Documentation

Overview

Package datastorage contains clients which are essentially wrappers over various data services such as google cloud datastore. The code in this package ensures that we isolate communication to these services from our business logic.

This isolation will help in better unit testing as well as a single place to change this communication logic as opposed to directly changing the business logic.

This file contains a list of interfaces that our data clients want to implement.

Index

• Variables
• type DataStoreClient
  • func NewDataStoreClient(ctx context.Context, cloudProject string) (*DataStoreClient, error)
  • func (c DataStoreClient) BatchPut(ctx context.Context, entities interface{}, keys interface{}) error
  • func (c DataStoreClient) Get(ctx context.Context, result interface{}, entityName string, key interface{}, ...) error
  • func (c DataStoreClient) Query(ctx context.Context, result interface{}, entityName string, ...) error
• type IDataClient
• type QueryFilter

Variables

var (
	ErrInsufficientArgs = errors.New("insufficent arguments")
	ErrConnection       = errors.New("connection error")
	ErrInvalidKey       = errors.New("invalid key")
	ErrInvalidType      = errors.New("invalid type")
	ErrEntityNotFound   = errors.New("entity not found")
	ErrInternal         = errors.New("internal error")
)

Types

type DataStoreClient

type DataStoreClient struct {
	// contains filtered or unexported fields
}

func NewDataStoreClient

func NewDataStoreClient(ctx context.Context, cloudProject string) (*DataStoreClient, error)

NewDataStoreClient function creates a new Datastore client and returns it if successful. On failure, returns an error. Note that this client is specifically designed to communicate with Google's Datastore service.

func (DataStoreClient) BatchPut

func (c DataStoreClient) BatchPut(
	ctx context.Context,
	entities interface{},
	keys interface{},
) error

BatchPut takes in a list of entities in the format []S{} where S is the struct's name. It also takes a list of keys in the format []*datastore.Key

The length of keys must match with that of entities which corresponds to a 1:1 mapping between them. Example usage:

entities := []A{
 { prop1: "p1", prop2: "p2" }
 { prop1: "p3", prop2: "p4" }
}

keys := []*datastore.Key {
 datastore.NameKey("A", "keyliteral", nil)
}

dsclient.BatchPut(ctx, entities, keys)

func (DataStoreClient) Get

func (c DataStoreClient) Get(ctx context.Context, result interface{}, entityName string, key interface{}, options ...interface{}) error

Get takes in the Entity name, entity key, options (in case the key needs ancestor) and an empty struct reference. Returns an error if the fetch was unsuccessful. Otherwise copies the entity data to the result argument which should be an empty struct reference.

Important Notes: 1. This function can accept 4 or 6 arguments depending on whether the entity in question requires an ancestor to be found or not. 2. The 4th and 6th (if present) arguments can be of type string or number because key can either be a NameKey or an IdKey. 3. If the result interface has a property called Key of type *datastore.Key, it will be autopopulated with the fetched entity's datastore Key.

Example usage: str := A{} Example 1: dsClient.Get(ctx, &str, "EntityA", "EntityAKey", "AncestorEntity", "AncestorEntityKey") Example 2: dsClient.Get(ctx, &str, "EntityA", 123, "AncestorEntity", 345) Example 3: dsClient.Get(ctx, &str, "EntityA", "k")

func (DataStoreClient) Query

func (c DataStoreClient) Query(
	ctx context.Context,
	result interface{},
	entityName string,
	filters []QueryFilter,
	order interface{},
	limit int,
	options ...interface{}) error

Query takes in entity name, list of query filters, order, limit and result in the format *[]S{} where S is struct's name. Returns an error if the fetch was unsuccessful. Otherwise copies the entity data to the result argument.

Important Notes: 1. The order attribute must either be a string field or nil. If nil, the query would use the default order to fetch the entity.

2. The limit attribute must be present and should be >= 0. If 0 the query will bring in all the records. A positive number will indicate a limit equal to that number. A negative limit will be ignore. *** The running time and number of external API calls made by this function scale linearly with the the query's limit attribute. Unless the result count is expected to be small, it is best to specify a limit; otherwise Query will continue until it finishes collecting results or the provided context expires.

Example usage: str := []A{}

queryFilters := []QueryFilter{
		{Field: "Field name", Operator: "=", Value: "Val"},
}

dsclient.Query(ctx, &str, "EntityA", queryFilters, "-attribute")

type IDataClient

type IDataClient interface {
	// Gets a single data item of the specified type, matching the specified key.
	// Optionally takes additional arguments that can be used for specific
	// implementations of this interface.
	//
	// Returns an error if the data item cannot be found.
	Get(
		ctx context.Context,
		result interface{},
		dataType string,
		key interface{},
		options ...interface{},
	) error

	// Query function performs a data storage query with the given filters,
	// order, limit and options. Returns the matching items from the query result.
	//
	// Filters should be tuples of property name, operator, and value.
	//
	// The order argument is used to specify the order of the query results.
	//
	// The limit argument is used to limit the number of items being read from
	// the data storage.
	//
	// The options argument can be used for specific implementations of this
	// function.
	//
	// The result is reflected in the "result" argument.
	Query(
		ctx context.Context,
		result interface{},
		dataType string,
		queryFilters []QueryFilter,
		order interface{},
		limit int,
		options ...interface{},
	) error

	// BatchPut function performs a data storage query to save
	// multiple entities given their corresponding keys into
	// the storage.
	//
	// Returns an error if the operation fails.
	BatchPut(
		ctx context.Context,
		entities interface{},
		keys interface{},
	) error
}

IDataClient interface has necessary data related functions that need to be implemented by the struct using it. For example, a datastore client struct implementing this interface would need to have datastore specific implementation of the functions mentioned in this interface.

type QueryFilter

type QueryFilter struct {
	Field    string
	Operator string
	Value    interface{}
}