README
¶
ddb
Helper types and for writing expressive and reusable DynamoDB code in Go without getting in the way.
Introduction
The DynamoDB API can be bit unwieldy any scary to look at. It has many different types and functions with no clear guidance on how to combine them into a maintainable codebase. This library therefore provides an opionated set of small tools that provides some guidance for accessing Dynamod from Go without abstracting too much and getting in the way.
Specifically, this project is not a ORM of sorts. It simply provides several small utility methods that fit together really nicely and allows your codebase to just focus on several modelling principles that are specific to DynamoDB:
Principle 1: Mapping entities onto Items
Unlike relational data modelling, it is common with DynamoDB to store multiple entities into a single table. For example: when using DynamoDB it would be common that you store the "user" entity in the same table as the "team" entity. You would then use indexes and/or attribute overloading in a clever way to model the many-to-one relation between them.
To facilitate this, the library allows (and expects!) you to explicitely map each "entity" onto an "item" before it can be stored in dynamodb. There should be only one "item" type per table and the item type will be used for marshalling and unmarshalling through the dynamodbattribute package.
Each entity must implement the Itemizer
and Deitemizer interface to encode how
the entity is turned into an item and vice versa.
This is better explained through an example and might sound like a lot of unessary work but the mapping logic is an important part of the modelling process and a primary concern of the programmer. This library allows you to focus on that.
Principle 2: Thinking in Access Patterns
Relations databases come with a dedicated language (SQL) for encoding how to the database is accessed. And although AWS announced PartiQL support for DynamoDB it doesn't provide all the options.
Instead it is recommend to think about "access patterns" in order to model the data of your application. In the "user" and "group" example above an acess pattern could be "get-all-users-in-group" or "assign-user-to-group".
Similar to the first principle, this library brings the implementation of these access patterns to the forefront. Each acces pattern is represented by a function (or method) that is expected to return an expression builder, one of the basic operations (Put, Get, Update, Delete, Check and Query) and an entity for Put operations.
Again, see one of the examples below to get a better feel for this.
Principle 3: Transactions as compositions of basic operations
While relation database have transactions as a core functionality, for dynamodb this has only been added at the end of 2018. As such there is now a mix of transaction and non-transactional operations exposed by the official API.
This library focusses on the newest set of transaction operations and allows you to compose several basic access pattern functions (see above) into larger transactions that modify a set of items atomically across multiple tables.
Examples
The best way to illustrate the principles above is through so lets look at a few. Slowly building up the complexity
Example 1: Storing a simple Foo entity
The simplest possible table (table1) is one that stores just one entity and only has a partition key:
// The simplest possible item has just one attribute: the partition key
type Table1Item struct{
PK string `dynamodbav:"pk"`
}
// Keys indicates which attributes of this item represent the partition and sort key
func (Table1Item) Keys() (pk, sk string) { return "pk", ""}
// The "Foo" entity is stored in table 1, it just has an identifier.
type Foo struct{
ID int
}
// We map the Foo entity onto the FooItem. With the partion key set to the
// identifier. This is done for operations that store entities into the table: Put
func (e Foo) Item() ddb.Item {
return Table1Item{PK: strconv.Itoa(e.ID)}
}
// FromItem unmaps the item back into an entity. This is used for operations that read from the
// database: Query, Get etc
func (e *Foo) FromItem(it ddb.Item) error {
e.ID, _ = strconv.Atoi(it.(*Table1Item).PK)
return nil
}
// PutFoo implements the access pattern for storing new foo entities.
func PutFoo(id int) (eb expression.Builder, p dynamodb.Put, it Itemizer) {
p.SetTableName("table1")
return eb, p, &Foo{ID: id}
}
docs
- Items can also implement the itemizer interface
- Examples for each operation
- Item returned from entity.Item() should be a pointer, else unmarshal will fail
backlog
- MUST try to see what happens if access patterns are added to the item.
- SHOULD panic/error if a "op" is used after running it
- COULD add an option that always runs singleton operations in a transaction alsop
- COULD add option for consisten read
- COULD add option for ReturnConsumedCapacity
- COULD make Do return an TXDB-like interface for composability of operations
- COULD turn a set of Put, and Deletes into a BatchWriteRequest as well
- COULD turn a set Get requests into a BatchGetItem/BatchWriteItem request
- SHOULD support "ReturnValues" while wi.Put/Delete/Update doesn't support it
- COULD create error types that show the dynamodb input for debugging
- COULD be nice to have an helper that decodes the whole result into a slice of entities
- COULD be nice to have an helper that decodes the result with the expectation there is only one item
- COULD share more code between QueryResult and ScanResult
Documentation
¶
Index ¶
- Variables
- func EnableEmptyCollections() func(o *Options)
- func MarshalMap(in interface{}, enableEmptyCollections bool) (map[string]*dynamodb.AttributeValue, error)
- func UnmarshalAll(r Result, v interface{}) (err error)
- type Deitemizer
- type Dynamo
- type Item
- type Itemizer
- type Logger
- type Option
- type Options
- type Querier
- type Reader
- type Result
- type Scanner
- type Writer
- func Check(eb expression.Builder, o dynamodb.ConditionCheck, key Itemizer) *Writer
- func Delete(eb expression.Builder, o dynamodb.Delete, key Itemizer) *Writer
- func NewWriter(opts ...Option) (w *Writer)
- func Put(eb expression.Builder, o dynamodb.Put, item Itemizer) *Writer
- func Update(eb expression.Builder, o dynamodb.Update, key Itemizer) *Writer
- func (tx *Writer) Check(eb expression.Builder, chk dynamodb.ConditionCheck, key Itemizer) *Writer
- func (tx *Writer) Delete(eb expression.Builder, del dynamodb.Delete, key Itemizer) *Writer
- func (tx *Writer) Put(eb expression.Builder, put dynamodb.Put, item Itemizer) *Writer
- func (tx *Writer) Run(ctx context.Context, ddb Dynamo) (r Result, err error)
- func (tx *Writer) Update(eb expression.Builder, upd dynamodb.Update, key Itemizer) *Writer
Constants ¶
This section is empty.
Variables ¶
var DefaultOptions = []Option{EnableEmptyCollections()}
DefaultOptions will be used when no options are specified
Functions ¶
func EnableEmptyCollections ¶ added in v0.7.1
func EnableEmptyCollections() func(o *Options)
EnableEmptyCollections is an option that enables the EnableEmptyCollections on all marshal calls
func MarshalMap ¶ added in v0.7.0
func MarshalMap(in interface{}, enableEmptyCollections bool) (map[string]*dynamodb.AttributeValue, error)
Copied from aws-sdk-go, because Encoder{} does not have EncodeMap()
func UnmarshalAll ¶ added in v0.6.0
UnmarshalAll will run through all results in 'r' and unmarshal all items into 'v'. It will consume the result in the process and it cannot scanned again afterwards. Each element of v must implement the Itemizer and Deitemizer interface.
Types ¶
type Deitemizer ¶
type Dynamo ¶
type Dynamo interface {
PutItemWithContext(
aws.Context,
*dynamodb.PutItemInput,
...request.Option,
) (*dynamodb.PutItemOutput, error)
GetItemWithContext(
aws.Context,
*dynamodb.GetItemInput,
...request.Option,
) (*dynamodb.GetItemOutput, error)
DeleteItemWithContext(
aws.Context,
*dynamodb.DeleteItemInput,
...request.Option,
) (*dynamodb.DeleteItemOutput, error)
UpdateItemWithContext(
aws.Context,
*dynamodb.UpdateItemInput,
...request.Option,
) (*dynamodb.UpdateItemOutput, error)
QueryWithContext(
aws.Context,
*dynamodb.QueryInput,
...request.Option,
) (*dynamodb.QueryOutput, error)
TransactWriteItemsWithContext(
aws.Context,
*dynamodb.TransactWriteItemsInput,
...request.Option,
) (*dynamodb.TransactWriteItemsOutput, error)
TransactGetItemsWithContext(
aws.Context,
*dynamodb.TransactGetItemsInput,
...request.Option,
) (*dynamodb.TransactGetItemsOutput, error)
ScanWithContext(
aws.Context,
*dynamodb.ScanInput,
...request.Option,
) (*dynamodb.ScanOutput, error)
}
Dynamo describes the strict sub-set official DynamoDB interface that this library uses.
func LoggedDynamo ¶ added in v0.6.2
LoggedDynamo returns a dynamo interface that logs every interaction with dynamodb to the provider logger
type Logger ¶ added in v0.6.2
type Logger interface {
Printf(format string, v ...interface{})
}
Logger interface can be implemented to log all interaction with DynamoDB
type Options ¶ added in v0.7.1
type Options struct {
// contains filtered or unexported fields
}
Options holds option values for all options that we support
type Querier ¶
type Querier struct {
// contains filtered or unexported fields
}
Querier holds a DynamoDB query
func Query ¶
func Query(b expression.Builder, in dynamodb.QueryInput) (q *Querier)
Query sets up a query that can be run to fetch
type Reader ¶ added in v0.6.0
type Reader struct {
// contains filtered or unexported fields
}
Reader represents one or more read to dynamodb
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner holds a DynamoDB query
type Writer ¶ added in v0.6.0
type Writer struct {
// contains filtered or unexported fields
}
Writer represents one or more DynamoDB write operations
func Check ¶
func Check(eb expression.Builder, o dynamodb.ConditionCheck, key Itemizer) *Writer
Check will setup a write with the check
func (*Writer) Check ¶ added in v0.6.0
func (tx *Writer) Check(eb expression.Builder, chk dynamodb.ConditionCheck, key Itemizer) *Writer
Check will add a check operation to the write
Source Files