dynastorev2

package module
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 21, 2025 License: Apache-2.0 Imports: 13 Imported by: 0

README

dynastorev2

This package provides a CRUD (create, read, update and delete) store for Amazon DynamoDB using the AWS Go SDK v2.

Go Report Card Documentation

Overview

This is a rewrite of the original dynastore with the main differences being:

  1. It uses the Generics feature added in Go 1.18
  2. It is built on AWS Go SDK v2.
  3. The API has been simplified.
  4. I am still learning how to use Generics in Go...

Example

	ctx := context.Background()

	cfg, err := config.LoadDefaultConfig(ctx)
	if err != nil {
		// handle error
	}

	client = dynamodb.NewFromConfig(cfg)
	customerStore := dynastorev2.New[string, string, []byte](client, "tickets-table")

	fields := map[string]any{
		"created": time.Now().UTC().Round(time.Millisecond),
	}

	res, err := customerStore.Create(ctx,
		"customer",                                 // partition key
		"01FCFSDXQ8EYFCNMEA7C2WJG74",               // sort key
		[]byte(`{"name": "Stax"}`),                 // value, in this case JSON encoded value
		customerStore.WriteWithExtraFields(fields), // extra fields which could be indexed in the future
	)
	if err != nil {
		// handle error
	}

	// print out the version from the mutation result, this is used for optimistic locking
	fmt.Println("version", res.Version)

Creates a record which looks like this in Amazon DynamoDB.

id (Partition Key) name (Sort Key) version payload expires created
customer 01FCFSDXQ8EYFCNMEA7C2WJG74 1 {"name": "Stax"} null 2022-04-10T06:27:16.994Z

Note: This library doesn't aim to provide a high level abstraction for Amazon DynamoDB, you will need to learn how it works to understand some of the limitations to use it successfully.

Implementation Tips

Before you get started i recommend you review some of the code examples provided in the Amazon DynamoDB documentation.

  1. Don't use Amazon DynamoDB if you have anything beyond a simple K/V compatible model until you understand your access patterns.
  2. Use a single-table design to model your data.
  3. Use Universally Unique Lexicographically Sortable Identifier (ULID) for sort keys, this will help ensure a rational order of data in the table. Sort key by default is sorted in descending order, oldest first, newest last, exploiting this behaviour may mitigate some of the limitations with Amazon DynamoDB.
  4. If your using WriteWithTTL you need to deal with the fact that Amazon DynamoDB doesn't delete expired data straight away, records can hang around for up to 48 hours according to the documentation.

Status

  • Added CRUD with conditional checks and tests
  • List with pagination
  • Optimistic Locking for Updates
  • Locking
  • Leasing

References

Prior work in this space:

Updates to the original API are based on a great blog post by @davecheney https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis

License

This code was authored by Mark Wolfe and licensed under the Apache 2.0 license.

Documentation

Index

Examples

Constants

View Source 
const (
	// DefaultPartitionKeyAttribute this is the default partition key attribute name
	DefaultPartitionKeyAttribute = "id"

	// DefaultSortKeyAttribute this is the default sort key attribute name
	DefaultSortKeyAttribute = "name"

	// DefaultExpiresAttribute this is the default name for the dynamodb expiration attribute
	DefaultExpiresAttribute = "expires"

	// DefaultVersionAttribute this is the default name for the dynamodb version attribute used for optimistic locking during creates and atomic updates
	DefaultVersionAttribute = "version"

	// DefaultPayloadAttribute this is the default attribute name containing the encoded payload of the record
	DefaultPayloadAttribute = "payload"
)

Variables

View Source 
var (
	// ErrReservedField extra fields provided have an entry which conflicts with the keys in the table
	ErrReservedField = errors.New("dynastorev2: extra fields contained name which conflicts with table keys attributes")

	// ErrDeleteFailedKeyNotExists delete failed due to constraint added which checks the record exists when deleting
	ErrDeleteFailedKeyNotExists = errors.New("dynastorev2: delete failed as the partition and sort keys didn't exist in the table")

	// ErrKeyNotExists get failed due to partition and sort keys didn't exist in the table
	ErrKeyNotExists = errors.New("dynastorev2: get failed as the partition and sort keys didn't exist in the table")
)

Functions

func ApplyDeleteOptions added in v0.4.0 

func ApplyDeleteOptions[P Key, S Key](v *DeleteOptions[P, S], opts ...DeleteOption[P, S])

ApplyDeleteOptions applies the provided option values to the DeleteOptions struct

func ApplyReadOptions added in v0.4.0 

func ApplyReadOptions[P Key, S Key](v *ReadOptions[P, S], opts ...ReadOption[P, S])

ApplyReadOptions applies the provided option values to the ReadOptions struct

func ApplyStoreOptions added in v0.4.0 

func ApplyStoreOptions[P Key, S Key, V any](v *StoreOptions[P, S, V], opts ...StoreOption[P, S, V])

ApplyStoreOptions applies the provided option values to the StoreOptions struct

func ApplyWriteOptions added in v0.4.0 

func ApplyWriteOptions[P Key, S Key, V any](v *WriteOptions[P, S, V], opts ...WriteOption[P, S, V])

ApplyWriteOptions applies the provided option values to the WriteOptions struct

Types

type DeleteOption 

type DeleteOption[P Key, S Key] interface {
	Apply(opts *DeleteOptions[P, S])
}

DeleteOption sets a specific delete option

type DeleteOptions added in v0.4.0 

type DeleteOptions[P Key, S Key] struct {
	// contains filtered or unexported fields
}

DeleteOptions holds all available delete configuration options

type Key 

type Key interface {
	string | constraints.Integer | []byte
}

Key ensures the partition or sort key used is a valid type for DynamoDB, note this is also referred to as the Primary Key in the AWS documentation.

Each key attribute must be a scalar (meaning that it can hold only a single value).

type OperationDetails 

type OperationDetails struct {
	Name         string
	PartitionKey string
	SortKey      string
}

func OperationDetailsFromContext 

func OperationDetailsFromContext(ctx context.Context) *OperationDetails

OperationName extracts the name of the operation being handled in the given context. If it is not known, it returns nil.

type OperationResult 

type OperationResult struct {
	Version          int64                   `json:"version,omitempty"`
	ConsumedCapacity *types.ConsumedCapacity `json:"consumed_capacity,omitempty"`
	LastEvaluatedKey string                  `json:"last_evaluated_key,omitempty"`
}

OperationResult returned with operations to provide some information about the update

type ReadOption 

type ReadOption[P Key, S Key] interface {
	Apply(opts *ReadOptions[P, S])
}

ReadOptions sets a specific read option

type ReadOptionFunc added in v0.4.0 

type ReadOptionFunc[P Key, S Key] func(*ReadOptions[P, S])

ReadOptionFunc wraps a function and implements the ReadOption interface

func (ReadOptionFunc[P, S]) Apply added in v0.4.0 

func (fn ReadOptionFunc[P, S]) Apply(opts *ReadOptions[P, S])

Apply calls the wrapped function

type ReadOptions added in v0.4.0 

type ReadOptions[P Key, S Key] struct {
	// contains filtered or unexported fields
}

ReadOptions holds all available read configuration options

type Store 

type Store[P Key, S Key, V any] struct {
	// contains filtered or unexported fields
}

Store store using aws sdk v2

func New 

func New[P Key, S Key, V any](client *dynamodb.Client, tableName string, options ...StoreOption[P, S, V]) *Store[P, S, V]

New creates and configures a new store using aws sdk v2

Example 
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/wolfeidau/dynastorev2"
)

func main() {
	ctx := context.Background()

	cfg, err := config.LoadDefaultConfig(ctx)
	if err != nil {
		// handle error
	}

	client := dynamodb.NewFromConfig(cfg)
	customerStore := dynastorev2.New[string, string, []byte](client, "tickets-table")

	fields := map[string]any{
		"created": time.Now().UTC().Round(time.Millisecond),
	}

	res, err := customerStore.Create(ctx,
		"customer",                                 // partition key
		"01FCFSDXQ8EYFCNMEA7C2WJG74",               // sort key
		[]byte(`{"name": "Stax"}`),                 // value, in this case JSON encoded value
		customerStore.WriteWithExtraFields(fields), // extra fields which could be indexed in the future
	)
	if err != nil {
		// handle error
	}

	// print out the version from the mutation result, this is used for optimistic locking
	fmt.Println("version", res.Version)
}

Output:

func (*Store[P, S, V]) Create 

func (t *Store[P, S, V]) Create(ctx context.Context, partitionKey P, sortKey S, value V, options ...WriteOption[P, S, V]) (*OperationResult, error)

Create a record in DynamoDB using the provided partition and sort keys, a payload containing the value

Note this will use a condition to ensure the specified partition and sort keys don't exist in DynamoDB.

func (*Store[P, S, V]) Delete 

func (t *Store[P, S, V]) Delete(ctx context.Context, partitionKey P, sortKey S, options ...DeleteOption[P, S]) error

Delete a record in DynamoDB using the provided partition and sort keys

func (*Store[P, S, V]) DeleteWithCheck 

func (t *Store[P, S, V]) DeleteWithCheck(enabled bool) DeleteOption[P, S]

DeleteWithCheck delete with a check condition to ensure the record exists

func (*Store[P, S, V]) Get 

func (t *Store[P, S, V]) Get(ctx context.Context, partitionKey P, sortKey S, options ...ReadOption[P, S]) (*OperationResult, V, error)

Get a record in DynamoDB using the provided partition and sort keys

func (*Store[P, S, V]) ListBySortKeyPrefix 

func (t *Store[P, S, V]) ListBySortKeyPrefix(ctx context.Context, partitionKey P, prefix string, options ...ReadOption[P, S]) (*OperationResult, []V, error)

ListBySortKeyPrefix perform a query of the DynamoDB using hte partition key and a string prefix for the sort key. This is typically used when hierarchies are stored in this partition. For example if we have a customer addresses with an sort key with a format of (customer id)/(address id), to list the addresses for a customer you list using the customer id as the prefix.

Notes: 1. You the sort key must be a string to support this operation, this is a limitation of the AWs SDK. 2. ListBySortKeyPrefix will also return expired records as these may hang around for up to 48 hours according to the documentation, see: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html

func (*Store[P, S, V]) ReadWithConsistentRead 

func (t *Store[P, S, V]) ReadWithConsistentRead(consistentRead bool) ReadOption[P, S]

ReadWithConsistentRead enable the consistent read flag when performing get operations

func (*Store[P, S, V]) ReadWithIndex added in v0.5.0 

func (t *Store[P, S, V]) ReadWithIndex(name, partKey, sortKey string) ReadOption[P, S]

ReadWithIndex provide an index name when performing list operations

func (*Store[P, S, V]) ReadWithLastEvaluatedKey 

func (t *Store[P, S, V]) ReadWithLastEvaluatedKey(lastEvaluatedKey string) ReadOption[P, S]

ReadWithLastEvaluatedKey provide a last evaluated key when performing list operations

func (*Store[P, S, V]) ReadWithLimit 

func (t *Store[P, S, V]) ReadWithLimit(limit int32) ReadOption[P, S]

ReadWithLimit provide a record count limit when performing list operations

func (*Store[P, S, V]) Update 

func (t *Store[P, S, V]) Update(ctx context.Context, partitionKey P, sortKey S, value V, options ...WriteOption[P, S, V]) (*OperationResult, error)

Update a record in DynamoDB using the provided partition and sort keys, a payload containing the value

Note this will use a condition to ensure the specified partition and sort keys exist in DynamoDB.

func (*Store[P, S, V]) WriteWithCreateConstraintDisabled added in v0.2.0 

func (t *Store[P, S, V]) WriteWithCreateConstraintDisabled(createConstraintDisabled bool) WriteOption[P, S, V]

WriteWithCreateConstraintDisabled disable the check on create for existence of the rows

func (*Store[P, S, V]) WriteWithExtraFields 

func (t *Store[P, S, V]) WriteWithExtraFields(extraFields map[string]any) WriteOption[P, S, V]

WriteWithExtraFields assign extra fields provided to the record when written or updated

func (*Store[P, S, V]) WriteWithTTL 

func (t *Store[P, S, V]) WriteWithTTL(ttl time.Duration) WriteOption[P, S, V]

WriteWithTTL assigns a time to live (TTL) to the record when it is created or updated

func (*Store[P, S, V]) WriteWithVersion 

func (t *Store[P, S, V]) WriteWithVersion(version int64) WriteOption[P, S, V]

WriteWithVersion adds a condition check the provided version to enable optimistic locking

type StoreHooks 

type StoreHooks[P Key, S Key, V any] struct {
	// RequestBuilt will be invoked prior to dispatching the request to the AWS SDK
	RequestBuilt     func(ctx context.Context, pk P, sk S, params any) context.Context
	ResponseReceived func(ctx context.Context, pk P, sk S, params any) context.Context
}

StoreHooks is a container for callbacks that can instrument the datastore

type StoreOption 

type StoreOption[P Key, S Key, V any] interface {
	Apply(opts *StoreOptions[P, S, V])
}

StoreOption sets a specific store option

func WithStoreHooks 

func WithStoreHooks[P Key, S Key, V any](storeHooks *StoreHooks[P, S, V]) StoreOption[P, S, V]

type StoreOptionFunc added in v0.4.0 

type StoreOptionFunc[P Key, S Key, V any] func(*StoreOptions[P, S, V])

StoreOptionFunc wraps a function and implements the StoreOption interface

func (StoreOptionFunc[P, S, V]) Apply added in v0.4.0 

func (fn StoreOptionFunc[P, S, V]) Apply(opts *StoreOptions[P, S, V])

Apply calls the wrapped function.

type StoreOptions added in v0.4.0 

type StoreOptions[P Key, S Key, V any] struct {
	// contains filtered or unexported fields
}

StoreOptions holds all available store configuration options

type WriteOption 

type WriteOption[P Key, S Key, V any] interface {
	Apply(opts *WriteOptions[P, S, V])
}

Option sets a specific write option

type WriteOptionFunc added in v0.4.0 

type WriteOptionFunc[P Key, S Key, V any] func(*WriteOptions[P, S, V])

WriteOptionFunc wraps a function and implements the WriteOption interface

func (WriteOptionFunc[P, S, V]) Apply added in v0.4.0 

func (fn WriteOptionFunc[P, S, V]) Apply(opts *WriteOptions[P, S, V])

Apply calls the wrapped function

type WriteOptions added in v0.4.0 

type WriteOptions[P Key, S Key, V any] struct {
	// contains filtered or unexported fields
}

options holds all available write configuration options

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.