flipbook

README

Flipbook

Flipbook is an event sourcing platform focusing on abstracting event storage, replication and reduction correct implementation complexity, leaving developers with a powerful, painless and minimalist API. The main features are:

• Event Store

  • gRPC API
  • Supported Backends
    • AWS DynamoDB

• Snapshot Store

  • gRPC API
  • Supported Backends
    • Redis

• SDKs

  • Golang
  • Python (Coming soon!)
  • NodeJS (Coming soon!)

• Postgres

• AWS DynamoDB

• Redis

• Input Validation

• Logging

• gRPC Server Instrumentation (middlewares for observability and tls)

• EventBus

• Tests & Benchmarks

  • go test ./test/functional

• Golang Event Sourcing SDK

• Typescript Event Sourcing SDK

Table of Contents

• Flipbook
  • Table of Contents
  • A word about complexity
  • Event Sourcing, a real-world explanation
    • Question 1
      • Answer 1: yes
      • Answer 2: no
      • Answer 3: no, but I need the state to be "auditable"
    • Question 2
      • Answer 1: yes
      • Answer 2: no
    • Question 3
      • Answer 1: yes
      • Answer 2: no
    • Question 4
      • Answer 1: yes
      • Answer 2: no
  • Flipbook Concepts
    • Version-Neutral State
    • Time-Neutral State
    • Event Reducer
    • Event Store
    • Event Bus
    • Snapshot Store
  • Flipbook Project
    • Repository Structure
    • Usage
    • Development
  • Flipbook Features
    • Event Store - gRPC API
    • Event Store - Supported Backends
      • AWS DynamoDB Event Store
    • Snapshot Store - gRPC API
    • Snapshot Store - Supported Backends
    • ToDos for v1

A word about complexity

We found that the complexity for event sourcing implementation can be broken down to 3 main challenges:

1. Making the best decision about using it as an architectural pattern.
2. Creating an optimal design for your states and events.
3. Implementing it correctly as a part of your software and infrastructure.

And for each statement above, we'll dive into the necessary information for anyone wishing to consider event sourcing as the right choice to solve a specific use case.

Event Sourcing, a real-world explanation

First things first, you need to decide if event sourcing is the right architecture pattern for your use case. You can search and find a lot (really, a lot) of good definitions for "event sourcing". Some more in the technical side, some more detailed, but we'll focus here on the real world use. And to make a good decision, we decided that the best criteria is to answer each question below.

Question 1

Do I need to be able to reproduce a state based on a sequence of changes caused by external operations?

Answer 1: yes

For example, a wallet ledger is good example, 'cause the actual balance for any wallet can be determined by reapplying all the events (transactions like deposits, payments, withdrawals and etc) tha 'caused some change in the final state of the balance. It's important to start familiarizing with common terms in the event sourcing world.

Terms	Example	Definition
"Event"		An event is always a description of what happened in the past containing information about an operation that lead to some change.
"State", "Aggregate"		"Is the combination of the original values in the object plus any modifications made to them." you can search for program state, object-oriented programming or other context-aware definitions of state in software. The core concept is always, the same.
"Command", "External Operation"
"Reduce", "Apply Event"
"Event Sourcing"

Answer 2: no

So you will be ok with a state persistence architecture ideal for saving the final state resulting from each operation. In this case we recommend to design the optimal state structure for your read/write operations.

• Relational: Postgres, MySQL, MariaDB and etc.
• Graphs: Neo4j, DGraph and etc.
• Documents (NoSQL): MongoDB, DynamoDB, Firestore and etc.
• Search Engine: Elasticsearch and etc.
• Timeseries: InfluxDB, Postgres, Prometheus, Thanos and etc.

And many, many others specialized engines for each data structure.

Answer 3: no, but I need the state to be "auditable"

You can follow the same approach commented on the Answer 2 and implement a properly audit log for your software/system.

Question 2

Do I know all the possible events that can mutate the state?

Answer 1: yes

To see if your concept of event is right, you can write down the field specification below for each event:

Field	Type	Example	Description
State ID	Unique Identifier	"cfd7cdf3-9f96-4932-9587-9566c16403ab", "some.nickname", "/unique/path/to/file"	Unique identifier of the mutated state that generated this event.
State Version	Integer	1, 2, null	A monotonic increasing sequence, greater than zero that places the event in the correct ascending order to be applied on the state. The 0 version is used to point to the last version and to store state metadata. If the state version is null we call it a version-neutral state.
Event Timestamp	Date and time	"2022-12-04T14:46:45.402Z", 1670165226, null	The temporal representation of a point in time when the event was generated. Yes, you can have events with this timestamp pointing to the future, and we can have events happening in the same time. For more information read about time-neutral state.
Event Name	String	"Approved", "Sent", "Bloomed"	An identifier for this event to be used when reducing an event sequence. It defines which logic to apply to the state using the Event Payload information. See event reducer for an extensive definition of this concept.
Event Payload	Any	{"amount":56,"from":"Winter","to":"Spring"}	All the information about the mutation caused on the state. It only has a meaning with the Event Name property.

Answer 2: no

Then we recommend to invest some time to design your use case through DDD practices.

Question 3

Does my use case implies multiple concurrent mutation operations happening over the same state?

Answer 1: yes

While this can be a symptom of a bad-design outcome for your data structure, sometimes it's an inevitable characteristic of your use case.

Answer 2: no

Question 4

Does my use case have a heavier workload on the write side?

Answer 1: yes

Answer 2: no

Flipbook Concepts

Version-Neutral State

Time-Neutral State

Event Reducer

Event Store

Event Bus

Snapshot Store

Flipbook Project

Repository Structure

Usage

Development

Flipbook Features

Event Store - gRPC API

Event Store - Supported Backends

AWS DynamoDB Event Store

Snapshot Store - gRPC API

Snapshot Store - Supported Backends

ToDos for v1

• SnapshotStore
  • Developer Documentation
  • Input Validation
  • Metrics Endpoint
  • Error Handling/Logging
  • CI (GitHub Actions)
    • Test Automation
    • Container Build
  • Backend Support
    • Redis (Ring Cluster)
• EventStore
  • Developer Documentation
  • Input Validation
  • Metrics Endpoint
  • Error Handling/Logging
  • CI (GitHub Actions)
    • Test Automation
    • Container Build
  • Backend Support
    • AWS DynamoDB
• Cloud Support
  • AWS
    • Terraform module for ECS
    • Terraform module for ElastiCache
    • Terraform module for DynamoDB
  • GCP
  • Azure