ego

README

eGo

eGo is a minimal library that help build event-sourcing and CQRS application through a simple interface, and it allows developers to describe their commands, events and states **are defined using google protocol buffers **. Under the hood, ego leverages Go-Akt to scale out and guarantee performant, reliable persistence.

Table of Content

• Features
  • Event Sourced Behavior
    • Howto
    • Events Stream
    • Projection
    • Events Store
    • Offset Store
  • Durable State Behavior
    • State Store
    • Howto
    • State Stream

Features

Event Sourced Behavior

The EventSourcedBehavior is crucial for maintaining data consistency, especially in distributed systems. It defines how to handle the various commands (requests to perform actions) that are always directed at the event sourced entity. In eGo commands sent to the EventSourcedBehavior are processed in order. When a command is processed, it may result in the generation of events, which are then stored in an event store. Every event persisted has a revision number and timestamp that can help track it. The EventSourcedBehavior in eGo is responsible for defining how to handle events that are the result of command handlers. The end result of events handling is to build the new state of the event sourced entity. When running in cluster mode, aggregate root are sharded.

• Commands handler: The command handlers define how to handle each incoming command, which validations must be applied, and finally, which events will be persisted if any. When there is no event to be persisted a nil can be returned as a no-op. Command handlers are the meat of the event sourced actor. They encode the business rules of your event sourced actor and act as a guardian of the event sourced entity consistency. The command handler must first validate that the incoming command can be applied to the current model state. Any decision should be solely based on the data passed in the commands and the state of the Behavior. In case of successful validation, one or more events expressing the mutations are persisted. Once the events are persisted, they are applied to the state producing a new valid state.
• Events handler: The event handlers are used to mutate the state of the event sourced entity by applying the events to it. Event handlers must be pure functions as they will be used when instantiating the event sourced entity and replaying the event store.

Howto

To define an event sourced entity, one needs to:

1. define the state of the event sourced entity using google protocol buffers message
2. define the various commands that will be handled by the event sourced entity
3. define the various events that are result of the command handlers and that will be handled by the event sourced entity to return the new state of the event sourced entity
4. define and make sure the events store is properly implemented.
5. implement the EventSourcedBehavior interface.
6. call the Entity method of eGo engine

Events Stream

Every event handled by event sourced entity are pushed to an events stream. That enables real-time processing of events without having to interact with the events store. Just use Subscribe method of Engine and start iterating through the messages and cast every message to the Event.

Projection

One can add a projection to the eGo engine to help build a read model. Projections in eGo rely on an offset store to track how far they have consumed events persisted by the write model. The offset used in eGo is a timestamp-based offset. One can also:

• remove a given projection: this will stop the projection and remove it from the system
• check the status of a given projection

Events Store

One can implement a custom events store. See EventsStore. eGo comes packaged with two events store:

• Postgres: Schema can be found here
• Memory (for testing purpose only)

Offsets Store

One can implement a custom offsets store. See OffsetStore. eGo comes packaged with two offset store:

• Postgres: Schema can be found here
• Memory (for testing purpose only)

Durable State Behavior

The DurableStateBehavior represents a type of Actor that persists its full state after processing each command instead of using event sourcing. This type of Actor keeps its current state in memory during command handling and based upon the command response persists its full state into a durable store. The store can be a SQL or NoSQL database. The whole concept is given the current state of the actor and a command produce a new state with a higher version as shown in this diagram: (State, Command) => State DurableStateBehavior reacts to commands which result in a new version of the actor state. Only the latest version of the actor state is persisted to the durable store. There is no concept of history regarding the actor state since this is not an event sourced actor. However, one can rely on the version number of the actor state and exactly know how the actor state has evolved overtime. DurableStateBehavior version number are numerically incremented by the command handler which means it is imperative that the newer version of the state is greater than the current version by one. DurableStateBehavior will attempt to recover its state whenever available from the durable state. During a normal shutdown process, it will persist its current state to the durable store prior to shutting down. This behavior help maintain some consistency across the actor state evolution.

State Store

One can implement a custom state store. See StateStore. eGo comes packaged with two state stores:

• Postgres: Schema can be found here
• Memory (for testing purpose only)

Howto

To define a durable state entity, one needs to:

1. define the state of the entity using google protocol buffers message
2. define the various commands that will be handled by the entity
3. define and make sure the durable state store is properly implemented.
4. implements the DurableStateBehavior interface
5. start eGo engine with the option durable store using WithStateStore
6. call the DurableStateEntity method of eGo engine

Events Stream

DurableStateBehavior full state is pushed to an events stream. That enables real-time processing of state without having to interact with the state store. Just use Subscribe method of Engine and start iterating through the messages and cast every message to the DurableState.

Cluster

The cluster mode heavily relies on Go-Akt clustering.

Mocks

eGo ships in some mocks

Examples

Check the examples

Installation

go get github.com/tochemey/ego

Sample

package main

import (
	"context"
	"errors"
	"log"
	"os"
	"os/signal"
	"syscall"
	"time"

	"github.com/google/uuid"
	"google.golang.org/protobuf/proto"

	"github.com/tochemey/ego/v3"
	"github.com/tochemey/ego/v3/plugins/eventstore/memory"
	samplepb "github.com/tochemey/ego/v3/example/pbs/sample/pb/v1"
)

func main() {
	// create the go context
	ctx := context.Background()
	// create the event store
	eventStore := memory.NewEventsStore()
	// connect the event store
	_ = eventStore.Connect(ctx)
	// create the ego engine
	engine := ego.NewEngine("Sample", eventStore)
	// start ego engine
	_ = engine.Start(ctx)
	// create a persistence id
	entityID := uuid.NewString()
	// create an entity behavior with a given id
	behavior := NewAccountBehavior(entityID)
	// create an entity
	_ = engine.Entity(ctx, behavior)

	// send some commands to the pid
	var command proto.Message
	// create an account
	command = &samplepb.CreateAccount{
		AccountId:      entityID,
		AccountBalance: 500.00,
	}
	// send the command to the actor. Please don't ignore the error in production grid code
	reply, _, _ := engine.SendCommand(ctx, entityID, command, time.Minute)
	account := reply.(*samplepb.Account)
	log.Printf("current balance: %v", account.GetAccountBalance())

	// send another command to credit the balance
	command = &samplepb.CreditAccount{
		AccountId: entityID,
		Balance:   250,
	}

	reply, _, _ = engine.SendCommand(ctx, entityID, command, time.Minute)
	account = reply.(*samplepb.Account)
	log.Printf("current balance: %v", account.GetAccountBalance())

	// capture ctrl+c
	interruptSignal := make(chan os.Signal, 1)
	signal.Notify(interruptSignal, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)
	<-interruptSignal

	// disconnect the event store
	_ = eventStore.Disconnect(ctx)
	// stop the actor system
	_ = engine.Stop(ctx)
	os.Exit(0)
}

// AccountBehavior implements EntityBehavior
type AccountBehavior struct {
	id string
}

// make sure that AccountBehavior is a true persistence behavior
var _ ego.EntityBehavior = (*AccountBehavior)(nil)

// NewAccountBehavior creates an instance of AccountBehavior
func NewAccountBehavior(id string) *AccountBehavior {
	return &AccountBehavior{id: id}
}

// ID returns the id
func (a *AccountBehavior) ID() string {
	return a.id
}

// InitialState returns the initial state
func (a *AccountBehavior) InitialState() ego.State {
	return ego.State(new(samplepb.Account))
}

// HandleCommand handles every command that is sent to the persistent behavior
func (a *AccountBehavior) HandleCommand(_ context.Context, command ego.Command, _ ego.State) (events []ego.Event, err error) {
	switch cmd := command.(type) {
	case *samplepb.CreateAccount:
		// TODO in production grid app validate the command using the prior state
		return []ego.Event{
			&samplepb.AccountCreated{
				AccountId:      cmd.GetAccountId(),
				AccountBalance: cmd.GetAccountBalance(),
			},
		}, nil

	case *samplepb.CreditAccount:
		// TODO in production grid app validate the command using the prior state
		return []ego.Event{
			&samplepb.AccountCredited{
				AccountId:      cmd.GetAccountId(),
				AccountBalance: cmd.GetBalance(),
			},
		}, nil

	default:
		return nil, errors.New("unhandled command")
	}
}

// HandleEvent handles every event emitted
func (a *AccountBehavior) HandleEvent(_ context.Context, event ego.Event, priorState ego.State) (state ego.State, err error) {
	switch evt := event.(type) {
	case *samplepb.AccountCreated:
		return &samplepb.Account{
			AccountId:      evt.GetAccountId(),
			AccountBalance: evt.GetAccountBalance(),
		}, nil

	case *samplepb.AccountCredited:
		account := priorState.(*samplepb.Account)
		bal := account.GetAccountBalance() + evt.GetAccountBalance()
		return &samplepb.Account{
			AccountId:      evt.GetAccountId(),
			AccountBalance: bal,
		}, nil

	default:
		return nil, errors.New("unhandled event")
	}
}

Versioning

The version system adopted in eGo deviates a bit from the standard semantic versioning system. The version format is as follows:

• The MAJOR part of the version will stay at v3 for the meantime.
• The MINOR part of the version will cater for any new features, breaking changes with a note on the breaking changes.
• The PATCH part of the version will cater for dependencies upgrades, bug fixes, security patches and co.

The versioning will remain like v3.x.x until further notice.

Contribution

Contributions are welcome! The project adheres to Semantic Versioning and Conventional Commits. This repo uses Earthly.

There are two ways you can become a contributor:

1. Request to become a collaborator, and then you can just open pull requests against the repository without forking it.
2. Follow these steps

• Fork the repository
• Create a feature branch
• Set your docker credentials on your fork using the following secret names: DOCKER_USER and DOCKER_PASS
• Submit a pull request

Test & Linter

Prior to submitting a pull request, please run:

earthly +test

Documentation

Index

• Variables
• type Command
• type DurableStateBehavior
• type Engine
  • func NewEngine(name string, eventsStore persistence.EventsStore, opts ...Option) *Engine
  • func (engine *Engine) AddProjection(ctx context.Context, name string, handler projection.Handler, ...) error
  • func (engine *Engine) DurableStateEntity(ctx context.Context, behavior DurableStateBehavior) error
  • func (engine *Engine) Entity(ctx context.Context, behavior EventSourcedBehavior) error
  • func (engine *Engine) IsProjectionRunning(ctx context.Context, name string) (bool, error)
  • func (engine *Engine) RemoveProjection(ctx context.Context, name string) error
  • func (engine *Engine) SendCommand(ctx context.Context, entityID string, cmd Command, timeout time.Duration) (resultingState State, revision uint64, err error)
  • func (engine *Engine) Start(ctx context.Context) error
  • func (engine *Engine) Started() bool
  • func (engine *Engine) Stop(ctx context.Context) error
  • func (engine *Engine) Subscribe() (eventstream.Subscriber, error)
• type Event
• type EventSourcedBehavior
• type Option
  • func WithCluster(provider discovery.Provider, partitionCount uint64, minimumPeersQuorum uint16, ...) Option
  • func WithLogger(logger log.Logger) Option
  • func WithStateStore(stateStore persistence.StateStore) Option
• type OptionFunc
  • func (f OptionFunc) Apply(e *Engine)
• type State

Variables

var (
	// ErrEngineNotStarted is returned when the eGo engine has not started
	ErrEngineNotStarted = errors.New("eGo engine has not started")
	// ErrUndefinedEntityID is returned when sending a command to an undefined entity
	ErrUndefinedEntityID = errors.New("eGo entity id is not defined")
	// ErrCommandReplyUnmarshalling is returned when unmarshalling command reply failed
	ErrCommandReplyUnmarshalling = errors.New("failed to parse command reply")
	// ErrDurableStateStoreRequired is returned when the eGo engine durable store is not set
	ErrDurableStateStoreRequired = errors.New("durable state store is required")
)

Types

type Command

type Command proto.Message

type DurableStateBehavior

type DurableStateBehavior interface {
	// ID defines the id that will be used in the event journal.
	// This helps track the entity in the events store.
	ID() string
	// InitialState returns the durable state actor initial state.
	// This is set as the initial state when there are no snapshots found the entity
	InitialState() State
	// HandleCommand processes every command sent to the DurableStateBehavior. One needs to use the command, the priorVersion and the priorState sent to produce a newState and newVersion.
	// This defines how to handle each incoming command, which validations must be applied, and finally, whether a resulting state will be persisted depending upon the response.
	// They encode the business rules of your durable state actor and act as a guardian of the actor consistency.
	// The command handler must first validate that the incoming command can be applied to the current model state.
	// Any decision should be solely based on the data passed in the command, the priorVersion and the priorState.
	// In case of successful validation and processing , the new state will be stored in the durable store depending upon response.
	// The actor state will be updated with the newState only if the newVersion is 1 more than the already existing state.
	HandleCommand(ctx context.Context, command Command, priorVersion uint64, priorState State) (newState State, newVersion uint64, err error)
}

DurableStateBehavior represents a type of Actor that persists its full state after processing each command instead of using event sourcing. This type of Actor keeps its current state in memory during command handling and based upon the command response persists its full state into a durable store. The store can be a SQL or NoSQL database. The whole concept is given the current state of the actor and a command produce a new state with a higher version as shown in this diagram: (State, Command) => State DurableStateBehavior reacts to commands which result in a new version of the actor state. Only the latest version of the actor state is persisted to the durable store. There is no concept of history regarding the actor state since this is not an event sourced actor. However, one can rely on the version number of the actor state and exactly know how the actor state has evolved overtime. State actor version number are numerically incremented by the command handler which means it is imperative that the newer version of the state is greater than the current version by one.

DurableStateBehavior will attempt to recover its state whenever available from the durable state. During a normal shutdown process, it will persist its current state to the durable store prior to shutting down. This behavior help maintain some consistency across the actor state evolution.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine represents the engine that empowers the various entities

func NewEngine

func NewEngine(name string, eventsStore persistence.EventsStore, opts ...Option) *Engine

NewEngine creates an instance of Engine

func (*Engine) AddProjection

func (engine *Engine) AddProjection(ctx context.Context, name string, handler projection.Handler, offsetStore offsetstore.OffsetStore, opts ...projection.Option) error

AddProjection add a projection to the running eGo engine and starts it

func (*Engine) DurableStateEntity

func (engine *Engine) DurableStateEntity(ctx context.Context, behavior DurableStateBehavior) error

DurableStateEntity creates a durable state entity. A DurableStateEntity persists its full state into a durable store without any history of the state evolution. A durable state entity receives a (non-persistent) command which is first validated if it can be applied to the current state. Here validation can mean anything, from simple inspection of a command message’s fields up to a conversation with several external services, for instance. If validation succeeds, a new state is generated from the command, representing the outcome of the command. The new state is persisted and, after successful persistence, used to change the actor’s state. During a normal shutdown process, it will persist its current state to the durable store prior to shutting down. One can use the SendCommand to send a command a durable state entity.

func (*Engine) Entity

func (engine *Engine) Entity(ctx context.Context, behavior EventSourcedBehavior) error

Entity creates an event sourced entity. Entity persists its full state into an events store that tracks the history based upon events that occurred. An event sourced entity receives a (non-persistent) command which is first validated if it can be applied to the current state. Here validation can mean anything, from simple inspection of a command message’s fields up to a conversation with several external services, for instance. If validation succeeds, events are generated from the command, representing the outcome of the command. These events are then persisted and, after successful persistence, used to change the actor’s state. When the event sourced entity needs to be recovered, only the persisted events are replayed of which we know that they can be successfully applied. In other words, events cannot fail when being replayed to a persistent actor, in contrast to commands. When there are no events to persist the event sourced entity will return the current state of the entity. One can use the SendCommand to send a command a durable state entity.

func (*Engine) IsProjectionRunning

func (engine *Engine) IsProjectionRunning(ctx context.Context, name string) (bool, error)

IsProjectionRunning returns true when the projection is active and running One needs to check the error to see whether this function does not return a false negative

func (*Engine) RemoveProjection

func (engine *Engine) RemoveProjection(ctx context.Context, name string) error

RemoveProjection stops and removes a given projection from the engine

func (*Engine) SendCommand

func (engine *Engine) SendCommand(ctx context.Context, entityID string, cmd Command, timeout time.Duration) (resultingState State, revision uint64, err error)

SendCommand sends command to a given entity ref. This will return: 1. the resulting state after the command has been handled and the emitted event/durable state persisted 2. nil when there is no resulting state or no event persisted 3. an error in case of error

func (*Engine) Start

func (engine *Engine) Start(ctx context.Context) error

Start starts the ego engine

func (*Engine) Started

func (engine *Engine) Started() bool

Started returns true when the eGo engine has started

func (*Engine) Stop

func (engine *Engine) Stop(ctx context.Context) error

Stop stops the ego engine

func (*Engine) Subscribe

func (engine *Engine) Subscribe() (eventstream.Subscriber, error)

Subscribe creates an events subscriber

type Event

type Event proto.Message

type EventSourcedBehavior

type EventSourcedBehavior interface {
	// ID defines the id that will be used in the event journal.
	// This helps track the entity in the events store.
	ID() string
	// InitialState returns the event sourced actor initial state.
	// This is set as the initial state when there are no snapshots found the entity
	InitialState() State
	// HandleCommand helps handle commands received by the event sourced actor. The command handlers define how to handle each incoming command,
	// which validations must be applied, and finally, which events will be persisted if any. When there is no event to be persisted a nil can
	// be returned as a no-op. Command handlers are the meat of the event sourced actor.
	// They encode the business rules of your event sourced actor and act as a guardian of the event sourced actor consistency.
	// The command must first validate that the incoming command can be applied to the current model state.
	//  Any decision should be solely based on the data passed in the commands and the state of the Behavior.
	// In case of successful validation, one or more events expressing the mutations are persisted.
	// Once the events are persisted, they are applied to the state producing a new valid state.
	// Every event emitted are processed one after the other in the same order they were emitted to guarantee consistency.
	// It is at the discretion of the application developer to know in which order a given command should return the list of events
	// This is really powerful when a command needs to return two events. For instance, an OpenAccount command can result in two events: one is AccountOpened and the second is AccountCredited
	HandleCommand(ctx context.Context, command Command, priorState State) (events []Event, err error)
	// HandleEvent handle events emitted by the command handlers. The event handlers are used to mutate the state of the event sourced actor by applying the events to it.
	// Event handlers must be pure functions as they will be used when instantiating the event sourced actor and replaying the event journal.
	HandleEvent(ctx context.Context, event Event, priorState State) (state State, err error)
}

EventSourcedBehavior defines an event sourced behavior when modeling a CQRS EventSourcedBehavior.

type Option

type Option interface {
	// Apply sets the Option value of a config.
	Apply(e *Engine)
}

Option is the interface that applies a configuration option.

func WithCluster

func WithCluster(provider discovery.Provider, partitionCount uint64, minimumPeersQuorum uint16, host string, remotingPort, gossipPort, peersPort int) Option

WithCluster enables cluster mode

func WithLogger

func WithLogger(logger log.Logger) Option

WithLogger sets the logger

func WithStateStore

func WithStateStore(stateStore persistence.StateStore) Option

WithStateStore sets the durable store. This is necessary when creating a durable state entity

type OptionFunc

type OptionFunc func(e *Engine)

OptionFunc implements the Option interface.

func (OptionFunc) Apply

func (f OptionFunc) Apply(e *Engine)

Apply applies the options to Engine

type State

type State proto.Message