stategate

README

stategate

A pluggable "Application State Gateway" that enforces the Event Sourcing Pattern for securely persisting & broadcasting application state changes

What is Event Sourcing?

Event sourcing persists the state of a business entity such an Order or a Customer as a sequence of state-changing events. Whenever the state of a business entity changes, a new event is appended to the list of events. Since saving an event is a single operation, it is inherently atomic. The application reconstructs an entity’s current state by replaying the events.

Applications persist events in an event store, which is a database of events. The store has an API for adding and retrieving an entity’s events. The event store also behaves like a message broker. It provides an API that enables services to subscribe to events. When a service saves an event in the event store, it is delivered to all interested subscribers.

• API Documentation

Features

• 9 Simple API Methods for interacting with application entites & events:

  • /stategate.EntityService/Set sets the current state value of an entity, adds it to the event log, then broadcast the event to all interested consumers(EventService.Stream)
  • /stategate.EntityService/Edit overwrites the k/v pairs present in the entity request without replacing the entire entity. It then adds the state change to the event log, then broadcast the event to all interested consumers(EventService.Stream)
  • /stategate.EntityService/Revert revert reverts an Entity to a previous version of itself by querying the event store- reverting an entity dispatches an event since it is a state change
  • /stategate.EntityService/Get gets an entity's current state
  • /stategate.EntityService/Del hard deletes an entity from current state store, adds it's state prior to deletion to the event log, then broadcast the event to all interested consumers(EventService.Stream)
  • /stategate.EntityService/Search queries the current state of entities
  • /stategate.EventService/Stream creates an event stream/subscription to changes to entities. Glob matching is supported.
  • /stategate.EventService/Search queries historical events
  • /stategate.EventService/Get get a single historical event

• Capture all changes to an application's state(entities) as a sequence of events.

• Stateless & horizontally scaleable

• Native gRPC support

  • protobuf schema

• Embedded REST support / (transcoding)

  • open api schema

• Embedded grpcweb support (transcoding)

• Metrics Server(prometheus/pprof)

• Authentication - JWT/OAuth with remote JWKS verification

• Authorization - Rego based Authorization engine

• Autogenerated Client gRPC SDK's

  • Go
    • 
  • Node
  • PHP
  • C#
  • Java
  • gRPC Web
  • Python
  • Ruby

• Structured JSON Logs

• Sample Kubernetes Manifest

• Sample Docker Compose

• Pluggable "Channel" Providers

  • In-Memory(won't scale horizontally)
    • fully-tested
  • Nats
    • fully-tested
  • Nats Streaming(Stan)
  • Redis
    • fully-tested
  • Kafka
  • RabbitMQ

• Pluggable "Storage" Providers

  • MongoDb
    • fully-tested
  • PostgreSQL
  • MySQL
  • Cassandra

Goals

• Create a simple API interface for storing state(entities) and subscribing to state changes(events) using pluggable channel & storage providers
• Capture all changes to an application's state/entities as a sequence of events.
• Safe to swap backend providers without changing client-side code
• Type-safe client's generated in many languages
• Safe to expose to the public internet due to fine-grained authentication/authorization model.
• Capture a persistant, immutable historical record of all state changes to entities using a pluggable storage provider
• Revert/Rollback an entity to any previous version of itself at any point in time
• Store identity(jwt.claims) & timestamp in event logs to capture who is changing what & when
• Easy deployment model - fully configureable via environmental variables

Concepts

• Storage Provider: A stategate storage provider is a pluggable, 3rd party database storage service.

  • Storage providers provide persistance for all current entities/events and should be scaled independently of stategate instances.

• Channel Provider: A stategate channel provider is a pluggable, 3rd party message-queue/channel service.

  • Channel providers provide a way for stategate to broadcast events to itself while scaling horizontally.
  • Channel providers should be scaled independently of stategate instances.

• Entity: An entity represents a single record(k/v pairs) with a unique key with a given type, belonging to a particular domain

    // Entity represents a single record(k/v pairs) with a unique key with a given [type](https://en.wikipedia.org/wiki/Type_system), belonging to a particular [domain](https://en.wikipedia.org/wiki/Domain-driven_design)
    // EventService clients should use the EntityService to persist & interact with the current state of an entity.
    message Entity {
      // the entity's business domain(ex: accounting)
      // must not be empty or contain spaces
      string domain =1[(validator.field) = {regex : "^\\S+$"}];
      // the entity's type (ex: user)
      // must not be empty or contain spaces
      string type =2[(validator.field) = {regex : "^\\S+$"}];
      // the entity's key (unique within type). 
      // must not be empty or contain spaces
      string key =3[(validator.field) = {regex : "^\\S+$"}];
      // the entity's values (k/v pairs)
      google.protobuf.Struct values = 4[(validator.field) = {msg_exists : true}];
    }
  

• Event:

    // Event is primitive that represents a single state change to an entity
    // Events are persisted to history & broadcasted to interested consumers(Stream) any time an entity is created/modified/deleted
    // Events are immutable after creation and may be searched.
    // EventService client's may search events to query previous state of an entity(s)
    message Event {
      // identifies the event(uuid v4).
      string id = 1[(validator.field) = {uuid_ver : 4}];
      // state of an Entity after it has been mutated
      Entity entity = 2[(validator.field) = {msg_exists : true}];
      // the invoked method that triggered the event
      string method =5[(validator.field) = {string_not_empty : true}];
      // the authentication claims of the event producer.
      google.protobuf.Struct claims =3[(validator.field) = {msg_exists : true}];
      // timestamp(ns) of when the event was received.
      int64 time =4[(validator.field) = {int_gt : 0}];
    }
  

Environmental Variables

# port to serve on (optional). defaults to 8080
STATEGATE_PORT=8080
# enable debug logs (optional)
STATEGATE_DEBUG=true
# disable all authentication & authorization(jwks, request policies, response policies) (optional)
STATEGATE_AUTH_DISABLED=false
# tls cert file (optional)
STATEGATE_TLS_CERT_FILE=/tmp/certs/stategate.cert
# tls key file (optional)
STATEGATE_TLS_KEY_FILE=/tmp/certs/stategate.key

# JSON Web Key Set remote URI used for fetching jwt signing keys for verification/validation (optional)
STATEGATE_JWKS_URI=https://www.googleapis.com/oauth2/v3/certs

# base64 encoded OPA rego policy executed on inbound requests from clients (optional)
STATEGATE_REQUEST_POLICY=cGFja2FnZSBzdGF0ZWdhdGUuYXV0aHoKCmRlZmF1bHQgYWxsb3cgPSB0cnVl
# base64 encoded OPA rego policy executed on responses sent to clients (optional)
STATEGATE_RESPONSE_POLICY=cGFja2FnZSBzdGF0ZWdhdGUuYXV0aHoKCmRlZmF1bHQgYWxsb3cgPSB0cnVl
# channel provider configuration(JSON) options: [inmem, redis, nats, stan, kafka] REQUIRED
STATEGATE_CHANNEL_PROVIDER={ "name": "redis", "addr": "localhost:6379" }
# STATEGATE_CHANNEL_PROVIDER={ "name": "nats", "addr": "localhost:4222" }
# STATEGATE_CHANNEL_PROVIDER={ "name": "stan", "addr": "localhost:4222" }
# STATEGATE_CHANNEL_PROVIDER={ "name": "inmem" }

# storage provider configuration(JSON) options: [mongo] REQUIRED
STATEGATE_STORAGE_PROVIDER={ "name": "mongo", "database": "testing", "addr": "mongodb://localhost:27017/testing" }

FAQ