stategate

README

stategate

A pluggable "Application State Gateway" that enforces the Event Sourcing Pattern for securely persisting & broadcasting application state changes

What is Event Sourcing?

Event sourcing persists the state of a business entity such an Order or a Customer as a sequence of state-changing events. Whenever the state of a business entity changes, a new event is appended to the list of events. Since saving an event is a single operation, it is inherently atomic. The application reconstructs an entity’s current state by replaying the events.

Applications persist events in an event store, which is a database of events. The store has an API for adding and retrieving an entity’s events. The event store also behaves like a message broker. It provides an API that enables services to subscribe to events. When a service saves an event in the event store, it is delivered to all interested subscribers.

• API Documentation

Features

• 6 simple API Methods for interacting with application state:

  • /stategate.StateService/Set sets an application state value(k/v pairs) adds it to the event log, then broadcast the event to all interested consumers(Stream method)
  • /stategate.StateService/Get gets an application state value(k/v pairs)
  • /stategate.StateService/Del deletes an application state value(k/v pairs)
  • /stategate.StateService/Search queries application state of a specific type/domain
  • /stategate.EventService/Stream creates an event stream/subscription to a given state type/domain
  • /stategate.EventService/Search queries historical events

• Capture all changes to an application's state as a sequence of events.

• Stateless & horizontally scaleable

• Native gRPC support

  • protobuf schema

• Embedded REST support / (transcoding)

  • open api schema

• Embedded grpcweb support (transcoding)

• Metrics Server(prometheus/pprof)

• Authentication - JWT/OAuth with remote JWKS verification

• Authorization - Rego based Authorization engine

• Autogenerated Client gRPC SDK's

  • Go
    • 
  • Node
  • PHP
  • C#
  • Java
  • gRPC Web
  • Python
  • Ruby

• Structured JSON Logs

• Sample Kubernetes Manifest

• Sample Docker Compose

• Pluggable "Channel" Providers

  • In-Memory(won't scale horizontally)
    • fully-tested
  • Nats
    • fully-tested
  • Nats Streaming(Stan)
  • Redis
    • fully-tested
  • Kafka
  • RabbitMQ

• Pluggable "Storage" Providers

  • MongoDb
    • fully-tested
  • PostgreSQL
  • MySQL
  • Cassandra

Goals

• Create a simple API interface for storing state and subscribing to state changes(events) using pluggable channel & storage providers
• Capture all changes to an application's state as a sequence of events.
• Safe to swap backend providers without changing client-side code
• Type-safe client's generated in many languages
• Safe to expose to the public internet due to fine-grained authentication/authorization model.
• Different combinations of Channel & Storage Providers are interoperable.
• Capture a persistant, immutable historical record of all state changes using a pluggable storage provider
• Store identity(jwt.claims) & timestamp in event logs to capture who is changing what & when
• Easy deployment model - fully configureable via environmental variables

Concepts

• Storage Provider: A stategate storage provider is a pluggable, 3rd party database storage service.

  • Storage providers provide persistance for all current state/events and should be scaled independently of stategate instances.

• Channel Provider: A stategate channel provider is a pluggable, 3rd party message-queue/channel service.

  • Channel providers provide a way for stategate to broadcast events to itself while scaling horizontally.
  • Channel providers should be scaled independently of stategate instances.

• State: state represents a single record(k/v pairs) with a unique key with a given type, belonging to a particular domain

  • Services/Users should use state related methods to persist & interact with the current state of an application/domain.

• Event: an event represents a state change- events are persisted and then emitted to consumers anytime state is created/updated.

  • Events are immutable after creation and may be searched.
  • Event Consumers may search events to query the previous state(s) of an state

Environmental Variables

# port to serve on (optional). defaults to 8080
STATEGATE_PORT=8080
# enable debug logs (optional)
STATEGATE_DEBUG=true
# disable all authentication & authorization(jwks, request policies, response policies) (optional)
STATEGATE_AUTH_DISABLED=false
# tls cert file (optional)
STATEGATE_TLS_CERT_FILE=/tmp/certs/stategate.cert
# tls key file (optional)
STATEGATE_TLS_KEY_FILE=/tmp/certs/stategate.key

# JSON Web Key Set remote URI used for fetching jwt signing keys for verification/validation (optional)
STATEGATE_JWKS_URI=https://www.googleapis.com/oauth2/v3/certs

# base64 encoded OPA rego policy executed on inbound requests from clients (optional)
STATEGATE_REQUEST_POLICY=cGFja2FnZSBzdGF0ZWdhdGUuYXV0aHoKCmRlZmF1bHQgYWxsb3cgPSB0cnVl
# base64 encoded OPA rego policy executed on responses sent to clients (optional)
STATEGATE_RESPONSE_POLICY=cGFja2FnZSBzdGF0ZWdhdGUuYXV0aHoKCmRlZmF1bHQgYWxsb3cgPSB0cnVl
# channel provider configuration(JSON) options: [inmem, redis, nats, stan, kafka] REQUIRED
STATEGATE_CHANNEL_PROVIDER={ "name": "redis", "addr": "localhost:6379" }
# STATEGATE_CHANNEL_PROVIDER={ "name": "nats", "addr": "localhost:4222" }
# STATEGATE_CHANNEL_PROVIDER={ "name": "stan", "addr": "localhost:4222" }
# STATEGATE_CHANNEL_PROVIDER={ "name": "inmem" }

# storage provider configuration(JSON) options: [mongo] REQUIRED
STATEGATE_STORAGE_PROVIDER={ "name": "mongo", "database": "testing", "addr": "mongodb://localhost:27017/testing" }

Notes