pillow

README

Pillow

Remove the boilerplate in your embedded NATS projects, and deploy to cloud platforms with added functionality!

[!WARNING] The API is unstable and subject to change until v1.0.0 is reached.

Getting Started

For simply embedding in a Go program:

go get github.com/Nintron27/pillow

then follow the embedded example.

For more examples:

• Reference the examples folder for examples of using pillow.
• This project was started and is being maintained to be used in the Stelo Finance project, so check that out for a real project example.

Features

• Removes boilerplate when using NATS embedded in Go, and implements sane defaults.
• Platform Adapters that automatically configure clustering, superclustering, and leaf nodes.
• Your needed feature here? Leave a feature request issue!

Platform Adapters Overview

The goal of Platform Adapters is to automatically configure your embedded nats server for certain network topologies when deployed on cloud platforms.

For example, the FlyioClustering Platform Adapter, enabled as such:

ns, err := pillow.Run(
  pillow.WithPlatformAdapter(context.TODO(), os.Getenv("ENV") == "prod", &pillow.FlyioClustering{
    ClusterName: "pillow-cluster", // Required, supply whatever value floats your boat
  }),
)

will use Flyio's Machine Runtime Environment and .internal DNS to configure your embedded server to:

1. Cluster with other machines in the same process group in the same region.
2. Form a supercluster with any other regions your process group may be deployed to.
3. Uniquely name your servers and append -<REGION> to your clusters' names.

Flyio

Currently there are only Platform Adapters for Flyio, and the two existing Adapters are:

• FlyioClustering: Auto clustering in region, supercluster regions together. (diagram)
• FlyioHubAndSpoke: Auto cluster primary region, and leaf node other regions' machines to your hub. (diagram)

Quirks w/ Platform Adapters

• You should supply pillow.WithPlatformAdapter last in your pillow.Run() call as it will override certain nats server options for platform specific reasons, such as overridding the server name so they are unique on every machine.
• FlyioClustering: Removing a region will cause any remaining machines will infinitely try to reconnect to the removed region, until they are restarted. This is probably fine, as it's just some network calls, but it is something to be aware of.
• FlyioClustering: When JetStream is enabled all your regions must have >= 3 nodes, as JetStream clustering requires this for a quorum.

[!CAUTION] Take great caution when switching adapters (such as HubAndSpoke to Clustering) as the cluster names and JS domains will change. Also caution with scaling down JS regions, as removing a node that contains a stream or kv bucket can cause an outage. You should evict them as a peer before removing the machine.

Forced Opinions

• NoSigs is forced to true for the nats-server configuration, as it generally doesn't align with embedding NATS in Go, but also causes problems with the Shutdown function due to nats-io/nats-server#6358.

Credit to @whaaaley for project name and icon, and @delaneyj for the inspiration.

Documentation

Index

• Constants
• Variables
• type FlyioClustering
  • func (c *FlyioClustering) Configure(ctx context.Context) Option
• type FlyioHubAndSpoke
  • func (c *FlyioHubAndSpoke) Configure(ctx context.Context) Option
• type Option
  • func WithLogging(enable bool) Option
  • func WithNATSServerOptions(natsServerOpts *server.Options) Option
  • func WithPlatformAdapter(ctx context.Context, enable bool, platformCfgr PlatformConfigurator) Option
  • func WithSystemUser(enable bool, username, password string) Option
  • func WithTimeout(t time.Duration) Option
  • func WithoutInProcessClient(enable bool) Option
• type PlatformConfigurator
• type Server
  • func Run(opts ...Option) (*Server, error)
  • func (s *Server) NATSClient() (*nats.Conn, error)
  • func (s *Server) Shutdown(ctx context.Context) error

Constants

const (
	ClusterPort  int = 4244
	GatewayPort  int = 7244
	LeafNodePort int = 7422
)

Variables

var ErrEnvVarNotFound = errors.New("Platform ENV not found")

A platform specific environment variable could not be found. This should only occur if you are trying to use a platform adapter like AdapterFlyio() while not actually deploying on Flyio.

If this error does occur when still deploying on the correct platform respective to your Adapter, this may be that the platform changed their environment implementation. If so, file an issue.

var ErrPlatformImplementationChanged = errors.New("Platform implementation changed")

Should only occur when a platform changes something in their implementation. Pillow makes assumeptions that platforms tooling will not change, but if it does this error is returned.

var ErrStartupTimedOut = errors.New("embedded nats server startup timed out")

ErrStartupTimedOut indicates that the embedded nats server exceeded its startup timeout duration.

Types

type FlyioClustering

type FlyioClustering struct {
	// The name to be used for clustering. Appended will be `-<REGION>` where REGION is the
	// Flyio region this cluster is in.
	ClusterName string
}

FlyioClustering will cluster all intra-region machines, and super cluster regions together. Note, if JetStream is enabled ALL regions must have >= 3 machines.

func (*FlyioClustering) Configure

func (c *FlyioClustering) Configure(ctx context.Context) Option

type FlyioHubAndSpoke

type FlyioHubAndSpoke struct {
	// The name to be used for the primary region cluster.
	ClusterName string

	// Disable clustering in the primary region
	//
	// This should only be enabled when the primary region has 1 node and you want to enable JS.
	// Changing to a clustered deployment later may result in losing any data in JS.
	DisableClustering bool
}

FlyioHubAndSpoke will cluster all machines in your primary region, and then all other regions will have their machines individually connect to your primary region cluster as leaf nodes.

Additionally, the primary region cluster will have the JS domain of `hub` and the leaf nodes will have the structure of `leaf-<REGION>-<MACHINE_ID>`

func (*FlyioHubAndSpoke) Configure

func (c *FlyioHubAndSpoke) Configure(ctx context.Context) Option

type Option

type Option func(*options) error

func WithLogging

func WithLogging(enable bool) Option

Enable NATS internal logging

func WithNATSServerOptions

func WithNATSServerOptions(natsServerOpts *server.Options) Option

Apply your own NATs Server Options. Note, this will override any existing options, so it's recommended to place first in the list of options in Run().

func WithPlatformAdapter

func WithPlatformAdapter(ctx context.Context, enable bool, platformCfgr PlatformConfigurator) Option

The supplied platformCfgr will configure your embedded nats server for a specific network topology when deployed on the corrisponding cloud platform.

For local development purposes it's recommended to disable this by checking for an environment variable, for example:

pillow.WithPlatformAdapter(context.TODO(), os.Getenv("ENV") == "prod", &pillow.FlyioClustering{
  ClusterName: "pillow-cluster",
})

func WithSystemUser

func WithSystemUser(enable bool, username, password string) Option

Duration to wait for embedded NATS to start. Defaults to 5 seconds if omitted

func WithTimeout

func WithTimeout(t time.Duration) Option

Duration to wait for embedded NATS to start. Defaults to 5 seconds if omitted

func WithoutInProcessClient

func WithoutInProcessClient(enable bool) Option

If enabled the clients returned from *Server.NATSClient() will communicate with the embedded server over the network instead of in-process.

type PlatformConfigurator

type PlatformConfigurator interface {
	Configure(context.Context) Option
}

type Server

type Server struct {
	// Reference to the underlying nats-server server.Server
	NATSServer *server.Server
	// contains filtered or unexported fields
}

func Run

func Run(opts ...Option) (*Server, error)

Applies all passed options and starts the NATS server, returning a server struct and error if there was a problem starting the server.

All configuration functions start with "With". Example WithLogging(true)

func (*Server) NATSClient

func (s *Server) NATSClient() (*nats.Conn, error)

Returns a new nats client.

Client will communicate in-process unless the embedded server was started with WithoutInProcessClient(true)

func (*Server) Shutdown

func (s *Server) Shutdown(ctx context.Context) error