beam

README

Beam

A library to break down an application into loosely coupled micro-services

Beam is a library to turn your application into a collection of loosely coupled micro-services. It implements an ultra-lightweight hub for the different components of an application to discover and consume each other, either in-memory or across the network.

Beam can be embedded with very little overhead by using Go channels. It also implements an efficient http2/tls transport which can be used to securely expose and consume any micro-service across a distributed system.

Because remote Beam sessions are regular HTTP2 over TLS sessions, they can be used in combination with any standard proxy or authentication middleware. This means Beam, when configured propely, can be safely exposed on the public Internet. It can also be embedded in an existing rest API using an http1 and websocket fallback.

How is it different from RPC or REST?

Modern micro-services are not a great fit for classical RPC or REST protocols because they often rely heavily on events, bi-directional communication, stream multiplexing, and some form of data synchronization. Sometimes these services have a component which requires raw socket access, either for performance (file transfer, event firehose, database access) or simply because they have their own protocol (dns, smtp, sql, ssh, zeromq, etc). These components typically need a separate set of tools because they are outside the scope of the REST and RPC tools. If there is also a websocket or ServerEvents transport, those require yet another layer of tools.

Instead of a clunky patchwork of tools, Beam implements in a single minimalistic library all the primitives needed by modern micro-services:

• Request/response with arbitrary structured data

• Asynchronous events flowing in real-time in both directions

• Requests and responses can flow in any direction, and can be arbitrarily nested, for example to implement a self-registering worker model

• Any request or response can include any number of streams, multiplexed in both directions on the same session.

• Any message serialization format can be plugged in: json, msgpack, xml, protobuf.

• As an optional convenience a minimalist key-value format is implemented. It is designed to be extremely fast to serialize and parse, dead-simple to implement, and suitable for both one-time data copy, file storage, and real-time synchronization.

• Raw file descriptors can be "attached" to any message, and passed under the hood using the best method available to each transport. The Go channel transport just passes os.File pointers around. The unix socket transport uses fd passing which makes it suitable for high-performance IPC. The tcp transport uses dedicated http2 streams. And as a bonus extension, a built-in tcp gateway can be used to proxy raw network sockets without extra overhead. That means Beam services can be used as smart gateways to a sql database, ssh or file transfer service, with unified auth, discovery and tooling and without performance penalty.

Design philosophy

An explicit goal of Beam is simplicity of implementation and clarity of spec. Porting it to any language should be as effortless as humanly possible.

Creators

Solomon Hykes

• http://twitter.com/solomonstre
• http://github.com/shykes

Copyright and license

Code and documentation copyright 2013-2014 Docker, inc. Code released under the Apache 2.0 license. Docs released under Creative commons.

Documentation

Index

• Constants
• Variables
• func Copy(dst Sender, src Receiver) (int, error)
• func Pipe() (*PipeReceiver, *PipeSender)
• type Handler
  • func (h Handler) Close() error
  • func (h Handler) Send(msg *Message) (Receiver, error)
• type Message
• type NopReceiver
  • func (r NopReceiver) Receive(mode int) (*Message, error)
• type NopSender
  • func (s NopSender) Close() error
  • func (s NopSender) Send(msg *Message) (Receiver, error)
• type Object
  • func Obj(dst Sender) *Object
  • func (o *Object) Attach(name string) (in Receiver, out *Object, err error)
  • func (o *Object) Connect() (net.Conn, error)
  • func (o *Object) Error(msg string, args ...interface{}) error
  • func (o *Object) Get() (string, error)
  • func (o *Object) Log(msg string, args ...interface{}) error
  • func (o *Object) Ls() ([]string, error)
  • func (o *Object) Set(vals ...string) error
  • func (o *Object) SetJson(val interface{}) error
  • func (o *Object) Spawn(cmd ...string) (out *Object, err error)
  • func (o *Object) Start() error
  • func (o *Object) Stop() error
  • func (o *Object) Watch() (Receiver, error)
• type PipeReceiver
  • func (r *PipeReceiver) Close() error
  • func (r *PipeReceiver) CloseWithError(err error) error
  • func (r *PipeReceiver) Receive(mode int) (*Message, error)
  • func (r *PipeReceiver) SendTo(dst Sender) (int, error)
• type PipeSender
  • func (w *PipeSender) Close() error
  • func (w *PipeSender) CloseWithError(err error) error
  • func (w *PipeSender) ReceiveFrom(src Receiver) (int, error)
  • func (w *PipeSender) Send(msg *Message) (Receiver, error)
• type Receiver
• type ReceiverFrom
• type Sender
  • func Repeater(payload *Message) Sender
  • func Task(f func(in Receiver, out Sender)) Sender
• type SenderTo
• type Server
  • func NewServer() *Server
  • func (s *Server) Catchall(h Sender) *Server
  • func (s *Server) Close() error
  • func (s *Server) OnAttach(h Sender) *Server
  • func (s *Server) OnError(h Sender) *Server
  • func (s *Server) OnGet(h Sender) *Server
  • func (s *Server) OnLog(h Sender) *Server
  • func (s *Server) OnLs(h Sender) *Server
  • func (s *Server) OnSpawn(h Sender) *Server
  • func (s *Server) OnStart(h Sender) *Server
  • func (s *Server) OnStop(h Sender) *Server
  • func (s *Server) OnVerb(v Verb, h Sender) *Server
  • func (s *Server) Send(msg *Message) (Receiver, error)
• type Tree
  • func NewTree() *Tree
  • func (t *Tree) Bind(name string, dst Sender) *Tree
• type Verb
  • func (v Verb) String() string

Constants

const (
	Ret int = 1 << iota
)

Variables

var (
	ErrIncompatibleSender   = errors.New("incompatible sender")
	ErrIncompatibleReceiver = errors.New("incompatible receiver")
)

var NotImplemented = Repeater(&Message{Verb: Error, Args: []string{"not implemented"}})

var RetPipe = retPipe{}

Functions

func Copy

func Copy(dst Sender, src Receiver) (int, error)

func Pipe

func Pipe() (*PipeReceiver, *PipeSender)

Types

type Handler

type Handler func(msg *Message) error

func (Handler) Close

func (h Handler) Close() error

func (Handler) Send

func (h Handler) Send(msg *Message) (Receiver, error)

type Message

type Message struct {
	Verb Verb
	Args []string
	Att  *os.File
	Ret  Sender
}

type NopReceiver

type NopReceiver struct{}

func (NopReceiver) Receive

func (r NopReceiver) Receive(mode int) (*Message, error)

type NopSender

type NopSender struct{}

func (NopSender) Close

func (s NopSender) Close() error

func (NopSender) Send

func (s NopSender) Send(msg *Message) (Receiver, error)

type Object

type Object struct {
	Sender
}

func Obj

func Obj(dst Sender) *Object

func (*Object) Attach

func (o *Object) Attach(name string) (in Receiver, out *Object, err error)

func (*Object) Connect

func (o *Object) Connect() (net.Conn, error)

func (*Object) Error

func (o *Object) Error(msg string, args ...interface{}) error

func (*Object) Get

func (o *Object) Get() (string, error)

func (*Object) Log

func (o *Object) Log(msg string, args ...interface{}) error

func (*Object) Ls

func (o *Object) Ls() ([]string, error)

func (*Object) Set

func (o *Object) Set(vals ...string) error

func (*Object) SetJson

func (o *Object) SetJson(val interface{}) error

func (*Object) Spawn

func (o *Object) Spawn(cmd ...string) (out *Object, err error)

func (*Object) Start

func (o *Object) Start() error

func (*Object) Stop

func (o *Object) Stop() error

func (*Object) Watch

func (o *Object) Watch() (Receiver, error)

type PipeReceiver

type PipeReceiver struct {
	// contains filtered or unexported fields
}

func (*PipeReceiver) Close

func (r *PipeReceiver) Close() error

func (*PipeReceiver) CloseWithError

func (r *PipeReceiver) CloseWithError(err error) error

func (*PipeReceiver) Receive

func (r *PipeReceiver) Receive(mode int) (*Message, error)

func (*PipeReceiver) SendTo

func (r *PipeReceiver) SendTo(dst Sender) (int, error)

type PipeSender

type PipeSender struct {
	// contains filtered or unexported fields
}

func (*PipeSender) Close

func (w *PipeSender) Close() error

func (*PipeSender) CloseWithError

func (w *PipeSender) CloseWithError(err error) error

func (*PipeSender) ReceiveFrom

func (w *PipeSender) ReceiveFrom(src Receiver) (int, error)

func (*PipeSender) Send

func (w *PipeSender) Send(msg *Message) (Receiver, error)

type Receiver

type Receiver interface {
	Receive(mode int) (*Message, error)
}

type ReceiverFrom

type ReceiverFrom interface {
	ReceiveFrom(Receiver) (int, error)
}

type Sender

type Sender interface {
	Send(msg *Message) (Receiver, error)
	Close() error
}

func Repeater

func Repeater(payload *Message) Sender

func Task

func Task(f func(in Receiver, out Sender)) Sender

type SenderTo

type SenderTo interface {
	SendTo(Sender) (int, error)
}

type Server

type Server struct {
	// contains filtered or unexported fields
}

func NewServer

func NewServer() *Server

func (*Server) Catchall

func (s *Server) Catchall(h Sender) *Server

func (*Server) Close

func (s *Server) Close() error

func (*Server) OnAttach

func (s *Server) OnAttach(h Sender) *Server

func (*Server) OnError

func (s *Server) OnError(h Sender) *Server

func (*Server) OnGet

func (s *Server) OnGet(h Sender) *Server

func (*Server) OnLog

func (s *Server) OnLog(h Sender) *Server

func (*Server) OnLs

func (s *Server) OnLs(h Sender) *Server

func (*Server) OnSpawn

func (s *Server) OnSpawn(h Sender) *Server

func (*Server) OnStart

func (s *Server) OnStart(h Sender) *Server

func (*Server) OnStop

func (s *Server) OnStop(h Sender) *Server

func (*Server) OnVerb

func (s *Server) OnVerb(v Verb, h Sender) *Server

func (*Server) Send

func (s *Server) Send(msg *Message) (Receiver, error)

type Tree

type Tree struct {
	*Server
	// contains filtered or unexported fields
}

func NewTree

func NewTree() *Tree

func (*Tree) Bind

func (t *Tree) Bind(name string, dst Sender) *Tree

type Verb

type Verb uint32

const (
	Ack Verb = iota
	Attach
	Connect
	Error
	File
	Get
	Log
	Ls
	Set
	Spawn
	Start
	Stop
	Watch
)

func (Verb) String

func (v Verb) String() string