libswarm

README

Libswarm

libswarm is a toolkit for composing network services.

It defines a standard interface for services in a distributed system to communicate with each other. This lets you:

1. Compose complex architectures from reusable building blocks
2. Avoid vendor lock-in by swapping any service out with another

An extensive library of services is included, and you can also write your own using a simple API.

Here are some examples of what you can do with libswarm:

• Aggregate all your Docker containers across multiple hosts and infrastructure providers, as if they were running on a single host.

• Bridge your in-house DNS-based service discovery with that new shiny Consul deployment, without getting locked into either.

• Swap in a new clustering and service discovery system, without changing any application code.

• Collect logs across an in-house Mesos cluster, a Cloud Foundry deployment and individual servers staggered in 3 different datacenters, forward them to your legacy syslog deployment, then perform custom analytics on them.

• Simulate your entire service topology in a single process, then scale it out simply by re-arranging adapters.

• Organize your application as loosely coupled services from day 1, without over-engineering.

Installation

You can install libswarm in a few simple steps. You will need to make Go installed on your host.

1. Download the current source code.

go get github.com/docker/libswarm

1. Make sure $GOPATH/bin is in your $PATH

export PATH=$GOPATH/bin:$PATH

1. Download or update the current dependencies.

cd $GOPATH/src/github.com/docker/libswarm
make deps

NOTE: You may also need bzr. On OS X, you can install it from here or via Home Brew: brew install bzr. On Debian & Ubuntu, apt-get install bzr and on Red Hat et al, yum install bzr.

1. Then compile swarmd with:

go install github.com/docker/libswarm/swarmd

Running libswarm

If $GOPATH/bin is in your PATH, you can invoke swarmd from the command line.

$ swarmd -h
NAME:
   swarmd - Compose distributed systems from lightweight services

USAGE:
   swarmd [global options] command [command options] [arguments...]

VERSION:
   0.0.1

COMMANDS:
   help, h	Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --version, -v	print the version
   --help, -h		show help

Built-in services

Docker server

Maintainer: Ben Firshman

This service runs a Docker remote API server, allowing the Docker client and other Docker tools to control libswarm services. With no arguments, it listens on port 4243, but you can specify any port you like using tcp://0.0.0.0:9999, unix:///tmp/docker etc.

Docker client

Maintainer: Aanand Prasad

This service can be used to control a Docker Engine from libswarm services. It takes one argument, the Docker host to connect to. For example: dockerclient tcp://10.1.2.3:4243

SSH tunnel

Help wanted!

Etcd

Help wanted!

SkyDNS

Help wanted!

SkyDNS2

Help wanted!

Geard

Clayton Coleman

Fork-exec

Solomon Hykes

Mesos

Help wanted!

Shipyard

Brian Goff

Fleet

Help wanted!

Google Compute

Brendan Burns

Rackspace Cloud

John Hopper

Orchard

Maintainer: Aanand Prasad

Control an Orchard host from libswarm. It takes two arguments, an Orchard API token and the name of the Orchard host to control.

Amazon EC2

Aaron Feng

README

Consul

Help wanted!

OpenStack Nova

Help wanted!

Digital Ocean

Help wanted!

SoftLayer

Help wanted!

ZeroRPC

Help wanted!

Debug

The debug service simply catches all messages and prints them on the terminal for inspection.

Testing libswarm with swarmd

The libswarm library ships with a simple daemon which can control services in your distributed system.

Usage examples

Run swarmd without arguments to list available services:

swarmd

Pass a service name as argument to load it:

swarmd fakeclient

You can pass arguments to the service, like a shell command:

swarmd 'dockerserver tcp://localhost:4243'

You can call multiple services. They will be executed in parallel, with the output of each backend connected to the input of the next, just like unix pipelines.

This allows for very powerful composition.

swarmd 'dockerserver tcp://localhost:4243' 'debug' 'dockerclient unix:///var/run/docker.sock'

Creators

Solomon Hykes

• http://twitter.com/solomonstre
• http://github.com/shykes

Copyright and license

Code and documentation copyright 2013-2014 Docker, inc. Code released under the Apache 2.0 license. Docs released under Creative commons.

Documentation

Index

• Constants
• Variables
• func Copy(dst Sender, src Receiver) (int, error)
• func Pipe() (*receiverWrapper, *senderWrapper)
• type Client
  • func AsClient(dst Sender) *Client
  • func (c *Client) Attach(name string) (in Receiver, out *Client, err error)
  • func (c *Client) Connect() (io.ReadWriteCloser, error)
  • func (c *Client) Error(msg string, args ...interface{}) error
  • func (c *Client) Get() (string, error)
  • func (c *Client) Log(msg string, args ...interface{}) error
  • func (c *Client) Ls() ([]string, error)
  • func (c *Client) Set(vals ...string) error
  • func (c *Client) SetJson(val interface{}) error
  • func (c *Client) Spawn(cmd ...string) (out *Client, err error)
  • func (c *Client) Start() error
  • func (c *Client) Stop() error
  • func (c *Client) Watch() (Receiver, error)
• type Handler
  • func (h Handler) Close() error
  • func (h Handler) Send(msg *Message) (Receiver, error)
• type Message
• type NopReceiver
  • func (r NopReceiver) Receive(mode int) (*Message, error)
• type NopSender
  • func (s NopSender) Close() error
  • func (s NopSender) Send(msg *Message) (Receiver, error)
• type Receiver
  • func WrapReceiver(r libchan.Receiver) Receiver
• type Sender
  • func Repeater(payload *Message) Sender
  • func WrapSender(s libchan.Sender) Sender
• type Server
  • func NewServer() *Server
  • func (s *Server) Catchall(h Sender) *Server
  • func (s *Server) Close() error
  • func (s *Server) OnAttach(h func(name string, ret Sender) error) *Server
  • func (s *Server) OnError(h func(...string) error) *Server
  • func (s *Server) OnGet(h func() (string, error)) *Server
  • func (s *Server) OnLog(h func(...string) error) *Server
  • func (s *Server) OnLs(h func() ([]string, error)) *Server
  • func (s *Server) OnSpawn(h func(cmd ...string) (Sender, error)) *Server
  • func (s *Server) OnStart(h func() error) *Server
  • func (s *Server) OnStop(h func() error) *Server
  • func (s *Server) OnVerb(v Verb, h Sender) *Server
  • func (s *Server) Send(msg *Message) (Receiver, error)
• type Verb
  • func VerbFromString(s string) (Verb, error)
  • func (v Verb) String() string

Constants

const (
	Ret int = 1 << iota
)

Variables

var NotImplemented = Repeater(notImplementedMsg)

var RetPipe = retPipe{}

Functions

func Copy

func Copy(dst Sender, src Receiver) (int, error)

func Pipe

func Pipe() (*receiverWrapper, *senderWrapper)

Types

type Client

type Client struct {
	Sender
}

func AsClient

func AsClient(dst Sender) *Client

func (*Client) Attach

func (c *Client) Attach(name string) (in Receiver, out *Client, err error)

func (*Client) Connect

func (c *Client) Connect() (io.ReadWriteCloser, error)

func (*Client) Error

func (c *Client) Error(msg string, args ...interface{}) error

func (*Client) Get

func (c *Client) Get() (string, error)

func (*Client) Log

func (c *Client) Log(msg string, args ...interface{}) error

func (*Client) Ls

func (c *Client) Ls() ([]string, error)

func (*Client) Set

func (c *Client) Set(vals ...string) error

func (*Client) SetJson

func (c *Client) SetJson(val interface{}) error

func (*Client) Spawn

func (c *Client) Spawn(cmd ...string) (out *Client, err error)

func (*Client) Start

func (c *Client) Start() error

func (*Client) Stop

func (c *Client) Stop() error

func (*Client) Watch

func (c *Client) Watch() (Receiver, error)

type Handler

type Handler func(msg *Message) error

func (Handler) Close

func (h Handler) Close() error

func (Handler) Send

func (h Handler) Send(msg *Message) (Receiver, error)

type Message

type Message struct {
	Verb
	Args []string
	Ret  Sender
	Att  io.ReadWriteCloser
}

type NopReceiver

type NopReceiver struct{}

func (NopReceiver) Receive

func (r NopReceiver) Receive(mode int) (*Message, error)

type NopSender

type NopSender struct{}

func (NopSender) Close

func (s NopSender) Close() error

func (NopSender) Send

func (s NopSender) Send(msg *Message) (Receiver, error)

type Receiver

type Receiver interface {
	Receive(mode int) (*Message, error)
}

func WrapReceiver

func WrapReceiver(r libchan.Receiver) Receiver

type Sender

type Sender interface {
	Send(msg *Message) (Receiver, error)
	Close() error
}

func Repeater

func Repeater(payload *Message) Sender

func WrapSender

func WrapSender(s libchan.Sender) Sender

type Server

type Server struct {
	// contains filtered or unexported fields
}

func NewServer

func NewServer() *Server

func (*Server) Catchall

func (s *Server) Catchall(h Sender) *Server

func (*Server) Close

func (s *Server) Close() error

func (*Server) OnAttach

func (s *Server) OnAttach(h func(name string, ret Sender) error) *Server

func (*Server) OnError

func (s *Server) OnError(h func(...string) error) *Server

func (*Server) OnGet

func (s *Server) OnGet(h func() (string, error)) *Server

func (*Server) OnLog

func (s *Server) OnLog(h func(...string) error) *Server

func (*Server) OnLs

func (s *Server) OnLs(h func() ([]string, error)) *Server

func (*Server) OnSpawn

func (s *Server) OnSpawn(h func(cmd ...string) (Sender, error)) *Server

func (*Server) OnStart

func (s *Server) OnStart(h func() error) *Server

func (*Server) OnStop

func (s *Server) OnStop(h func() error) *Server

func (*Server) OnVerb

func (s *Server) OnVerb(v Verb, h Sender) *Server

func (*Server) Send

func (s *Server) Send(msg *Message) (Receiver, error)

type Verb

type Verb uint32

const (
	Ack Verb = iota
	Attach
	Connect
	Error
	File
	Get
	Log
	Ls
	Set
	Spawn
	Start
	Stop
	Watch
)

func VerbFromString

func VerbFromString(s string) (Verb, error)

func (Verb) String

func (v Verb) String() string