mp

README

MessagePasser

MessagePasser is an implementation of a cross-machine messaging protocol. Currently, it's in its super-early stages of development, so not much exists (sans a Client+Server implementation in Go). At the time of writing, it's version 0.01a. I'm using it a bit "in production", but it's still very much in development, and the user-facing API isn't set in stone yet.

Overview

The goal of MessagePasser (MP) is to allow two processes to talk to each other, plain and simple. MP takes a client-server model, so it doesn't matter if the two processes are both on localhost, nor does it matter if they're both behind NATs. So long as the server is accessible, they can talk.

The base unit of communication in MP is the Connection. This is a bidirectional channel between two parties, which either party may use to send bytes to the other. Connections are managed by Clients, who all talk to the Server, which is just a glorified router (with user authentication built in).

Authentication to the server is super simple -- it looks something like this:

1. Client sends name + secret (or username and password, if you'd prefer), to the Server
2. If the Server likes this name + secret, it acknowledges that everything's good. Otherwise, it closes the connection.
3. Client can now talk to other clients.

In order to start talking to another Client (establish a Connection), you need to have the other Client's name and the type of Connection you want to establish (protocol name). Both of these are strings. If you can provide both of those things, you have all you need to set up a Connection.

Quick Start

Server

It's highly recommended that you just use the default server in examples/server.go. Simply run it with go run server.go [port-number]. If you do use it, PLEASE change the authentication function. ANYONE is allowed in using the default one. This is probably not what you want.

Important parts of the server:

• Auth functions take a string and a secret, and they return whether or not that pair of credentials is valid.
• Translators are interfaces we use to put a *Message on to the wire. So yes, it's entirely possible to have your Server speak JSON with a few extra lines of code.

Client

A simple setup with two clients (i.e. a friendly echo server) is located in examples/client.go. It details how to make new clients, connections, etc.

Points of note for the clients:

• See note on Translators above for the Server section
• NewConnectionHandlers are how we determine how to handle an incoming connection request, based on the protocol that the new connection has specified.

Points of note for connections:

• When one side closes, the other side is not (yet) notified to close. You need to write to get an "unreachable" notification -- this will be fixed very soon.
• Read/Write are synonymous (sans signatures) to ReadMessage/WriteMessage.

Docs

Immediate TODOs

• Finish up docs
• Adding support for multiple message formats (JSON, Gob, Protobuf, etc.) talking to the same server at once.
• Testing testing testing testing testing

Future Plans

• Client implementations in other languages (Java, Python, JS, ...)

I found a bug!

You mean you found my latest feature! Bug reports/pull requests are welcome :)

Documentation

Overview

Package mp is a library used to pass messages between multiple processes. MessagePasser enforces no format on the messages passed between the clients; its only goal is to get the message from point A to point B.

Basics

The basis of MessagePasser is the Connection: a two-way channel of communication between two processes. Connections are issued through Clients, which are all linked to the same Server.

Clients are identified by their name -- examples of valid names range from "desktop" to "net.gbiv.AbstractNetworkAdaptorFactoryImpl"; anything is fair game (so long as it's not blank and doesn't contain a ':').

Connections are created in one of two ways:

• A user calls Client.MakeConnection(proto, otherClient string)
• A remote user requested that a new Connection be made with the current client using Client.MakeConnection.

In the former case, you need to pass in the identifier for another client that is currently connected (otherClient) and a protocol string -- the protocol string can be any string; it's entirely user-defined and is interpreted entirely by user code.

Assuming the protocol is recognized by `otherClient`, both the client initiating the Connection and otherClient will then have Connections through which they can talk with each other. The lifetime of a Connection is bound to the lifetime of the Client that you got it from, and the lifetime of the other Connection that it's tied to. (Naturally, you can Close() one side of a Connection without issue)

One of the goals of this project is to be language independent; all of the encoding/decoding of Messages is done using MessageTranslator implementations, TODO: (Finish this after completing multiprotocol support in the Server).

Examples

See the examples/ subdirectory for examples.

Index

• Constants
• type Authenticator
• type Client
  • func NewClient(name string, server io.ReadWriteCloser, tm TranslatorMaker, ...) *Client
  • func (c *Client) Authenticate(password []byte) error
  • func (c *Client) Close() error
  • func (c *Client) MakeConnection(otherClient, proto string) (Connection, error)
  • func (c *Client) Run() error
• type Connection
• type MappedConnectionHandler
  • func NewMappedConnectionHandler() *MappedConnectionHandler
  • func (m *MappedConnectionHandler) AddMapping(str string, fn func(Connection))
  • func (m *MappedConnectionHandler) IncomingConnection(proto string, accept func() Connection)
• type Message
• type MessageTranslator
  • func NewGobTranslator(r io.Reader, w io.Writer) MessageTranslator
  • func NewJSONTranslator(r io.Reader, w io.Writer) MessageTranslator
• type MetaType
• type NewConnectionHandler
• type Server
  • func NewServer(auth Authenticator, maker TranslatorMaker) *Server
  • func (s *Server) Close()
  • func (s *Server) Listen(l net.Listener) error
• type TranslatorMaker

Constants

const (
	MetaNone = MetaType(iota)
	MetaWAT  // Invalid/unknown meta type received
	MetaNoSuchConnection
	MetaUnknownProto
	MetaClientClosed
	MetaConnSyn
	MetaConnAck
	MetaConnClosed
	MetaAuth
	MetaAuthOk
	MetaAuthFailure
)

Constants to represent each Meta type.

Types

type Authenticator

type Authenticator func(name string, secret []byte) bool

Authenticator is a function that returns whether or not the given name/secret pair is valid, and that the sender of it should be allowed to connect to the server.

`secret` may be nil.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the entity used to connect to Servers. Using a Client, you can send/receive new Connection requests.

func NewClient

func NewClient(
	name string,
	server io.ReadWriteCloser,
	tm TranslatorMaker,
	ch NewConnectionHandler) *Client

NewClient creates a new Client instance with the given name + server communications

func (*Client) Authenticate

func (c *Client) Authenticate(password []byte) error

Authenticate allows the client to perform its initial handshake and authenticate itself with a Server. This should be called before Client.Run().

func (*Client) Close

func (c *Client) Close() error

Close closes a Client's connection to the server, makes Client.Run() exit (eventually), and closes all Connection instances that this Client created.

func (*Client) MakeConnection

func (c *Client) MakeConnection(otherClient, proto string) (Connection, error)

MakeConnection attempts to make a Connection with `otherClient` using the `proto` protocol. On success, you'll get a shiny new Connection instance and nil error. If anything goes wrong, you'll get an error.

The Client will need to be authenticated and running when you call MakeConnection().

func (*Client) Run

func (c *Client) Run() error

Run is the main "event loop" for a Client. In it, the Client receives messages from the Server and dispatches them to Connections. You should call Authenticate() prior to calling Run().

type Connection

type Connection interface {
	// Note that Read([]byte) *is allowed* to concatenate messages
	// and, as a result, may drop messages with empty data. If you
	// expect to be able to receive messages with empty data, please
	// use ReadMessage.
	//
	// Similarly, Write() is allowed to merge requests into larger
	// Messages in order to increase efficiency and such.
	// Currently, it doesn't do that, but it's allowed to
	io.ReadWriteCloser

	// Reads the contents of a single message sent to us, blocking if necessary.
	// This may return a nil []byte if no Data was sent with the message.
	// If we returned a message, error is nil.
	ReadMessage() ([]byte, error)

	// Writes a message, guaranteeing that the []byte given is the full body
	// of the message. Assuming a reliable transport between Connections,
	// for every WriteMessage you do on one side, there will be a corresponding
	// ReadMessage you may do on the other.
	WriteMessage([]byte) error

	// Gets the name of the client that our other Connection resides on.
	OtherClient() string
}

Connection is a two-way message-passing connection. When you establish a Connection with a client, a Connection instance is given to a client on each side, so both sides may communicate with each other.

Guarantees we make you:

• Messages are guaranteed to be sent in the order you Write them. So, as long as the underlying Translator implementations deliver messages in order, messages will be delivered in order.
• One connection calling Close()

Misc:

• If you're blocked in Write() or WriteMessage() and call Close(), then Write()/WriteMessage() will not necessarily exit immediately.
• When Write/WriteMessage returns, we know nothing about whether or not the message reached the linked Connection instance. We just know that the message hit the server without issue; nothing more.

type MappedConnectionHandler

type MappedConnectionHandler struct {
	// contains filtered or unexported fields
}

MappedConnectionHandler is a NewConnectionHandler that keeps an internal mapping of protocol -> function. When IncomingConnection(proto, conn) is called, this ConnectionHandler will see if the user supplied a function for the given proto. If so, it'll fire off the function mapped to proto in a goroutine. Otherwise, it will refuse the connection attempt.

Usage:

h := NewMappedConnectionHandler()
h.AddMapping("echo", func(c Connection) {
	defer c.Close()
	for {
		msg, err := c.ReadMessage()
		if err != nil {
			log.Println(err)
			return
		}
		err = c.WriteMessage(msg)
		if err != nil {
			log.Println(err)
			return
		}

	}
})

h.AddMapping("ping in 1 second", func(c Connection) {
	defer c.Close()
	time.Sleep(time.Second)
	c.WriteMessage([]byte("Ping!"))
})

Please note that the func you provide gains ownership of the Connection that is passed to it. So it's your responsibility to close the Connection when you're done using it.

func NewMappedConnectionHandler

func NewMappedConnectionHandler() *MappedConnectionHandler

NewMappedConnectionHandler Creates and initializes a new instance of MappedConnectionHandler

func (*MappedConnectionHandler) AddMapping

func (m *MappedConnectionHandler) AddMapping(str string, fn func(Connection))

AddMapping maps a protocol name to a function to be run in a goroutine.

This is not safe to be called concurrently with IncomingConnection().

func (*MappedConnectionHandler) IncomingConnection

func (m *MappedConnectionHandler) IncomingConnection(proto string, accept func() Connection)

IncomingConnection allows us to implement the NewConnectionHandler interface

type Message

type Message struct {
	Meta         MetaType
	OtherClient  string
	ConnectionID string
	Data         []byte
}

Message is the type that Clients/Servers send over the wire. Clients/Servers should never expose raw Message instances to the user -- this is only exposed so you can roll your own MessageTranslator.

type MessageTranslator

type MessageTranslator interface {
	ReadMessage() (*Message, error)
	WriteMessage(*Message) error
}

MessageTranslator is a type that can read a message from a Reader and write a message to a Writer in some format. Instances of MessageTranslator should be able to handle ReadMessage and WriteMessage being called concurrently.

func NewGobTranslator

func NewGobTranslator(r io.Reader, w io.Writer) MessageTranslator

NewGobTranslator creates a new MessageTranslator that reads/write messages using the Gob message format.

func NewJSONTranslator

func NewJSONTranslator(r io.Reader, w io.Writer) MessageTranslator

NewJSONTranslator creates a new MessageTranslator that reads/write messages using the JSON message format.

type MetaType

type MetaType uint8

MetaType describes the intent of a message -- some messages are meant to simply be delivered to a Connection, while others are meant to setup connections, etc.

type NewConnectionHandler

type NewConnectionHandler interface {
	// IncomingConnection is called when another Client wants to make a
	// connection with the calling Client. `proto` is the kind of service
	// that the initiating Client wants access to, and `accept` is the function
	// you call to get the Connection. Note that accept is finicky -- it *must*
	// be called before IncomingConnection() terminates if you want to accept the
	// Connection. Additionally, you may only call it once per call to
	// IncomingConnection. If you violate either of these, accept() panics.
	IncomingConnection(proto string, accept func() Connection)
}

NewConnectionHandler is an interface that Clients call when a request for a new Connection comes in.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is an implementation of a MessagePassing Server. (Golint made me do this)

func NewServer

func NewServer(auth Authenticator, maker TranslatorMaker) *Server

NewServer creates a new Server instance that uses the given Authenticator and TranslatorMaker

func (*Server) Close

func (s *Server) Close()

Close closes and shuts down the server. It will cause Listen() on the current instance to exit, and will sever the server's connection to all Clients.

func (*Server) Listen

func (s *Server) Listen(l net.Listener) error

Listen listens for new Client connections to the Server. When Listen exits, the Server will continue to serve Clients that are already connected.

type TranslatorMaker

type TranslatorMaker func(io.Reader, io.Writer) MessageTranslator

TranslatorMaker is a function that, given a Reader and Writer, returns a MessageTranslator that reads from and writes to the given Reader and Writer.