evio

README

evio is an event loop networking framework that is fast and small. It makes direct epoll and kqueue syscalls rather than using the standard Go net package, and works in a similar manner as libuv and libevent.

The goal of this project is to create a server framework for Go that performs on par with Redis and Haproxy for packet handling. My hope is to use this as a foundation for Tile38 and a future L7 proxy for Go... and a bunch of other stuff.

Just to be perfectly clear

This project is not intended to be a general purpose replacement for the standard Go net package or goroutines. It's for building specialized services such as key value stores, L7 proxies, static websites, etc.

You would not want to use this framework if you need to handle long-running requests (milliseconds or more). For example, a web api that needs to connect to a mongo database, authenticate, and respond; just use the Go net/http package instead.

There are many popular event loop based applications in the wild such as Nginx, Haproxy, Redis, and Memcached. All of these are single-threaded and very fast and written in C.

The reason I wrote this framework is so I can build certain network services that perform like the C apps above, but I also want to continue to work in Go.

Features

• Fast single-threaded event loop
• Simple API
• Low memory usage
• Supports tcp, udp, and unix sockets
• Allows multiple network binding on the same event loop
• Flexible ticker event
• Fallback for non-epoll/kqueue operating systems by simulating events with the net package
• Ability to wake up connections from long running background operations
• Dial an outbound connection and process/proxy on the event loop
• SO_REUSEPORT socket option

Getting Started

Installing

To start using evio, install Go and run go get:

$ go get -u github.com/tidwall/evio

This will retrieve the library.

Usage

Starting a server is easy with evio. Just set up your events and pass them to the Serve function along with the binding address(es). Each connections receives an ID that's passed to various events to differentiate the clients. At any point you can close a client or shutdown the server by return a Close or Shutdown action from an event.

Example echo server that binds to port 5000:

package main

import "github.com/tidwall/evio"

func main() {
	var events evio.Events
	events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
		out = in
		return
	}
	if err := evio.Serve(events, "tcp://localhost:5000"); err != nil {
		panic(err.Error())
	}
}

Here the only event being used is Data, which fires when the server receives input data from a client. The exact same input data is then passed through the output return value, which is then sent back to the client.

Connect to the echo server:

$ telnet localhost 5000

Events

The event type has a bunch of handy events:

• Serving fires when the server is ready to accept new connections.
• Opened fires when a connection has opened.
• Closed fires when a connection has closed.
• Detach fires when a connection has been detached using the Detach return action.
• Data fires when the server receives new data from a connection.
• Prewrite fires prior to all write attempts from the server.
• Postwrite fires immediately after every write attempt.
• Tick fires immediately after the server starts and will fire again after a specified interval.

Multiple addresses

An server can bind to multiple addresses and share the same event loop.

evio.Serve(events, "tcp://192.168.0.10:5000", "unix://socket")

Ticker

The Tick event fires ticks at a specified interval. The first tick fires immediately after the Serving events.

events.Tick = func() (delay time.Duration, action Action){
	log.Printf("tick")
	delay = time.Second
	return
}

Wake up

A connection can be woken up using the Wake function that is made available through the Serving event. This is useful for when you need to offload an operation to a background goroutine and then later notify the event loop that it's time to send some data.

Example echo server that when encountering the line "exec" it waits 5 seconds before responding.

var srv evio.Server
var mu sync.Mutex
var execs = make(map[int]int)

events.Serving = func(srvin evio.Server) (action evio.Action) {
	srv = srvin // hang on to the server control, which has the Wake function
	return
}
events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
	if in == nil {
		// look for `in` param equal to `nil` following a wake call.
		mu.Lock()
		for execs[id] > 0 {
			out = append(out, "exec\r\n"...)
			execs[id]--
		}
		mu.Unlock()
	} else if string(in) == "exec\r\n" {
		go func(){
			// do some long running operation
			time.Sleep(time.Second*5)
			mu.Lock()
			execs[id]++
			mu.Unlock()
			srv.Wake(id)
		}()
	} else {
		out = in
	}
	return
}

Dial out

An outbound connection can be created by using the Dial function that is made available through the Serving event. Dialing a new connection will return a new connection ID and attach that connection to the event loop in the same manner as incoming connections. This operation is completely non-blocking including any DNS resolution.

All new outbound connection attempts will immediately fire an Opened event and end with a Closed event. A failed connection will send the connection error through the Closed event.

var srv evio.Server
var mu sync.Mutex
var execs = make(map[int]int)

events.Serving = func(srvin evio.Server) (action evio.Action) {
    srv = srvin // hang on to the server control, which has the Dial function
    return
}
events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
    if string(in) == "dial\r\n" {
        id := srv.Dial("tcp://google.com:80")
        // We now established an outbound connection to google.
        // Treat it like you would incoming connection.
    } else {
        out = in
    }
    return
}

Data translations

The Translate function wraps events and provides a ReadWriter that can be used to translate data off the wire from one format to another. This can be useful for transparently adding compression or encryption.

For example, let's say we need TLS support:

var events Events

// ... fill the events with happy functions

cer, err := tls.LoadX509KeyPair("certs/ssl-cert-snakeoil.pem", "certs/ssl-cert-snakeoil.key")
if err != nil {
	log.Fatal(err)
}
config := &tls.Config{Certificates: []tls.Certificate{cer}}

// wrap the events with a TLS translator

events = evio.Translate(events, nil, 
	func(id int, rw io.ReadWriter) io.ReadWriter {
		return tls.Server(evio.NopConn(rw), config)
	},
)

log.Fatal(evio.Serve(events, "tcp://0.0.0.0:443"))

Here we wrapped the event with a TLS translator. The evio.NopConn function is used to converts the ReadWriter a net.Conn so the tls.Server() call will work.

There's a working TLS example at examples/http-server/main.go that binds to port 8080 and 4443 using an developer SSL certificate. The 8080 connections will be insecure and the 4443 will be secure.

$ cd examples/http-server
$ go run main.go --tlscert example.pem
2017/11/02 06:24:33 http server started on port 8080
2017/11/02 06:24:33 https server started on port 4443

$ curl http://localhost:8080
Hello World!
$ curl -k https://localhost:4443
Hello World!

UDP

The Serve function can bind to UDP addresses.

• The Opened event will fire when a UDP packet is received from a new remote address.
• The Closed event will fire when the server is shutdown or the Close action is explicitly returned from an event.
• The Wake and Dial operations are not available to UDP connections.
• All incoming and outgoing packets are not buffered and sent individually.

SO_REUSEPORT

Servers can utilize the SO_REUSEPORT option which allows multiple sockets on the same host to bind to the same port.

Just provide reuseport=true to an address:

evio.Serve(events, "tcp://0.0.0.0:1234?reuseport=true"))

More examples

Please check out the examples subdirectory for a simplified redis clone, an echo server, and a very basic http server with TLS support.

To run an example:

$ go run examples/http-server/main.go
$ go run examples/redis-server/main.go
$ go run examples/echo-server/main.go

Performance

Benchmarks

These benchmarks were run on an ec2 c4.xlarge instance in single-threaded mode (GOMAXPROC=1) over Ipv4 localhost. Check out benchmarks for more info.

Contact

Josh Baker @tidwall

License

evio source code is available under the MIT License.

Documentation

Index

• func NopConn(rw io.ReadWriter) net.Conn
• func Serve(events Events, addr ...string) error
• type Action
• type Events
  • func Translate(events Events, should func(id int, info Info) bool, ...) Events
• type Info
• type InputStream
  • func (is *InputStream) Begin(packet []byte) (data []byte)
  • func (is *InputStream) End(data []byte)
• type Options
• type Server

Functions

func NopConn

func NopConn(rw io.ReadWriter) net.Conn

NopConn returns a net.Conn with a no-op LocalAddr, RemoteAddr, SetDeadline, SetWriteDeadline, SetReadDeadline, and Close methods wrapping the provided ReadWriter rw.

func Serve

func Serve(events Events, addr ...string) error

Serve starts handling events for the specified addresses.

Addresses should use a scheme prefix and be formatted like `tcp://192.168.0.10:9851` or `unix://socket`. Valid network schemes:

tcp   - bind to both IPv4 and IPv6
tcp4  - IPv4
tcp6  - IPv6
udp   - bind to both IPv4 and IPv6
udp4  - IPv4
udp6  - IPv6
unix  - Unix Domain Socket

The "tcp" network scheme is assumed when one is not specified.

Types

type Action

type Action int

Action is an action that occurs after the completion of an event.

const (
	// None indicates that no action should occur following an event.
	None Action = iota
	// Detach detaches the client. Not available for UDP connections.
	Detach
	// Close closes the client.
	Close
	// Shutdown shutdowns the server.
	Shutdown
)

type Events

type Events struct {
	// Serving fires when the server can accept connections. The server
	// parameter has information and various utilities.
	Serving func(server Server) (action Action)
	// Opened fires when a new connection has opened.
	// The info parameter has information about the connection such as
	// it's local and remote address.
	// Use the out return value to write data to the connection.
	// The opts return value is used to set connection options.
	Opened func(id int, info Info) (out []byte, opts Options, action Action)
	// Closed fires when a connection has closed.
	// The err parameter is the last known connection error.
	Closed func(id int, err error) (action Action)
	// Detached fires when a connection has been previously detached.
	// Once detached it's up to the receiver of this event to manage the
	// state of the connection. The Closed event will not be called for
	// this connection.
	// The conn parameter is a ReadWriteCloser that represents the
	// underlying socket connection. It can be freely used in goroutines
	// and should be closed when it's no longer needed.
	Detached func(id int, rwc io.ReadWriteCloser) (action Action)
	// Data fires when a connection sends the server data.
	// The in parameter is the incoming data.
	// Use the out return value to write data to the connection.
	Data func(id int, in []byte) (out []byte, action Action)
	// Prewrite fires prior to every write attempt.
	// The amount parameter is the number of bytes that will be attempted
	// to be written to the connection.
	Prewrite func(id int, amount int) (action Action)
	// Postwrite fires immediately after every write attempt.
	// The amount parameter is the number of bytes that was written to the
	// connection.
	// The remaining parameter is the number of bytes that still remain in
	// the buffer scheduled to be written.
	Postwrite func(id int, amount, remaining int) (action Action)
	// Tick fires immediately after the server starts and will fire again
	// following the duration specified by the delay return value.
	Tick func() (delay time.Duration, action Action)
}

Events represents the server events for the Serve call. Each event has an Action return value that is used manage the state of the connection and server.

func Translate

func Translate(
	events Events,
	should func(id int, info Info) bool,
	translate func(id int, rd io.ReadWriter) io.ReadWriter,
) Events

Translate provides a utility for performing byte level translation on the input and output streams for a connection. This is useful for things like compression, encryption, TLS, etc. The function wraps existing events and returns new events that manage the translation. The `should` parameter is an optional function that can be used to ignore or accept the translation for a specific connection. The `translate` parameter is a function that provides a ReadWriter for each new connection and returns a ReadWriter that performs the actual translation.

type Info

type Info struct {
	// Closing is true when the connection is about to close. Expect a Closed
	// event to fire soon.
	Closing bool
	// AddrIndex is the index of server address that was passed to the Serve call.
	AddrIndex int
	// LocalAddr is the connection's local socket address.
	LocalAddr net.Addr
	// RemoteAddr is the connection's remote peer address.
	RemoteAddr net.Addr
}

Info represents a information about the connection

type InputStream

type InputStream struct {
	// contains filtered or unexported fields
}

InputStream is a helper type for managing input streams inside the Data event.

func (*InputStream) Begin

func (is *InputStream) Begin(packet []byte) (data []byte)

Begin accepts a new packet and returns a working sequence of unprocessed bytes.

func (*InputStream) End

func (is *InputStream) End(data []byte)

End shift the stream to match the unprocessed data.

type Options

type Options struct {
	// TCPKeepAlive (SO_KEEPALIVE) socket option.
	TCPKeepAlive time.Duration
	// ReuseInputBuffer will forces the connection to share and reuse the
	// same input packet buffer with all other connections that also use
	// this option.
	// Default value is false, which means that all input data which is
	// passed to the Data event will be a uniquely copied []byte slice.
	ReuseInputBuffer bool
}

Options are set when the client opens.

type Server

type Server struct {
	// The addrs parameter is an array of listening addresses that align
	// with the addr strings passed to the Serve function.
	Addrs []net.Addr
	// Wake is a goroutine-safe function that triggers a Data event
	// (with a nil `in` parameter) for the specified id.  Not available for
	// UDP connections.
	Wake func(id int) (ok bool)
	// Dial is a goroutine-safe function makes a connection to an external
	// server and returns a new connection id. The new connection is added
	// to the event loop and is managed exactly the same way as all the
	// other connections. This operation only fails if the server/loop has
	// been shut down. An `id` that is not zero means the operation succeeded
	// and then there always be exactly one Opened and one Closed event
	// following this call. Look for socket errors from the Closed event.
	// Not available for UDP connections.
	Dial func(addr string, timeout time.Duration) (id int)
}

Server represents a server context which provides information about the running server and has control functions for managing state.