gnet

README

English | 🇨🇳中文

📖 Introduction

gnet is an event-driven networking framework that is fast and small. It makes direct epoll and kqueue syscalls rather than using the standard Go net package, and works in a similar manner as netty and libuv.

The goal of this project is to create a server framework for Go that performs on par with Redis and Haproxy for packet handling.

gnet sells itself as a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go which works on transport layer with TCP/UDP/Unix-Socket protocols, so it allows developers to implement their own protocols of application layer upon gnet for building diversified network applications, for instance, you get an HTTP Server or Web Framework if you implement HTTP protocol upon gnet while you have a Redis Server done with the implementation of Redis protocol upon gnet and so on.

gnet derives from the project: evio while having a much higher performance.

🚀 Features

• High-performance event-loop under multi-threads/goroutines model
• Built-in load balancing algorithm: Round-Robin
• Built-in goroutine pool powered by the library ants
• Built-in memory pool with bytes powered by the library pool
• Concise APIs
• Efficient memory usage: Ring-Buffer
• Supporting multiple protocols: TCP, UDP, and Unix Sockets
• Supporting two event-notification mechanisms: epoll on Linux and kqueue on FreeBSD
• Supporting asynchronous write operation
• Flexible ticker event
• SO_REUSEPORT socket option
• Additional load-balancing algorithms: Random, Least-Connections, Consistent-hashing and so on
• New event-notification mechanism: IOCP on Windows platform
• TLS support
• Implementation of gnet Client

💡 Key Designs

Multiple-Threads/Goroutines Model

Multiple Reactors Model

gnet redesigns and implements a new built-in multiple-threads/goroutines model: 『Multiple Reactors』 which is also the default multiple-threads model of netty, Here's the schematic diagram:

and it works as the following sequence diagram:

Multiple Reactors + Goroutine-Pool Model

You may ask me a question: what if my business logic in EventHandler.React contains some blocking code which leads to blocking in event-loop of gnet, what is the solution for this kind of situation？

As you know, there is a most important tenet when writing code under gnet: you should never block the event-loop in the EventHandler.React, otherwise, it will lead to a low throughput in your gnet server, which is also the most important tenet in netty.

And the solution for that could be found in the subsequent multiple-threads/goroutines model of gnet: 『Multiple Reactors with thread/goroutine pool』which pulls you out from the blocking mire, it will construct a worker-pool with fixed capacity and put those blocking jobs in EventHandler.React into the worker-pool to make the event-loop goroutines non-blocking.

The architecture diagram of『Multiple Reactors with thread/goroutine pool』networking model architecture is in here:

and it works as the following sequence diagram:

gnet implements the networking model of 『Multiple Reactors with thread/goroutine pool』by the aid of a high-performance goroutine pool called ants that allows you to manage and recycle a massive number of goroutines in your concurrent programs, the full features and usages in ants are documented here.

gnet integrates ants and provides the pool.NewWorkerPool method that you can invoke to instantiate a ants pool where you are able to put your blocking code logic in EventHandler.React and invoke the function of gnet.Conn.AsyncWrite to send out data asynchronously in worker pool after you finish the blocking process and get the output data, which makes the goroutine of event-loop non-blocking.

The details about integrating gnet with ants are shown here.

Auto-scaling Ring Buffer

gnet utilizes Ring-Buffer to buffer network data and manage memories in networking.

🎉 Getting Started

Prerequisites

gnet requires Go 1.9 or later.

Installation

go get -u github.com/panjf2000/gnet

gnet is available as a Go module, with Go 1.11 Modules support (Go 1.11+), just simply import "github.com/panjf2000/gnet" in your source code and go [build|run|test] will download the necessary dependencies automatically.

Usage Examples

The detailed documentation is located in here: docs of gnet, but let's pass through the brief instructions first.

It is easy to create a network server with gnet. All you have to do is just make your implementation of gnet.EventHandler interface and register your event-handler functions to it, then pass it to the gnet.Serve function along with the binding address(es). Each connection is represented as a gnet.Conn interface that is passed to various events to differentiate the clients. At any point you can close a client or shutdown the server by return a Close or Shutdown action from an event.

The simplest example to get you started playing with gnet would be the echo server. So here you are, a simplest echo server upon gnet that is listening on port 9000:

Echo server without blocking logic

package main

import (
	"log"

	"github.com/panjf2000/gnet"
)

type echoServer struct {
	*gnet.EventServer
}

func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	out = c.Read()
	c.ResetBuffer()
	return
}

func main() {
	echo := new(echoServer)
	log.Fatal(gnet.Serve(echo, "tcp://:9000", gnet.WithMulticore(true)))
}

As you can see, this example of echo server only sets up the EventHandler.React function where you commonly write your main business code and it will be invoked once the server receives input data from a client. The output data will be then sent back to that client by assigning the out variable and return it after your business code finish processing data(in this case, it just echo the data back).

Echo server with blocking logic

package main

import (
	"log"
	"time"

	"github.com/panjf2000/gnet"
	"github.com/panjf2000/gnet/pool"
)

type echoServer struct {
	*gnet.EventServer
	pool *pool.WorkerPool
}

func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	data := append([]byte{}, c.Read()...)
	c.ResetBuffer()

	// Use ants pool to unblock the event-loop.
	_ = es.pool.Submit(func() {
		time.Sleep(1 * time.Second)
		c.AsyncWrite(data)
	})

	return
}

func main() {
	p := pool.NewWorkerPool()
	defer p.Release()
	
	echo := &echoServer{pool: p}
	log.Fatal(gnet.Serve(echo, "tcp://:9000", gnet.WithMulticore(true)))
}

Like I said in the 『Multiple Reactors + Goroutine-Pool Model』section, if there are blocking code in your business logic, then you ought to turn them into non-blocking code in any way, for instance you can wrap them into a goroutine, but it will result in a massive amount of goroutines if massive traffic is passing through your server so I would suggest you utilize a goroutine pool like ants to manage those goroutines and reduce the cost of system resources.

For more examples, check out here: examples of gnet.

I/O Events

Current supported I/O events in gnet:

• EventHandler.OnInitComplete is activated when the server is ready to accept new connections.
• EventHandler.OnOpened is activated when a connection has opened.
• EventHandler.OnClosed is activated when a connection has closed.
• EventHandler.React is activated when the server receives new data from a connection. (usually it is where you write the code of business logic)
• EventHandler.Tick is activated immediately after the server starts and will fire again after a specified interval.
• EventHandler.PreWrite is activated just before any data is written to any client socket.

Ticker

The EventHandler.Tick event fires ticks at a specified interval. The first tick fires immediately after the Serving events and if you intend to set up a ticker event, remember to pass an option: gnet.WithTicker(true) to gnet.Serve.

events.Tick = func() (delay time.Duration, action Action){
	log.Printf("tick")
	delay = time.Second
	return
}

UDP

The gnet.Serve function can bind to UDP addresses.

• All incoming and outgoing packets will not be buffered but sent individually.
• The EventHandler.OnOpened and EventHandler.OnClosed events are not available for UDP sockets, only the React event.

Multi-threads

The gnet.WithMulticore(true) indicates whether the server will be effectively created with multi-cores, if so, then you must take care of synchronizing memory between all event callbacks, otherwise, it will run the server with a single thread. The number of threads in the server will be automatically assigned to the value of runtime.NumCPU().

Load balancing

The current built-in load balancing algorithm in gnet is Round-Robin.

SO_REUSEPORT

Servers can utilize the SO_REUSEPORT option which allows multiple sockets on the same host to bind to the same port and the OS kernel takes care of the load balancing for you, it wakes one socket per accpet event coming to resolved the thundering herd.

Just use functional options to set up SO_REUSEPORT and you can enjoy this feature:

gnet.Serve(events, "tcp://:9000", gnet.WithMulticore(true), gnet.WithReusePort(true)))

📊 Performance

Contrasts to the similar networking libraries

On Linux (epoll)

Test Environment

# Machine information
        OS : Ubuntu 18.04/x86_64
       CPU : 8 Virtual CPUs
    Memory : 16.0 GiB

# Go version and configurations
Go Version : go1.12.9 linux/amd64
GOMAXPROCS=8

Echo Server

HTTP Server

On FreeBSD (kqueue)

Test Environment

# Machine information
        OS : macOS Mojave 10.14.6/x86_64
       CPU : 4 CPUs
    Memory : 8.0 GiB

# Go version and configurations
Go Version : go version go1.12.9 darwin/amd64
GOMAXPROCS=4

Echo Server

HTTP Server

📄 License

Source code in gnet is available under the MIT License.

👏 Contributors

Please read our Contributing Guidelines before opening a PR and thank you to all the developers who already made contributions to gnet!

🙏 Thanks

• evio
• netty
• ants
• pool

📚 Relevant Articles

• A Million WebSockets and Go
• Going Infinite, handling 1M websockets connections in Go
• gnet: 一个轻量级且高性能的 Golang 网络库

JetBrains OS licenses

gnet had been being developed with GoLand IDE under the free JetBrains Open Source license(s) granted by JetBrains s.r.o., hence I would like to express my thanks here.

💰 Backers

Support us with a monthly donation and help us continue our activities.

💎 Sponsors

Become a bronze sponsor with a monthly donation of $10 and get your logo on our README on Github.

☕️ Buy me a coffee

        

Documentation

Overview

gnet is an event-driven networking framework that is fast and small. It makes direct epoll and kqueue syscalls rather than using the standard Go net package, and works in a similar manner as netty and libuv.

The goal of this project is to create a server framework for Go that performs on par with Redis and Haproxy for packet handling.

gnet sells itself as a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go which works on transport layer with TCP/UDP/Unix-Socket protocols, so it allows developers to implement their own protocols of application layer upon gnet for building diversified network applications, for instance, you get an HTTP Server or Web Framework if you implement HTTP protocol upon gnet while you have a Redis Server done with the implementation of Redis protocol upon gnet and so on.

Echo server built upon gnet is shown below:

package main

import (
	"log"

	"github.com/panjf2000/gnet"
)

type echoServer struct {
	*gnet.EventServer
}

func (es *echoServer) React(c gnet.Conn) (out []byte, action gnet.Action) {
	out = c.Read()
	c.ResetBuffer()
	return
}

func main() {
	echo := new(echoServer)
	log.Fatal(gnet.Serve(echo, "tcp://:9000", gnet.WithMulticore(true)))
}

Index

• func Serve(eventHandler EventHandler, addr string, opts ...Option) error
• type Action
• type Conn
• type EventHandler
• type EventServer
  • func (es *EventServer) OnClosed(c Conn, err error) (action Action)
  • func (es *EventServer) OnInitComplete(svr Server) (action Action)
  • func (es *EventServer) OnOpened(c Conn) (out []byte, action Action)
  • func (es *EventServer) PreWrite()
  • func (es *EventServer) React(c Conn) (out []byte, action Action)
  • func (es *EventServer) Tick() (delay time.Duration, action Action)
• type IEventLoopGroup
• type Option
  • func WithMulticore(multicore bool) Option
  • func WithOptions(options Options) Option
  • func WithReusePort(reusePort bool) Option
  • func WithTCPKeepAlive(tcpKeepAlive time.Duration) Option
  • func WithTicker(ticker bool) Option
• type Options
• type Server

Functions

func Serve

func Serve(eventHandler EventHandler, addr string, opts ...Option) error

Serve starts handling events for the specified addresses.

Addresses should use a scheme prefix and be formatted like `tcp://192.168.0.10:9851` or `unix://socket`. Valid network schemes:

tcp   - bind to both IPv4 and IPv6
tcp4  - IPv4
tcp6  - IPv6
udp   - bind to both IPv4 and IPv6
udp4  - IPv4
udp6  - IPv6
unix  - Unix Domain Socket

The "tcp" network scheme is assumed when one is not specified.

Types

type Action

type Action int

Action is an action that occurs after the completion of an event.

const (
	// None indicates that no action should occur following an event.
	None Action = iota

	// Close closes the connection.
	Close

	// Shutdown shutdowns the server.
	Shutdown
)

type Conn

type Conn interface {
	// Context returns a user-defined context.
	Context() (ctx interface{})

	// SetContext sets a user-defined context.
	SetContext(ctx interface{})

	// LocalAddr is the connection's local socket address.
	LocalAddr() (addr net.Addr)

	// RemoteAddr is the connection's remote peer address.
	RemoteAddr() (addr net.Addr)

	// Read reads all data from inbound ring-buffer without moving "read" pointer, which means
	// it does not evict the data from ring-buffer actually and those data will present in ring-buffer until the
	// ResetBuffer method is invoked.
	Read() (buf []byte)

	// ResetBuffer resets the inbound ring-buffer, which means all data in the inbound ring-buffer has been evicted.
	ResetBuffer()

	// ReadN reads bytes with the given length from inbound ring-buffer and event-loop-buffer, it would move
	// "read" pointer, which means it will evict the data from buffer and it can't be revoked (put back to buffer),
	// it reads data from the inbound ring-buffer and event-loop-buffer when the length of the available data is equal
	// to the given "n", otherwise, it will not read any data from the inbound ring-buffer. So you should use this
	// function only if you know exactly the length of subsequent TCP streams based on the protocol, like the
	// Content-Length attribute in an HTTP request which indicates you how much data you should read from inbound ring-buffer.
	ReadN(n int) (size int, buf []byte)

	// BufferLength returns the length of available data in the inbound ring-buffer.
	BufferLength() (size int)

	// AyncWrite writes data to client/connection asynchronously.
	AsyncWrite(buf []byte)

	// Wake triggers a React event for this connection.
	Wake()
}

Conn is a interface of gnet connection.

type EventHandler

type EventHandler interface {
	// OnInitComplete fires when the server is ready for accepting connections.
	// The server parameter has information and various utilities.
	OnInitComplete(server Server) (action Action)

	// OnOpened fires when a new connection has been opened.
	// The info parameter has information about the connection such as
	// it's local and remote address.
	// Use the out return value to write data to the connection.
	OnOpened(c Conn) (out []byte, action Action)

	// OnClosed fires when a connection has been closed.
	// The err parameter is the last known connection error.
	OnClosed(c Conn, err error) (action Action)

	// PreWrite fires just before any data is written to any client socket.
	PreWrite()

	// React fires when a connection sends the server data.
	// Invoke c.Read() or c.ReadN(n) within the parameter c to read incoming data from client/connection.
	// Use the out return value to write data to the client/connection.
	React(c Conn) (out []byte, action Action)

	// Tick fires immediately after the server starts and will fire again
	// following the duration specified by the delay return value.
	Tick() (delay time.Duration, action Action)
}

EventHandler represents the server events' callbacks for the Serve call. Each event has an Action return value that is used manage the state of the connection and server.

type EventServer

type EventServer struct {
}

EventServer is a built-in implementation of EventHandler which sets up each method with a default implementation, you can compose it with your own implementation of EventHandler when you don't want to implement all methods in EventHandler.

func (*EventServer) OnClosed

func (es *EventServer) OnClosed(c Conn, err error) (action Action)

OnClosed fires when a connection has been closed. The err parameter is the last known connection error.

func (*EventServer) OnInitComplete

func (es *EventServer) OnInitComplete(svr Server) (action Action)

OnInitComplete fires when the server is ready for accepting connections. The server parameter has information and various utilities.

func (*EventServer) OnOpened

func (es *EventServer) OnOpened(c Conn) (out []byte, action Action)

OnOpened fires when a new connection has been opened. The info parameter has information about the connection such as it's local and remote address. Use the out return value to write data to the connection.

func (*EventServer) PreWrite

func (es *EventServer) PreWrite()

PreWrite fires just before any data is written to any client socket.

func (*EventServer) React

func (es *EventServer) React(c Conn) (out []byte, action Action)

React fires when a connection sends the server data. Invoke c.Read() or c.ReadN(n) within the parameter c to read incoming data from client/connection. Use the out return value to write data to the client/connection.

func (*EventServer) Tick

func (es *EventServer) Tick() (delay time.Duration, action Action)

Tick fires immediately after the server starts and will fire again following the duration specified by the delay return value.

type IEventLoopGroup

type IEventLoopGroup interface {
	// contains filtered or unexported methods
}

IEventLoopGroup represents a set of event-loops.

type Option

type Option func(opts *Options)

Option is a function that will set up option.

func WithMulticore

func WithMulticore(multicore bool) Option

WithMulticore ...

func WithOptions

func WithOptions(options Options) Option

WithOptions sets up all options.

func WithReusePort

func WithReusePort(reusePort bool) Option

WithReusePort ...

func WithTCPKeepAlive

func WithTCPKeepAlive(tcpKeepAlive time.Duration) Option

WithTCPKeepAlive ...

func WithTicker

func WithTicker(ticker bool) Option

WithTicker ...

type Options

type Options struct {
	// Multicore indicates whether the server will be effectively created with multi-cores, if so,
	// then you must take care with synchonizing memory between all event callbacks, otherwise,
	// it will run the server with single thread. The number of threads in the server will be automatically
	// assigned to the value of runtime.NumCPU().
	Multicore bool

	// ReusePort indicates whether to set up the SO_REUSEPORT option.
	ReusePort bool

	// Ticker indicates whether the ticker has been set up.
	Ticker bool

	// TCPKeepAlive (SO_KEEPALIVE) socket option.
	TCPKeepAlive time.Duration
}

Options are set when the client opens.

type Server

type Server struct {
	// Multicore indicates whether the server will be effectively created with multi-cores, if so,
	// then you must take care of synchronizing the shared data between all event callbacks, otherwise,
	// it will run the server with single thread. The number of threads in the server will be automatically
	// assigned to the value of runtime.NumCPU().
	Multicore bool

	// The addrs parameter is an array of listening addresses that align
	// with the addr strings passed to the Serve function.
	Addr net.Addr

	// NumLoops is the number of loops that the server is using.
	NumLoops int

	// ReUsePort indicates whether SO_REUSEPORT is enable.
	ReUsePort bool

	// TCPKeepAlive (SO_KEEPALIVE) socket option.
	TCPKeepAlive time.Duration
}

Server represents a server context which provides information about the running server and has control functions for managing state.