redcon

package module
v1.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 20, 2023 License: MIT Imports: 14 Imported by: 247

README

REDCON
GoDoc

Redis compatible server framework for Go

Features

  • Create a Fast custom Redis compatible server in Go
  • Simple interface. One function ListenAndServe and two types Conn & Command
  • Support for pipelining and telnet commands
  • Works with Redis clients such as redigo, redis-py, node_redis, and jedis
  • TLS Support
  • Compatible pub/sub support
  • Multithreaded

Installing

go get -u github.com/tidwall/redcon

Example

Here's a full example of a Redis clone that accepts:

  • SET key value
  • GET key
  • DEL key
  • PING
  • QUIT
  • PUBLISH channel message
  • SUBSCRIBE channel

You can run this example from a terminal:

go run example/clone.go

package main

import (
	"log"
	"strings"
	"sync"

	"github.com/tidwall/redcon"
)

var addr = ":6380"

func main() {
	var mu sync.RWMutex
	var items = make(map[string][]byte)
	var ps redcon.PubSub
	go log.Printf("started server at %s", addr)
	err := redcon.ListenAndServe(addr,
		func(conn redcon.Conn, cmd redcon.Command) {
			switch strings.ToLower(string(cmd.Args[0])) {
			default:
				conn.WriteError("ERR unknown command '" + string(cmd.Args[0]) + "'")
			case "ping":
				conn.WriteString("PONG")
			case "quit":
				conn.WriteString("OK")
				conn.Close()
			case "set":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				items[string(cmd.Args[1])] = cmd.Args[2]
				mu.Unlock()
				conn.WriteString("OK")
			case "get":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.RLock()
				val, ok := items[string(cmd.Args[1])]
				mu.RUnlock()
				if !ok {
					conn.WriteNull()
				} else {
					conn.WriteBulk(val)
				}
			case "del":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				_, ok := items[string(cmd.Args[1])]
				delete(items, string(cmd.Args[1]))
				mu.Unlock()
				if !ok {
					conn.WriteInt(0)
				} else {
					conn.WriteInt(1)
				}
			case "publish":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				conn.WriteInt(ps.Publish(string(cmd.Args[1]), string(cmd.Args[2])))
			case "subscribe", "psubscribe":
				if len(cmd.Args) < 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				command := strings.ToLower(string(cmd.Args[0]))
				for i := 1; i < len(cmd.Args); i++ {
					if command == "psubscribe" {
						ps.Psubscribe(conn, string(cmd.Args[i]))
					} else {
						ps.Subscribe(conn, string(cmd.Args[i]))
					}
				}
			}
		},
		func(conn redcon.Conn) bool {
			// Use this function to accept or deny the connection.
			// log.Printf("accept: %s", conn.RemoteAddr())
			return true
		},
		func(conn redcon.Conn, err error) {
			// This is called when the connection has been closed
			// log.Printf("closed: %s, err: %v", conn.RemoteAddr(), err)
		},
	)
	if err != nil {
		log.Fatal(err)
	}
}

TLS Example

Redcon has full TLS support through the ListenAndServeTLS function.

The same example is also provided for serving Redcon over TLS.

go run example/tls/clone.go

Benchmarks

Redis: Single-threaded, no disk persistence.

$ redis-server --port 6379 --appendonly no

redis-benchmark -p 6379 -t set,get -n 10000000 -q -P 512 -c 512
SET: 941265.12 requests per second
GET: 1189909.50 requests per second

Redcon: Single-threaded, no disk persistence.

$ GOMAXPROCS=1 go run example/clone.go

redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 2018570.88 requests per second
GET: 2403846.25 requests per second

Redcon: Multi-threaded, no disk persistence.

$ GOMAXPROCS=0 go run example/clone.go

$ redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 1944390.38 requests per second
GET: 3993610.25 requests per second

Running on a MacBook Pro 15" 2.8 GHz Intel Core i7 using Go 1.7

Contact

Josh Baker @tidwall

License

Redcon source code is available under the MIT License.

Documentation

Overview

Package redcon implements a Redis compatible server framework

Index

Constants

View Source 
const (
	Integer = ':'
	String  = '+'
	Bulk    = '$'
	Array   = '*'
	Error   = '-'
)

Various RESP kinds

Variables

This section is empty.

Functions

func AppendAny added in v1.3.0 

func AppendAny(b []byte, v interface{}) []byte

AppendAny appends any type to valid Redis type. 

nil             -> null
error           -> error (adds "ERR " when first word is not uppercase)
string          -> bulk-string
numbers         -> bulk-string
[]byte          -> bulk-string
bool            -> bulk-string ("0" or "1")
slice           -> array
map             -> array with key/value pairs
SimpleString    -> string
SimpleInt       -> integer
Marshaler       -> raw bytes
everything-else -> bulk-string representation using fmt.Sprint()

func AppendArray 

func AppendArray(b []byte, n int) []byte

AppendArray appends a Redis protocol array to the input bytes.

func AppendBulk 

func AppendBulk(b []byte, bulk []byte) []byte

AppendBulk appends a Redis protocol bulk byte slice to the input bytes.

func AppendBulkFloat added in v1.0.2 

func AppendBulkFloat(dst []byte, f float64) []byte

AppendBulkFloat appends a float64, as bulk bytes.

func AppendBulkInt added in v1.0.2 

func AppendBulkInt(dst []byte, x int64) []byte

AppendBulkInt appends an int64, as bulk bytes.

func AppendBulkString 

func AppendBulkString(b []byte, bulk string) []byte

AppendBulkString appends a Redis protocol bulk string to the input bytes.

func AppendBulkUint added in v1.0.2 

func AppendBulkUint(dst []byte, x uint64) []byte

AppendBulkUint appends an uint64, as bulk bytes.

func AppendError 

func AppendError(b []byte, s string) []byte

AppendError appends a Redis protocol error to the input bytes.

func AppendInt 

func AppendInt(b []byte, n int64) []byte

AppendInt appends a Redis protocol int64 to the input bytes.

func AppendNull 

func AppendNull(b []byte) []byte

AppendNull appends a Redis protocol null to the input bytes.

func AppendOK 

func AppendOK(b []byte) []byte

AppendOK appends a Redis protocol OK to the input bytes.

func AppendString 

func AppendString(b []byte, s string) []byte

AppendString appends a Redis protocol string to the input bytes.

func AppendTile38 

func AppendTile38(b []byte, data []byte) []byte

AppendTile38 appends a Tile38 message to the input bytes.

func AppendUint 

func AppendUint(b []byte, n uint64) []byte

AppendUint appends a Redis protocol uint64 to the input bytes.

func ListenAndServe 

func ListenAndServe(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) error

ListenAndServe creates a new server and binds to addr configured on "tcp" network net.

func ListenAndServeNetwork 

func ListenAndServeNetwork(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) error

ListenAndServeNetwork creates a new server and binds to addr. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func ListenAndServeNetworkTLS 

func ListenAndServeNetworkTLS(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) error

ListenAndServeNetworkTLS creates a new TLS server and binds to addr. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func ListenAndServeTLS 

func ListenAndServeTLS(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) error

ListenAndServeTLS creates a new TLS server and binds to addr configured on "tcp" network net.

func Serve added in v1.0.0 

func Serve(ln net.Listener,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) error

Serve creates a new server and serves with the given net.Listener.

Types

type Command 

type Command struct {
	// Raw is a encoded RESP message.
	Raw []byte
	// Args is a series of arguments that make up the command.
	Args [][]byte
}

Command represent a command

func Parse 

func Parse(raw []byte) (Command, error)

Parse parses a raw RESP message and returns a command.

type Conn 

type Conn interface {
	// RemoteAddr returns the remote address of the client connection.
	RemoteAddr() string
	// Close closes the connection.
	Close() error
	// WriteError writes an error to the client.
	WriteError(msg string)
	// WriteString writes a string to the client.
	WriteString(str string)
	// WriteBulk writes bulk bytes to the client.
	WriteBulk(bulk []byte)
	// WriteBulkString writes a bulk string to the client.
	WriteBulkString(bulk string)
	// WriteInt writes an integer to the client.
	WriteInt(num int)
	// WriteInt64 writes a 64-bit signed integer to the client.
	WriteInt64(num int64)
	// WriteUint64 writes a 64-bit unsigned integer to the client.
	WriteUint64(num uint64)
	// WriteArray writes an array header. You must then write additional
	// sub-responses to the client to complete the response.
	// For example to write two strings:
	//
	//   c.WriteArray(2)
	//   c.WriteBulkString("item 1")
	//   c.WriteBulkString("item 2")
	WriteArray(count int)
	// WriteNull writes a null to the client
	WriteNull()
	// WriteRaw writes raw data to the client.
	WriteRaw(data []byte)
	// WriteAny writes any type to the client.
	//   nil             -> null
	//   error           -> error (adds "ERR " when first word is not uppercase)
	//   string          -> bulk-string
	//   numbers         -> bulk-string
	//   []byte          -> bulk-string
	//   bool            -> bulk-string ("0" or "1")
	//   slice           -> array
	//   map             -> array with key/value pairs
	//   SimpleString    -> string
	//   SimpleInt       -> integer
	//   everything-else -> bulk-string representation using fmt.Sprint()
	WriteAny(any interface{})
	// Context returns a user-defined context
	Context() interface{}
	// SetContext sets a user-defined context
	SetContext(v interface{})
	// SetReadBuffer updates the buffer read size for the connection
	SetReadBuffer(bytes int)
	// Detach return a connection that is detached from the server.
	// Useful for operations like PubSub.
	//
	//   dconn := conn.Detach()
	//   go func(){
	//       defer dconn.Close()
	//       cmd, err := dconn.ReadCommand()
	//       if err != nil{
	//           fmt.Printf("read failed: %v\n", err)
	//	         return
	//       }
	//       fmt.Printf("received command: %v", cmd)
	//       hconn.WriteString("OK")
	//       if err := dconn.Flush(); err != nil{
	//           fmt.Printf("write failed: %v\n", err)
	//	         return
	//       }
	//   }()
	Detach() DetachedConn
	// ReadPipeline returns all commands in current pipeline, if any
	// The commands are removed from the pipeline.
	ReadPipeline() []Command
	// PeekPipeline returns all commands in current pipeline, if any.
	// The commands remain in the pipeline.
	PeekPipeline() []Command
	// NetConn returns the base net.Conn connection
	NetConn() net.Conn
}

Conn represents a client connection

type DetachedConn 

type DetachedConn interface {
	// Conn is the original connection
	Conn
	// ReadCommand reads the next client command.
	ReadCommand() (Command, error)
	// Flush flushes any writes to the network.
	Flush() error
}

DetachedConn represents a connection that is detached from the server

type Handler added in v1.0.1 

type Handler interface {
	ServeRESP(conn Conn, cmd Command)
}

A Handler responds to an RESP request.

type HandlerFunc added in v1.0.1 

type HandlerFunc func(conn Conn, cmd Command)

The HandlerFunc type is an adapter to allow the use of ordinary functions as RESP handlers. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler that calls f.

func (HandlerFunc) ServeRESP added in v1.0.1 

func (f HandlerFunc) ServeRESP(conn Conn, cmd Command)

ServeRESP calls f(w, r)

type Kind 

type Kind int

Kind is the kind of command 

const (
	// Redis is returned for Redis protocol commands
	Redis Kind = iota
	// Tile38 is returnd for Tile38 native protocol commands
	Tile38
	// Telnet is returnd for plain telnet commands
	Telnet
)

func ReadNextCommand 

func ReadNextCommand(packet []byte, argsbuf [][]byte) (
	complete bool, args [][]byte, kind Kind, leftover []byte, err error,
)

ReadNextCommand reads the next command from the provided packet. It's possible that the packet contains multiple commands, or zero commands when the packet is incomplete. 'argsbuf' is an optional reusable buffer and it can be nil. 'complete' indicates that a command was read. false means no more commands. 'args' are the output arguments for the command. 'kind' is the type of command that was read. 'leftover' is any remaining unused bytes which belong to the next command. 'err' is returned when a protocol error was encountered.

type Marshaler added in v1.3.3 

type Marshaler interface {
	MarshalRESP() []byte
}

Marshaler is the interface implemented by types that can marshal themselves into a Redis response type from an *Any call. The return value is not check for validity.

type PubSub added in v1.4.0 

type PubSub struct {
	// contains filtered or unexported fields
}

PubSub is a Redis compatible pub/sub server

func (*PubSub) Psubscribe added in v1.4.0 

func (ps *PubSub) Psubscribe(conn Conn, channel string)

Psubscribe a connection to PubSub

func (*PubSub) Publish added in v1.4.0 

func (ps *PubSub) Publish(channel, message string) int

Publish a message to subscribers

func (*PubSub) Subscribe added in v1.4.0 

func (ps *PubSub) Subscribe(conn Conn, channel string)

Subscribe a connection to PubSub

type RESP added in v1.2.0 

type RESP struct {
	Type  Type
	Raw   []byte
	Data  []byte
	Count int
}

func ReadNextRESP added in v1.2.0 

func ReadNextRESP(b []byte) (n int, resp RESP)

ReadNextRESP returns the next resp in b and returns the number of bytes the took up the result.

func (RESP) Bytes added in v1.4.6 

func (r RESP) Bytes() []byte

func (RESP) Exists added in v1.5.1 

func (r RESP) Exists() bool

func (RESP) Float added in v1.4.6 

func (r RESP) Float() float64

func (RESP) ForEach added in v1.2.0 

func (r RESP) ForEach(iter func(resp RESP) bool)

ForEach iterates over each Array element

func (RESP) Int added in v1.4.6 

func (r RESP) Int() int64

func (RESP) Map added in v1.5.0 

func (r RESP) Map() map[string]RESP

Map returns a key/value map of an Array. The receiver RESP must be an Array with an equal number of values, where the value of the key is followed by the key. Example: key1,value1,key2,value2,key3,value3

func (RESP) MapGet added in v1.5.1 

func (r RESP) MapGet(key string) RESP

func (RESP) String added in v1.4.6 

func (r RESP) String() string

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader represent a reader for RESP or telnet commands.

func NewReader 

func NewReader(rd io.Reader) *Reader

NewReader returns a command reader which will read RESP or telnet commands.

func (*Reader) ReadCommand 

func (rd *Reader) ReadCommand() (Command, error)

ReadCommand reads the next command.

func (*Reader) ReadCommands added in v1.4.1 

func (rd *Reader) ReadCommands() ([]Command, error)

ReadCommands reads the next pipeline commands.

type ServeMux added in v1.0.1 

type ServeMux struct {
	// contains filtered or unexported fields
}

ServeMux is an RESP command multiplexer.

func NewServeMux added in v1.0.1 

func NewServeMux() *ServeMux

NewServeMux allocates and returns a new ServeMux.

func (*ServeMux) Handle added in v1.0.1 

func (m *ServeMux) Handle(command string, handler Handler)

Handle registers the handler for the given command. If a handler already exists for command, Handle panics.

func (*ServeMux) HandleFunc added in v1.0.1 

func (m *ServeMux) HandleFunc(command string, handler func(conn Conn, cmd Command))

HandleFunc registers the handler function for the given command.

func (*ServeMux) ServeRESP added in v1.0.1 

func (m *ServeMux) ServeRESP(conn Conn, cmd Command)

ServeRESP dispatches the command to the handler.

type Server 

type Server struct {

	// AcceptError is an optional function used to handle Accept errors.
	AcceptError func(err error)
	// contains filtered or unexported fields
}

Server defines a server for clients for managing client connections.

func NewServer 

func NewServer(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) *Server

NewServer returns a new Redcon server configured on "tcp" network net.

func NewServerNetwork 

func NewServerNetwork(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) *Server

NewServerNetwork returns a new Redcon server. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func (*Server) Addr 

func (s *Server) Addr() net.Addr

Addr returns server's listen address

func (*Server) Close 

func (s *Server) Close() error

Close stops listening on the TCP address. Already Accepted connections will be closed.

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe() error

ListenAndServe serves incoming connections.

func (*Server) ListenServeAndSignal 

func (s *Server) ListenServeAndSignal(signal chan error) error

ListenServeAndSignal serves incoming connections and passes nil or error when listening. signal can be nil.

func (*Server) Serve added in v1.0.0 

func (s *Server) Serve(ln net.Listener) error

Serve serves incoming connections with the given net.Listener.

func (*Server) SetIdleClose added in v1.4.0 

func (s *Server) SetIdleClose(dur time.Duration)

SetIdleClose will automatically close idle connections after the specified duration. Use zero to disable this feature.

type SimpleError added in v1.6.0 

type SimpleError error

SimpleError is for representing an error without adding the "ERR" prefix from an *Any call.

type SimpleInt added in v1.3.2 

type SimpleInt int

SimpleInt is for representing a non-bulk representation of a int from an *Any call.

type SimpleString added in v1.3.2 

type SimpleString string

SimpleString is for representing a non-bulk representation of a string from an *Any call.

type TLSServer 

type TLSServer struct {
	*Server
	// contains filtered or unexported fields
}

TLSServer defines a server for clients for managing client connections.

func NewServerNetworkTLS 

func NewServerNetworkTLS(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) *TLSServer

NewServerNetworkTLS returns a new TLS Redcon server. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func NewServerTLS 

func NewServerTLS(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) *TLSServer

NewServerTLS returns a new Redcon TLS server configured on "tcp" network net.

func (*TLSServer) Close 

func (s *TLSServer) Close() error

Close stops listening on the TCP address. Already Accepted connections will be closed.

func (*TLSServer) ListenAndServe 

func (s *TLSServer) ListenAndServe() error

ListenAndServe serves incoming connections.

func (*TLSServer) ListenServeAndSignal 

func (s *TLSServer) ListenServeAndSignal(signal chan error) error

ListenServeAndSignal serves incoming connections and passes nil or error when listening. signal can be nil.

type Type added in v1.2.0 

type Type byte

Type of RESP

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

Writer allows for writing RESP messages.

func BaseWriter 

func BaseWriter(c Conn) *Writer

BaseWriter returns the underlying connection writer, if any

func NewWriter 

func NewWriter(wr io.Writer) *Writer

NewWriter creates a new RESP writer.

func (*Writer) Buffer 

func (w *Writer) Buffer() []byte

Buffer returns the unflushed buffer. This is a copy so changes to the resulting []byte will not affect the writer.

func (*Writer) Flush 

func (w *Writer) Flush() error

Flush writes all unflushed Write* calls to the underlying writer.

func (*Writer) SetBuffer 

func (w *Writer) SetBuffer(raw []byte)

SetBuffer replaces the unflushed buffer with new bytes.

func (*Writer) WriteAny added in v1.3.0 

func (w *Writer) WriteAny(v interface{})

WriteAny writes any type to client. 

nil             -> null
error           -> error (adds "ERR " when first word is not uppercase)
string          -> bulk-string
numbers         -> bulk-string
[]byte          -> bulk-string
bool            -> bulk-string ("0" or "1")
slice           -> array
map             -> array with key/value pairs
SimpleString    -> string
SimpleInt       -> integer
everything-else -> bulk-string representation using fmt.Sprint()

func (*Writer) WriteArray 

func (w *Writer) WriteArray(count int)

WriteArray writes an array header. You must then write additional sub-responses to the client to complete the response. For example to write two strings: 

c.WriteArray(2)
c.WriteBulkString("item 1")
c.WriteBulkString("item 2")

func (*Writer) WriteBulk 

func (w *Writer) WriteBulk(bulk []byte)

WriteBulk writes bulk bytes to the client.

func (*Writer) WriteBulkString 

func (w *Writer) WriteBulkString(bulk string)

WriteBulkString writes a bulk string to the client.

func (*Writer) WriteError 

func (w *Writer) WriteError(msg string)

WriteError writes an error to the client.

func (*Writer) WriteInt 

func (w *Writer) WriteInt(num int)

WriteInt writes an integer to the client.

func (*Writer) WriteInt64 

func (w *Writer) WriteInt64(num int64)

WriteInt64 writes a 64-bit signed integer to the client.

func (*Writer) WriteNull 

func (w *Writer) WriteNull()

WriteNull writes a null to the client

func (*Writer) WriteRaw 

func (w *Writer) WriteRaw(data []byte)

WriteRaw writes raw data to the client.

func (*Writer) WriteString 

func (w *Writer) WriteString(msg string)

WriteString writes a string to the client.

func (*Writer) WriteUint64 added in v1.0.1 

func (w *Writer) WriteUint64(num uint64)

WriteUint64 writes a 64-bit unsigned integer to the client.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.