roxter

README

roxter

Memcached proxy, focused on:

• Speed
• Sharding
• Memcache Binary Protocol

So, what ? why ?

This is heavily inspired by moxi from couchbase, however we prefer to use plain memcached and have a bit more control of our data.

Unfortunately the most used memcached gem for rails (dalli) and the most performing proxy (nutcracker) don't talk to each other.

The proxy keeps a list of active memcached servers, rebalancing keys in case they go kaput.

How fast?

Fast enough.

Starting 100 clients running 500 requests each.
               user     system      total        real
moxiset:     2.250000   1.840000   4.090000 (  2.655543)
roxterset:   2.270000   1.630000   3.900000 (  2.585357)
moxiget:     2.400000   2.150000   4.550000 (  2.922284)
roxterget:   2.540000   1.800000   4.340000 (  2.826632)

Features left behind

• SASL / PLAIN authentication
• ASCII protocol (check twitter's nutcracker if you need this)

Installation

go get -u github.com/lxfontes/roxter/roxter

Usage

Assuming 2 memcache servers running on localhost 11210 and 11211:

roxter -bind ":11212" -server 127.0.0.1:11210 -server 127.0.0.1:11211

This will setup a listener on 11212 and split keys between 11210 and 11211.

ACK

• @bradfitz - Selector Idea
• @mncaudill - Ketama
• @uken

Documentation

Index

• Constants
• Variables
• type HashRing
  • func NewRing(n int) (h *HashRing)
  • func (h *HashRing) AddNode(n string, s int)
  • func (h *HashRing) Bake()
  • func (h *HashRing) Hash(s string) string
• type MCConn
  • func Dial(addr net.Addr) (*MCConn, error)
  • func (self *MCConn) Close() error
  • func (self *MCConn) Ping() bool
  • func (self *MCConn) RunSingle(req *MCMessage) (*MCMessage, error)
• type MCMessage
  • func (self *MCMessage) Marshal(buf *bytes.Buffer) error
  • func (self *MCMessage) Unmarshal(nc io.Reader) error
• type Proxy
  • func NewProxy(servers ...string) *Proxy
  • func NewProxyFromSelector(ss ServerSelector) *Proxy
  • func (self *Proxy) ListenAndServe(addr string) error
  • func (self *Proxy) Serve(l net.Listener) error
• type ServerList
  • func (self *ServerList) Monitor()
  • func (self *ServerList) PickServer(key string) (net.Addr, error)
  • func (self *ServerList) SetServers(servers ...string) error
• type ServerSelector

Constants

const DefaultTimeout = time.Duration(100) * time.Millisecond

Variables

var (
	ErrNotFound  = errors.New("mc: not found")
	ErrNoServers = errors.New("mc: no servers available")
	ErrTimeout   = errors.New("mc: connection timeout")
)

Types

type HashRing

type HashRing struct {
	DefaultSpots int
	Ticks        tickArray
	Length       int
}

func NewRing

func NewRing(n int) (h *HashRing)

func (*HashRing) AddNode

func (h *HashRing) AddNode(n string, s int)

Adds a new node to a hash ring n: name of the server s: multiplier for default number of ticks (useful when one cache node has more resources, like RAM, than another)

func (*HashRing) Bake

func (h *HashRing) Bake()

func (*HashRing) Hash

func (h *HashRing) Hash(s string) string

type MCConn

type MCConn struct {
	// contains filtered or unexported fields
}

func Dial

func Dial(addr net.Addr) (*MCConn, error)

func (*MCConn) Close

func (self *MCConn) Close() error

func (*MCConn) Ping

func (self *MCConn) Ping() bool

not meant to guarantee that memcache is not full simple ping

func (*MCConn) RunSingle

func (self *MCConn) RunSingle(req *MCMessage) (*MCMessage, error)

type MCMessage

type MCMessage struct {
	Header mcheader
	Extras []byte
	Key    string
	Value  []byte
}

func (*MCMessage) Marshal

func (self *MCMessage) Marshal(buf *bytes.Buffer) error

func (*MCMessage) Unmarshal

func (self *MCMessage) Unmarshal(nc io.Reader) error

type Proxy

type Proxy struct {
	MaxIdle int
	// contains filtered or unexported fields
}

func NewProxy

func NewProxy(servers ...string) *Proxy

func NewProxyFromSelector

func NewProxyFromSelector(ss ServerSelector) *Proxy

func (*Proxy) ListenAndServe

func (self *Proxy) ListenAndServe(addr string) error

func (*Proxy) Serve

func (self *Proxy) Serve(l net.Listener) error

type ServerList

type ServerList struct {
	// contains filtered or unexported fields
}

func (*ServerList) Monitor

func (self *ServerList) Monitor()

func (*ServerList) PickServer

func (self *ServerList) PickServer(key string) (net.Addr, error)

func (*ServerList) SetServers

func (self *ServerList) SetServers(servers ...string) error

type ServerSelector

type ServerSelector interface {
	PickServer(key string) (net.Addr, error)
	Monitor()
}