memcached

README

memcached

Simple memcached server.

###Usage go get github.com/Skipor/memcached/cmd/memcached memcached --help for all options memcached to start server on default port memcached -aof-name ./memcached.aof to start server with AOF persistence on memcached -aof-name ./memcached.aof -sync 2s sync log not on every command, but every passed period

Features

• No third party packages for app, server and cache logic.
  • That was tack constraint.
  • I have used only small github.com/facebookgo/stackerr package for errors with stack traces.
• Parallel reads, serialized writes.
• Items data sync.Pool recycle.
• Low allocation text protocol parse.
• AOF persistence with configurable sync options.
  • Every command can be synced, or sync can be done by ticker.
  • Low latency fast log rotation.
    • When log grows large, background goroutine takes fast snapshot under read lock.
    • Snapshot is writen without log lock.
• Many unit and integration tests.

TODO

• LRU crawler
  • goroutine waking up on ticker; take RLock; start goroutine per queue; find expired; save them; take write lock and remove them.
• Refactor integration load tester. Do more load tests and comparation with original memcached.
• Test CLI helper functions
• RDB persistence
  • point-in-time snapshots of your dataset at specified intervals
  • current version already use snapshots to archive large AOF, so it is easy to implement such feature
• Profiling with pprof and optimisation
• Snapshot consistency check
  • Now check happen only in build with debug tag
• Optional
  • replace state snapshot usage as AOF rotation

Documentation

Index

• Constants
• Variables
• type Config
• type ConnMeta
• type CorruptedError
  • func (e *CorruptedError) Error() string
• type Server
  • func NewServer(conf Config) (s *Server, err error)
  • func (s *Server) ListenAndServe() error
  • func (s *Server) Serve(l net.Listener) error
  • func (s *Server) Stop()

Constants

const (
	MaxKeySize         = 250
	MaxItemSize        = 128 * (1 << 20) // 128 MB.
	DefaultMaxItemSize = 1 << 20
	MaxCommandSize     = 1 << 12

	MaxRelativeExptime = 60 * 60 * 24 * 30 // 30 days.

	Separator = "\r\n"

	SetCommand    = "set"
	GetCommand    = "get"
	GetsCommand   = "gets"
	DeleteCommand = "delete"

	NoReplyOption = "noreply"

	StoredResponse      = "STORED"
	ValueResponse       = "VALUE"
	EndResponse         = "END"
	DeletedResponse     = "DELETED"
	NotFoundResponse    = "NOT_FOUND"
	ErrorResponse       = "ERROR"
	ClientErrorResponse = "CLIENT_ERROR"
	ServerErrorResponse = "SERVER_ERROR"

	// Implementation specific consts.
	InBufferSize  = 16 * (1 << 10)
	OutBufferSize = 16 * (1 << 10)
)

const DefaultAddr = ":11211"

const SnapshotCommand = "\x00 LOG FILE STARTS WITH GOB ENCODED CACHE SNAPSHOT \x00" + Separator

SnapshotCommand indicate that there is cache snapshot after it. First byte is invalid for any memcached command, so it is possible to understand is command SnapshotCommand by first byte.

Variables

var (
	ErrTooLargeKey          = errors.New("too large key")
	ErrTooLargeItem         = errors.New("too large item")
	ErrInvalidOption        = errors.New("invalid option")
	ErrTooManyFields        = errors.New("too many fields")
	ErrMoreFieldsRequired   = errors.New("more fields required")
	ErrTooLargeCommand      = errors.New("command length is too big")
	ErrEmptyCommand         = errors.New("empty command")
	ErrFieldsParseError     = errors.New("fields parse error ")
	ErrInvalidLineSeparator = errors.New("invalid line separator")
	ErrInvalidCharInKey     = errors.New("key contains invalid characters")
)

var ErrStoped = errors.New("memcached server have been stoped")

Types

type Config

type Config struct {
	Addr           string
	LogDestination io.Writer
	LogLevel       log.Level

	MaxItemSize int64
	Cache       cache.Config

	FixCorruptedAOF bool
	AOF             aof.Config
}

type ConnMeta

type ConnMeta struct {
	Pool        *recycle.Pool
	MaxItemSize int
}

connMeta is data shared between connections.

type CorruptedError

type CorruptedError struct {
	Err error
}

func (*CorruptedError) Error

func (e *CorruptedError) Error() string

type Server

type Server struct {
	ConnMeta
	Addr         string
	Log          log.Logger
	NewCacheView func() cache.View
	// contains filtered or unexported fields
}

Server serves memcached text protocol over tcp. Only Cache field is required, other have reasonable defaults.

func NewServer

func NewServer(conf Config) (s *Server, err error)

func (*Server) ListenAndServe

func (s *Server) ListenAndServe() error

func (*Server) Serve

func (s *Server) Serve(l net.Listener) error

func (*Server) Stop

func (s *Server) Stop()