hraftd

package module
v0.0.0-...-d605994 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 16, 2021 License: MIT Imports: 32 Imported by: 0

README

hraftd

hraftd is a reference usage of the Hashicorp Raft. Read more about raft brief.

Hashicorp provide a nice implementation of the Raft consensus protocol, and it’s at the heart of InfluxDB, consul (amongst other systems).

It is rewritten from otoolep/hraftd and this blog post.

image

Raft is a distributed consensus protocol, meaning its purpose is to ensure that a set of nodes -- a cluster -- agree on the state of some arbitrary state machine, even when nodes are vulnerable to failure and network partitions.

Distributed consensus is a fundamental concept when it comes to building fault-tolerant systems.

A simple example like hraftd makes it easy to study the Raft consensus protocol in general, and Hashicorp's Raft implementation in particular. It can be run on Linux, OSX, and Windows.

Reading and writing keys

The reference implementation is a very simple in-memory key-value store. You can set/get a key by sending a request to the HTTP bind address (which defaults to localhost:11000):

curl -v -H "Content-Type: application/json" localhost:11000/hraftd/key -d '{"foo": "bar"}'
curl localhost:11000/hraftd/key/foo

or use httpie

http :11000/hraftd/key foo=barst
http :11000/hraftd/key/foo

Running hraftd

Starting and running a hraftd cluster is easy. Download hraftd like so:

go get github.com/bingoohuang/hraftd/cmd/hraftd

Run your first hraftd node like so:

hraftd

You can now set a key and read its value back:

curl -v -H "Content-Type: application/json" localhost:11000/hraftd/key -d '{"user1": "batman"}'
curl localhost:11000/hraftd/key/user1
Bring up a cluster locally

Let's bring up 3-node cluster. That way we can tolerate the failure of 1 node:

hraftd --rjoin :11000 --haddr :11000
hraftd --rjoin :11000 --haddr :11001
hraftd --rjoin :11000 --haddr :11002

or user supervisord

supervisord -c supervisord.ini

This example shows each hraftd node running on the same host, so each node must listen on different ports. This would not be necessary if each node ran on a different host.

This tells each new node to join the existing node. Once joined, each node now knows about the key:

curl localhost:11000/hraftd/key/user1 localhost:11001/hraftd/key/user1 localhost:11002/hraftd/key/user1

Furthermore, you can add a second key:

curl localhost:11000/hraftd/key -d '{"user2": "robin"}'

Confirm that the new key has been set like so:

$ curl localhost:11000/hraftd/key/user2 localhost:11001/hraftd/key/user2 localhost:11002/hraftd/key/user2
{"user2":"robin"}{"user2":"robin"}{"user2":"robin"}
$ curl localhost:11000/hraftd/raft/stats |jq

output:

{
  "applied_index": "7",
  "commit_index": "7",
  "fsm_pending": "0",
  "last_contact": "3.685538ms",
  "last_log_index": "7",
  "last_log_term": "8",
  "last_snapshot_index": "0",
  "last_snapshot_term": "0",
  "latest_configuration": [
    {
      "id": "192.168.10.101:11001,192.168.10.101:12001",
      "address": "192.168.10.101:12001",
      "suffrage": "Voter"
    },
    {
      "id": "192.168.10.101:11003,192.168.10.101:12003",
      "address": "192.168.10.101:12003",
      "suffrage": "Voter"
    },
    {
      "id": "192.168.10.101:11004,192.168.10.101:12004",
      "address": "192.168.10.101:12004",
      "suffrage": "Voter"
    },
    {
      "id": "192.168.10.101:11002,192.168.10.101:12002",
      "address": "192.168.10.101:12002",
      "suffrage": "Voter"
    }
  ],
  "latest_configuration_index": "0",
  "num_peers": "2",
  "protocol_version": "3",
  "protocol_version_max": "3",
  "protocol_version_min": "0",
  "snapshot_version_max": "1",
  "snapshot_version_min": "0",
  "state": "Follower",
  "term": "8"
}

curl localhost:11000/hraftd/raft/cluster | python -m json.tool

or

http :11000/hraftd/raft/cluster

output:

{
  "current": {
    "address": ":12000",
    "id": ":11000,:12000",
    "state": "Follower"
  },
  "leader": {
    "address": ":12002",
    "id": ":11002,:12002",
    "state": "Leader"
  },
  "servers": [
    {
      "address": ":12000",
      "id": ":11000,:12000",
      "state": "Follower"
    },
    {
      "address": ":12001",
      "id": ":11001,:12001",
      "state": "Follower"
    },
    {
      "address": ":12002",
      "id": ":11002,:12002",
      "state": "Leader"
    }
  ]
}
Multi-node Clustering realistically

What follows is a detailed example of running a multi-node hraftd cluster.

Imagine you have 3 machines, with the IP addresses 192.168.0.1, 192.168.0.2, and 192.168.0.3 respectively. Let's also assume that each machine can reach the other two machines using these addresses.

You should start the nodes (eg at 192.168.0.1,192.168.0.2,192.168.0.3) like so:

hraftd --haddr :11000 --rjoin 192.168.0.1:11000

Specifically using ports 11000 and 12000 is not required. You can use other ports if you wish.

Note how each node listens on its own address, but joins to the address of the leader node. The second and third nodes will start, join the leader at 192.168.0.2:11000, and a 3-node cluster will be formed.

$ lsof -i tcp:11000
COMMAND     PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
XiYouProx  5344 bingoo  115u  IPv4 0x8708c2e9ace67df5      0t0  TCP localhost:59712->localhost:irisa (ESTABLISHED)
hraftd    73100 bingoo    9u  IPv6 0x8708c2e9bc6d07ad      0t0  TCP *:irisa (LISTEN)
hraftd    73100 bingoo   11u  IPv6 0x8708c2e9bc6d0dcd      0t0  TCP localhost:irisa->localhost:59712 (ESTABLISHED)

$ ps -ef|grep raft
  502 73100 35405   0 10:39上午 ttys001    0:00.21 hraftd -haddr :11000 -raddr :12000 -rjoin :11000
  502 72406 46043   0 10:30上午 ttys008    0:02.41 hraftd -haddr :11001 -raddr :12001 -rjoin :11000
  502 72405 46094   0 10:30上午 ttys009    0:06.71 hraftd -haddr :11002 -raddr :12002 -rjoin :11000

remove node from cluster:

$ http -v DELETE :11001/hraftd/raft/remove id=192.168.10.101:11004,192.168.10.101:12004
DELETE /raft/remove HTTP/1.1
Accept: application/json, */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 51
Content-Type: application/json
Host: localhost:11001
User-Agent: HTTPie/1.0.2

{
    "id": "192.168.10.101:11004,192.168.10.101:12004"
}

HTTP/1.1 200 OK
Content-Length: 34
Content-Type: application/json; charset=utf-8
Date: Thu, 13 Feb 2020 15:49:15 GMT

{
    "msg": "OK",
    "ok": true
}
Stale reads

Because any node will answer a GET request, and nodes may "fall behind" updates, stale reads are possible. Again, hraftd is a simple program, for the purpose of demonstrating a distributed key-value store. If you are particularly interested in learning more about issue, you should check out rqlite. rqlite allows the client to control read consistency, allowing the client to trade off read-responsiveness and correctness.

Read-consistency support could be ported to hraftd if necessary.

Tolerating failure

Kill the leader process and watch one of the other nodes be elected leader. The keys are still available for query on the other nodes, and you can set keys on the new leader. Furthermore, when the first node is restarted, it will rejoin the cluster and learn about any updates that occurred while it was down.

A 3-node cluster can tolerate the failure of a single node, but a 5-node cluster can tolerate the failure of two nodes. But 5-node clusters require that the leader contact a larger number of nodes before any change e.g. setting a key's value, can be considered committed.

Leader-forwarding

Automatically forwarding requests to set keys to the current leader is not implemented. The client must always send requests to change a key to the leader or an error will be returned.

Production use of Raft

For a production-grade example of using Hashicorp's Raft implementation, to replicate a SQLite database, check out rqlite.

Resources

  1. Raft 论文 - 中文译版
  2. The Raft Consensus Algorithm，包括可视化，包括节点状态
  3. Raft算法原理
  4. 基于 hashicorp/raft 的分布式一致性实战教学
  5. raft 可视化
  6. Leto - 基于 raft 快速实现一个 key-value 存储系统
  7. 使用 Raft 实现 VIP 功能
  8. Raft implementation in Go, 信息入口在twitter上

Documentation

Index

Constants

View Source 
const (
	// RaftPath is the http prefix for raft apis
	RaftPath = "/hraftd/raft"
	// KeyPath is the http prefix for raft key operations
	KeyPath = "/hraftd/key"
	// DoJobPath is the http prefix for raft dojob operations
	DoJobPath = "/hraftd/dojob"
)
View Source 
const ContentTypeJSON = "application-type/json"

ContentTypeJSON is the JSON Content-Type.

View Source 
const StateFollower = "Follower"

StateFollower is the state string for follower.

View Source 
const StateLeader = "Leader"

StateLeader is the state string for leader.

View Source 
const XOriginRemoteAddr = "X-Origin-RemoteAddr"

XOriginRemoteAddr is the const of header for original remote addr to real address sourcing.

Variables

View Source 
var (
	// ErrDealerNoExists is the error for the dealer not exists.
	ErrDealerNoExists = errors.New("dealer does not exist")
	// ErrDealerContinue is the error for the dealer bypass and should continue
	ErrDealerContinue = errors.New("dealer bypass and should continue")
)
View Source 
var (
	// ErrNotLeader is returned when a node attempts to execute a leader-only operation.
	ErrNotLeader = errors.New("not leader")
	// ErrOpenTimeout is returned when the Store does not apply its initial logs within the specified time.
	ErrOpenTimeout = errors.New("timeout waiting for initial logs application")
)
View Source 
var DefaultLogger = NewSLogger()

DefaultLogger is the default global logger. nolint

Functions

func CheckMethod 

func CheckMethod(m string, f ServeHTTPFn, w http.ResponseWriter, r *http.Request)

CheckMethod checks the method and invoke f.

func CheckMethodE 

func CheckMethodE(m string, f ServeHTTPFnE, w http.ResponseWriter, r *http.Request)

CheckMethodE checks the method and invoke f.

func CloneMap 

func CloneMap(m map[string]string) map[string]string

CloneMap clones a map.

func EmptyThen 

func EmptyThen(s, t string) string

EmptyThen returns t if s is empty.

func EqualsThen 

func EqualsThen(s, e, t string) string

EqualsThen returns t if s equals to e.

func FormatTime 

func FormatTime(t time.Time) string

FormatTime format time. FormatTime format time.

func GetJSON 

func GetJSON(addr string, v interface{}) (string, error)

GetJSON does HTTP GET parse response as JSON.

func HasPrefix 

func HasPrefix(s string, p ...string) bool

HasPrefix tells s has any prefix of p...

func If 

func If(i bool, s, t string) string

If returns s if i else t.

func IsRelativeForward 

func IsRelativeForward(statusCode int, locationHeader string) bool

IsRelativeForward tells the statusCode is 301/302 and locationHeader is relative.

func Join 

func Join(logger LevelLogger, joinAddr, raftAddr string, nodeID NodeID) error

Join joins current node (raftAddr and nodeID) to joinAddr.

func Jsonify 

func Jsonify(v interface{}) string

Jsonify jsonifies v to JSON string.

func Jsonify4Print 

func Jsonify4Print(v interface{}) string

Jsonify4Print jsonifies v to JSON string for printing only.

func JsonifyBytes 

func JsonifyBytes(v interface{}) []byte

JsonifyBytes jsonifies v to JSON []byte.

func ParseIps 

func ParseIps(host string) []string

ParseIps parses IP addresses from the host string which maybe an IP or domain name.

func ParseJSON 

func ParseJSON(v string) (m interface{}, err error)

ParseJSON parses string to v.

func ParseSuffrage 

func ParseSuffrage(s string) raft.ServerSuffrage

ParseSuffrage parses s to raft.ServerSuffrage.

func ParseTime 

func ParseTime(s string) (time.Time, error)

ParseTime parses time.

func PathExists 

func PathExists(p string) bool

PathExists returns true if the given path exists.

func PostJSON 

func PostJSON(addr string, body, v interface{}) (int, string, error)

PostJSON posts body as JSON and parse response as JSON.

func ReadBytes 

func ReadBytes(object io.ReadCloser) []byte

ReadBytes ...

func ReadPeersJSON 

func ReadPeersJSON(path string) (raft.Configuration, error)

ReadPeersJSON consumes a legacy peers.json file in the format of the old JSON peer store and creates a new-style configuration structure. This can be used to migrate this data or perform manual recovery when running protocol versions that can interoperate with older, unversioned Raft servers. This should not be used once server IDs are in use, because the old peers.json file didn't have support for these, nor non-voter suffrage types.

func ReadString 

func ReadString(object io.ReadCloser) string

ReadString ...

func ReverseProxy 

func ReverseProxy(targetHost, targetPath string, timeout time.Duration) *httputil.ReverseProxy

ReverseProxy reverse proxy originalPath to targetHost with targetPath.

func TryParseJSON 

func TryParseJSON(v string) interface{}

TryParseJSON try parse v as JSON，return parsed object or failed to source v.

func WaitInterrupt 

func WaitInterrupt()

WaitInterrupt waits on interrupt signal.

func WriteAsJSON 

func WriteAsJSON(m interface{}, w http.ResponseWriter)

WriteAsJSON writes m as JSON.

func WriteAsText 

func WriteAsText(s string, w http.ResponseWriter)

WriteAsText writes s as text/plain.

Types

type ApplyInterceptor 

type ApplyInterceptor func(l *raft.Log, cmd Command) bool

ApplyInterceptor defines the raft log apply interceptor prototype.

type Arg 

type Arg struct {
	Bootstrap bool

	InMem       bool
	RaftAddr    string
	RaftAdv     string
	RaftNodeDir string
	NodeID      NodeID
	HTTPAddr    string
	HTTPAdv     string
	JoinAddrs   string

	IfaceName string // 绑定网卡名称

	JoinAddrSlice []string

	HostIP string

	ApplyInterceptor ApplyInterceptor `json:"-"`
	LogDealer        `json:"-"`
	LoggerMore       `json:"-"`
}

Arg Command line parameters.

func CreateArg 

func CreateArg(p ViperProvider, flagOptionFns ...FlagOptionFn) *Arg

CreateArg creates Arg by ViperProvider implementation.

func DefineFlags 

func DefineFlags(p FlagProvider, flagOptionFns ...FlagOptionFn) *Arg

DefineFlags define raft args.

func MakeArg 

func MakeArg() *Arg

MakeArg makes a Arg.

func (*Arg) ConvertToZeroHost 

func (a *Arg) ConvertToZeroHost(addr string) string

ConvertToZeroHost tries to bind localip:port to :port.

func (*Arg) Fix 

func (a *Arg) Fix()

Fix fixes the arg for some defaults.

func (*Arg) Intercept 

func (a *Arg) Intercept(l *raft.Log, c Command) (interface{}, bool)

Intercept intercepts the raft log applying.

func (*Arg) Join 

func (a *Arg) Join() error

Join joins the current not to raft cluster.

type BindAddr 

type BindAddr string

BindAddr is the address for bind.

func (BindAddr) URL 

func (a BindAddr) URL(path string) string

URL returns the HTTP access URL with relative path.

func (BindAddr) URLRaftJoin 

func (a BindAddr) URLRaftJoin() string

URLRaftJoin is http://httpAddr/raft/join

type Command 

type Command struct {
	Op    string `json:"op,omitempty"`
	Key   string `json:"key,omitempty"`
	Value string `json:"value,omitempty"`
	Time  string `json:"time"`
}

Command defines raft log value's structure.

type ConfigEntry 

type ConfigEntry struct {
	// ID is the ID of the server (a UUID, usually).
	ID raft.ServerID `json:"id"`

	// Address is the host:port of the server.
	Address raft.ServerAddress `json:"address"`

	// NonVoter controls the suffrage. We choose this sense so people
	// can leave this out and get a Voter by default.
	Suffrage string `json:"suffrage"`
}

ConfigEntry is used when decoding a new-style peers.json.

type Dealer 

type Dealer struct {
	Fn      reflect.Value
	ReqType reflect.Type
}

Dealer defines the job dealer structure.

type DealerMap 

type DealerMap struct {
	Dealers map[string]Dealer
}

DealerMap keep mapping dealer path to registered dealer.

func MakeDealerMap 

func MakeDealerMap() DealerMap

MakeDealerMap makes a DealerMap.

func (*DealerMap) Invoke 

func (s *DealerMap) Invoke(dealerName string, requestBody []byte) (x interface{}, err error)

Invoke invokes the registered dealer function.

func (*DealerMap) RegisterJobDealer 

func (s *DealerMap) RegisterJobDealer(dealerName string, dealer interface{}) error

RegisterJobDealer registers path dealers.

type Dialer 

type Dialer func(ctx context.Context, net, addr string) (c net.Conn, err error)

Dialer defines dialer function alias.

func TimeoutDialer 

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) Dialer

TimeoutDialer returns functions of connection dialer with timeout settings for http.Transport Dial field. https://gist.github.com/c4milo/275abc6eccbfd88ad56ca7c77947883a HTTP client with support for read and write timeouts which are missing in Go's standard library.

type DistributedApplier 

type DistributedApplier interface {
	Distribute(NodeID NodeID, item Identifier)
}

DistributedApplier is the data applier with NodeID.

type Distributor 

type Distributor struct {
	// sticky to the previous nodeID when redistribute every time.
	StickyMap map[string]NodeID
}

Distributor is the role to charge the distribution among the hraft cluster nodes.

func NewDistributor 

func NewDistributor() *Distributor

NewDistributor makes a new Distributor.

func (*Distributor) CleanSticky 

func (d *Distributor) CleanSticky()

CleanSticky cleans the sticky map state.

func (*Distributor) Distribute 

func (d *Distributor) Distribute(peers []Peer, data, emptyReceiver interface{}) interface{}

Distribute do the distribution.

func (*Distributor) Put 

func (d *Distributor) Put(id string, nodeID NodeID)

Put puts the node ID related to id directly.

type FlagNames 

type FlagNames struct {
	Rmem  string `default:"rmem"`
	Haddr string `default:"haddr"`
	Hadv  string `default:"hadv"`
	Raddr string `default:"raddr"`
	Radv  string `default:"radv"`
	Rdir  string `default:"rdir"`
	Rjoin string `default:"rjoin"`
	Iface string `default:"iface"`
}

FlagNames defines struct for flag names.

type FlagOptionFn 

type FlagOptionFn func(flagNames *FlagNames)

FlagOptionFn defines FlagOption option func prototype.

func FlagHaddr 

func FlagHaddr(name string) FlagOptionFn

FlagHaddr defines HTTPAddr flag name. If empty, disable the flag.

func FlagHadv 

func FlagHadv(name string) FlagOptionFn

FlagHadv defines HTTPAdv flag name. If empty, disable the flag.

func FlagIface 

func FlagIface(name string) FlagOptionFn

FlagIface defines IfaceName flag name. If empty, disable the flag.

func FlagRaddr 

func FlagRaddr(name string) FlagOptionFn

FlagRaddr defines RaftAddr flag name. If empty, disable the flag.

func FlagRadv 

func FlagRadv(name string) FlagOptionFn

FlagRadv defines RaftAdv flag name. If empty, disable the flag.

func FlagRdir 

func FlagRdir(name string) FlagOptionFn

FlagRdir defines RaftNodeDir flag name. If empty, disable the flag.

func FlagRjoin 

func FlagRjoin(name string) FlagOptionFn

FlagRjoin defines JoinAddrs flag name. If empty, disable the flag.

func FlagRmem 

func FlagRmem(name string) FlagOptionFn

FlagRmem defines InMem flag name. If empty, disable the flag.

type FlagProvider 

type FlagProvider interface {
	BoolVar(p *bool, name string, value bool, usage string)
	StringVar(p *string, name string, value string, usage string)
}

FlagProvider defines the interface for flag definitions required for hraftd.

type Identifier 

type Identifier interface {
	ID() string
}

Identifier gives the ID getter.

type JobRsp 

type JobRsp struct {
	OK   bool                `json:"ok"`
	Msg  string              `json:"msg,omitempty"`
	Data jsoniter.RawMessage `json:"data,omitempty"`
}

JobRsp defines the Job Response structure.

type JoinRequest 

type JoinRequest struct {
	Addr   string `json:"addr"`
	NodeID NodeID `json:"id"`
}

JoinRequest defines the Raft join request.

func (*JoinRequest) Fix 

func (r *JoinRequest) Fix(remoteAddr string)

Fix fixes the join request's host.

type LevelLogger 

type LevelLogger interface {
	// Printf prints info.
	Printf(format string, data ...interface{})
	// Debugf prints debug.
	Debugf(format string, data ...interface{})
	// Infof prints info.
	Infof(format string, data ...interface{})
	// Warnf prints warn messages.
	Warnf(format string, data ...interface{})
	// Errorf prints error messages.
	Errorf(format string, data ...interface{})
	// Panicf prints error messages and panics.
	Panicf(format string, data ...interface{})
}

LevelLogger ...

type LevelLoggerAdapter 

type LevelLoggerAdapter struct{ Logger }

LevelLoggerAdapter adapters Logger to LevelLogger.

func NewLogrusAdapter 

func NewLogrusAdapter(logrus *logrus.Logger) *LevelLoggerAdapter

NewLogrusAdapter news a LogrusAdapter.

func (LevelLoggerAdapter) Debugf 

func (l LevelLoggerAdapter) Debugf(format string, data ...interface{})

Debugf prints debug.

func (LevelLoggerAdapter) Errorf 

func (l LevelLoggerAdapter) Errorf(format string, data ...interface{})

Errorf prints error messages.

func (LevelLoggerAdapter) Infof 

func (l LevelLoggerAdapter) Infof(format string, data ...interface{})

Infof prints info.

func (LevelLoggerAdapter) Panicf 

func (l LevelLoggerAdapter) Panicf(format string, data ...interface{})

Panicf prints error messages and panic.

func (LevelLoggerAdapter) Printf 

func (l LevelLoggerAdapter) Printf(format string, data ...interface{})

Printf prints info.

func (LevelLoggerAdapter) Warnf 

func (l LevelLoggerAdapter) Warnf(format string, data ...interface{})

Warnf prints warn messages.

type LogDealer 

type LogDealer struct {
	DealerMap
}

LogDealer defines the structure of log dealer.

func MakeLogDealer 

func MakeLogDealer() LogDealer

MakeLogDealer makes a LogDealer.

func (*LogDealer) RegisterLogDealer 

func (l *LogDealer) RegisterLogDealer(cmdName string, dealerFn interface{}) error

RegisterLogDealer registers the dealer for the command which name is cmdName.

type LogLevel 

type LogLevel int

LogLevel defines log levels. 

const (
	// LogLevelDebug debug log level.
	LogLevelDebug LogLevel = iota
	// LogLevelInfo info log level.
	LogLevelInfo
	// LogLevelWarn warn log level.
	LogLevelWarn
	// LogLevelError error log level.
	LogLevelError
)

func ParseLogLevel 

func ParseLogLevel(l string) LogLevel

ParseLogLevel parses log level.

func (LogLevel) String 

func (l LogLevel) String() string

type Logger 

type Logger interface {
	// SetLogLevel sets the log level.
	SetLogLevel(logLevel LogLevel)
	// GetLogLevel returns the log level.
	GetLogLevel() LogLevel
	// GetIOWriter returns io.Writer.
	GetIOWriter() io.Writer
	// Logf prints log.
	Logf(level LogLevel, format string, data ...interface{})
}

Logger defines logger interface.

type LoggerMore 

type LoggerMore interface {
	Logger
	LevelLogger
}

LoggerMore ...

type LogrusAdapter 

type LogrusAdapter struct {
	Logrus *logrus.Logger
}

LogrusAdapter adapts the logrus to Logger.

func (LogrusAdapter) GetIOWriter 

func (l LogrusAdapter) GetIOWriter() io.Writer

GetIOWriter returns io.Writer.

func (LogrusAdapter) GetLogLevel 

func (l LogrusAdapter) GetLogLevel() LogLevel

GetLogLevel returns the log level.

func (LogrusAdapter) Logf 

func (l LogrusAdapter) Logf(logLevel LogLevel, format string, data ...interface{})

Logf prints log.

func (*LogrusAdapter) SetLogLevel 

func (l *LogrusAdapter) SetLogLevel(logLevel LogLevel)

SetLogLevel sets the log level.

type NodeID 

type NodeID string

NodeID is the raft node ID.

func (*NodeID) Fix 

func (r *NodeID) Fix(host string)

Fix fixes the ID component to full host:port.

func (NodeID) HTTPAddr 

func (r NodeID) HTTPAddr() string

HTTPAddr returns the HTTP bind address in the ID.

func (NodeID) RaftAddr 

func (r NodeID) RaftAddr() string

RaftAddr returns the Raft bind addr in the ID.

func (NodeID) URL 

func (r NodeID) URL(relativePath string) string

URL returns the HTTP access URL with relative path.

func (NodeID) URLRaftCluster 

func (r NodeID) URLRaftCluster() string

URLRaftCluster is http://httpAddr/raft/cluster.

func (NodeID) URLRaftJoin 

func (r NodeID) URLRaftJoin() string

URLRaftJoin is http://httpAddr/raft/join.

func (NodeID) URLRaftState 

func (r NodeID) URLRaftState() string

URLRaftState is http://httpAddr/raft/state.

type NodeState 

type NodeState struct {
	StartTime string `json:"startTime"`
	NodeID    string `json:"nodeID"`
	Hostname  string `json:"hostname"`
	IP        string `json:"IP"`
}

NodeState is raft cluster node state.

type Peer 

type Peer struct {
	Address  string `json:"address"`
	ID       NodeID `json:"id"`
	State    string `json:"state"`
	Suffrage string `json:"suffrage"`
}

Peer defines the peers information.

func (Peer) AnyStateOf 

func (p Peer) AnyStateOf(states ...string) bool

AnyStateOf returns true if current state of peer is in any of states.

func (Peer) DistributeJob 

func (p Peer) DistributeJob(logger LevelLogger, path string, req interface{}, rsp interface{}) error

DistributeJob distributes job to the peer node in the raft clusters.

func (Peer) IsActive 

func (p Peer) IsActive() bool

IsActive means the peer is active state (leader or follower) in the cluster.

type RaftCluster 

type RaftCluster struct {
	Current Peer   `json:"current"`
	Leader  Peer   `json:"leader"`
	Servers []Peer `json:"servers"`
}

RaftCluster is raft cluster.

func Cluster 

func Cluster(logger LevelLogger, nodeID NodeID) (v RaftCluster, err error)

Cluster retrieves the RaftCluster. nolint:gofumpt

func (RaftCluster) ActivePeers 

func (r RaftCluster) ActivePeers() []Peer

ActivePeers returns active peers in the cluster.

type RaftStore 

type RaftStore struct {
	*Arg
	// contains filtered or unexported fields
}

RaftStore is a simple key-value store, where all changes are made via Raft consensus.

func New 

func New(arg *Arg) *RaftStore

New returns a new Store.

func (*RaftStore) Apply 

func (s *RaftStore) Apply(l *raft.Log) interface{}

Apply applies a Raft log entry to the key-value store.

func (*RaftStore) Cluster 

func (s *RaftStore) Cluster() (RaftCluster, error)

Cluster returns the raft cluster state.

func (*RaftStore) Delete 

func (s *RaftStore) Delete(key string) error

Delete deletes the given key.

func (*RaftStore) Get 

func (s *RaftStore) Get(key string) (v string, ok bool)

Get returns the value for the given key.

func (*RaftStore) IsLeader 

func (s *RaftStore) IsLeader() bool

IsLeader tells the current node is raft leader or not.

func (*RaftStore) Join 

func (s *RaftStore) Join(nodeID, addr string) error

Join joins a node, identified by nodeID and located at addr, to this store. The node must be ready to respond to Raft communications at that address.

func (*RaftStore) LeadServer 

func (s *RaftStore) LeadServer() (Peer, error)

LeadServer returns the raft lead server.

func (*RaftStore) LeaderAddr 

func (s *RaftStore) LeaderAddr() string

LeaderAddr returns the address of the current leader. Returns blank if no leader.

func (*RaftStore) LeaderCh 

func (s *RaftStore) LeaderCh() <-chan bool

LeaderCh is used to get a channel which delivers signals on acquiring or losing leadership. It sends true if we become the leader, and false if we lose it. The channel is not buffered, and does not block on writes.

func (*RaftStore) NodeState 

func (s *RaftStore) NodeState() string

NodeState returns the state of current node.

func (*RaftStore) Open 

func (s *RaftStore) Open() error

Open opens the store. If enableSingle is set, and there are no existing peers, then this node becomes the first node, and therefore leader, of the cluster. localID should be the server identifier for this node.

func (*RaftStore) RaftStats 

func (s *RaftStore) RaftStats() map[string]interface{}

RaftStats returns raft stats.

func (*RaftStore) Remove 

func (s *RaftStore) Remove(nodeID string) error

Remove removes the node, with the given nodeID, from the cluster.

func (*RaftStore) Restore 

func (s *RaftStore) Restore(rc io.ReadCloser) error

Restore stores the key-value store to a previous state.

func (*RaftStore) Set 

func (s *RaftStore) Set(key, value string) error

Set sets the value for the given key.

func (*RaftStore) Snapshot 

func (s *RaftStore) Snapshot() (raft.FSMSnapshot, error)

Snapshot returns a snapshot of the key-value store.

func (*RaftStore) WaitForApplied 

func (s *RaftStore) WaitForApplied(timeout time.Duration) error

WaitForApplied waits for all Raft log entries to to be applied to the underlying database.

func (*RaftStore) WaitForAppliedIndex 

func (s *RaftStore) WaitForAppliedIndex(idx uint64, timeout time.Duration) error

WaitForAppliedIndex blocks until a given log index has been applied, or the timeout expires.

func (*RaftStore) WaitForLeader 

func (s *RaftStore) WaitForLeader(timeout time.Duration) (string, error)

WaitForLeader blocks until a leader is detected, or the timeout expires.

type Rsp 

type Rsp struct {
	OK   bool        `json:"ok"`
	Msg  string      `json:"msg,omitempty"`
	Data interface{} `json:"data,omitempty"`
}

Rsp defines the Raft join response.

type SLogger 

type SLogger struct {
	Writer
	IOWriter io.Writer
	Level    LogLevel
	LevelLogger
}

SLogger defines the simplest logger implementation of Interface.

func NewSLogger 

func NewSLogger() *SLogger

NewSLogger creates a new SLogger.

func (*SLogger) GetIOWriter 

func (l *SLogger) GetIOWriter() io.Writer

GetIOWriter returns io.Writer.

func (SLogger) GetLogLevel 

func (l SLogger) GetLogLevel() LogLevel

GetLogLevel gets the log level.

func (SLogger) Logf 

func (l SLogger) Logf(level LogLevel, format string, data ...interface{})

Logf prints log.

func (*SLogger) SetLogLevel 

func (l *SLogger) SetLogLevel(level LogLevel)

SetLogLevel sets the log level.

type ServeHTTPFn 

type ServeHTTPFn func(w http.ResponseWriter, r *http.Request)

ServeHTTPFn defines ServeHTTP function prototype.

type ServeHTTPFnE 

type ServeHTTPFnE func(w http.ResponseWriter, r *http.Request) error

ServeHTTPFnE defines ServeHTTP function prototype.

type Service 

type Service struct {
	Store Store
	Ln    net.Listener
	*Arg

	LeaderCh chan bool
	DealerMap
}

Service provides HTTP service.

func Create 

func Create(arg *Arg) *Service

Create returns an uninitialized service.

func (*Service) Addr 

func (s *Service) Addr() net.Addr

Addr returns the address on which the Service is listening.

func (*Service) Close 

func (s *Service) Close() error

Close closes the service.

func (*Service) GoStartHTTP 

func (s *Service) GoStartHTTP() (err error)

GoStartHTTP starts the http server in go routine.

func (*Service) IsLeader 

func (s *Service) IsLeader() bool

IsLeader tells the current node is raft leader or not.

func (*Service) RaftCluster 

func (s *Service) RaftCluster() (RaftCluster, error)

RaftCluster returns raft cluster.

func (*Service) ServeHTTP 

func (s *Service) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP allows Service to serve HTTP requests.

func (*Service) Set 

func (s *Service) Set(k, v string) error

Set sets the key-value to the raft log.

func (*Service) StartAll 

func (s *Service) StartAll() error

StartAll starts the http and raft service.

func (*Service) StartRaft 

func (s *Service) StartRaft() error

StartRaft starts the raft service.

type StdLogger 

type StdLogger struct {
	Out io.Writer // destination for output

	CallDepth       int  // used for print logger file and line number
	PrintCallerInfo bool // switcher to print caller info
	// contains filtered or unexported fields
}

A StdLogger represents an active logging object that generates lines of output to an io.Writer. Each logging operation makes a single call to the Writer's Write method. A Logger can be used simultaneously from multiple goroutines; it guarantees to serialize access to the Writer.

func NewStdLogger 

func NewStdLogger(out io.Writer) *StdLogger

NewStdLogger creates a new Logger. The out variable sets the destination to which log data will be written. The prefix appears at the beginning of each generated log line. The flag argument defines the logging properties.

func (*StdLogger) Output 

func (l *StdLogger) Output(s string) error

Output writes the output for a logging event. The string s contains the text to print after the prefix specified by the flags of the Logger. A newline is appended if the last character of s is not already a newline. CallDepth is used to recover the PC and is provided for generality, although at the moment on all pre-defined paths it will be 2.

func (*StdLogger) Print 

func (l *StdLogger) Print(v ...interface{})

Print calls l.Output to print to the logger. Arguments are handled in the manner of fmt.Print. nolint:gofumpt

type Store 

type Store interface {
	// Get returns the value for the given key.
	Get(key string) (string, bool)

	// IsLeader tells the current node is raft leader or not.
	IsLeader() bool

	// Set sets the value for the given key, via distributed consensus.
	Set(key, value string) error

	// Delete removes the given key, via distributed consensus.
	Delete(key string) error

	// Join joins the node, identified by nodeID and reachable at addr, to the cluster.
	Join(nodeID string, addr string) error

	// Remove removes node from the cluster
	Remove(nodeID string) error

	// RaftStats returns the raft stats
	RaftStats() map[string]interface{}

	// Cluster returns the raft cluster servers
	Cluster() (RaftCluster, error)

	// LeadServer returns the raft lead server
	LeadServer() (Peer, error)

	// WaitForLeader blocks until a leader is detected, or the timeout expires.
	WaitForLeader(timeout time.Duration) (string, error)

	// WaitForApplied waits for all Raft log entries to to be applied to the
	// underlying database.
	WaitForApplied(timeout time.Duration) error

	// LeaderCh is used to get a channel which delivers signals on
	// acquiring or losing leadership. It sends true if we become
	// the leader, and false if we lose it. The channel is not buffered,
	// and does not block on writes.
	LeaderCh() <-chan bool

	// NodeState returns the state of current node
	NodeState() string
}

Store is the interface Raft-backed key-value stores must implement.

type Ticker 

type Ticker struct {
	// contains filtered or unexported fields
}

Ticker defines a ticker.

func NewTicker 

func NewTicker(d time.Duration, startInstantly bool, tickerFns ...func()) *Ticker

NewTicker creates a new ticker.

func (*Ticker) StartAsync 

func (j *Ticker) StartAsync(tickerFns ...func())

StartAsync starts the ticker. if tickerFns are passed, they will overwrite the previous passed in NewTicker call.

func (*Ticker) StopAsync 

func (j *Ticker) StopAsync()

StopAsync stops the ticker.

type ViperProvider 

type ViperProvider interface {
	SetDefault(key string, value interface{})
	GetBool(key string) bool
	GetString(key string) string
}

ViperProvider defines the args getter provider.

type Writer 

type Writer interface {
	Print(...interface{})
}

Writer log writer interface.

Source Files

View all Source files

Directories

Path Synopsis
cmd
hraftd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.