etcddb

README

etcd Payment Channel Storage

To enable etcd server as a payment channel storage in snet-daemon configure the following properties in the JSON config file: ( Please note this is applicable only when the block chain is enabled)

• payment_channel_storage_type
• payment_channel_storage_client
• payment_channel_storage_server

etcd storage type

There are two payment channel storage types which are currently supported by snet daemon: memory and etcd. memory storage type is used in configuration where only one service replica is used by snet-daemon or for testing purposes.

To run snet-daemon with several replicas set the payment_channel_storage_type is now initialized from Organizatio metadata:

{
  "payment_channel_storage_type": "etcd"
}

etcd client configuration

payment_channel_storage_client JSON map can be used to configure payment channel storage etcd client.

Field name	Description	Default Value
connection_timeout	timeout for failing to establish a connection	5 seconds
request_timeout	per request timeout	3 seconds
endpoints	list of etcd cluster endpoints (host:port)	["http://127.0.0.1:2379"]

Endpoints consist of a list of URLs which points to etcd cluster servers.

The following config describes a client which connects to 3 etcd server nodes and the data is retrieved from Organization Metadata:

{
	"payment_channel_storage_client": {
		"connection_timeout": "5s",
		"request_timeout": "3s",
		"endpoints": ["http://127.0.0.1:2379", "http://127.0.0.2:2379", "http://127.0.0.3:2379"]
	}
}

etcd client configuration ( https mode)

if the client end point is https, then you will need to add the following on your configuration to use the certificates to connect "payment_channel_cert_path": "", "payment_channel_ca_path": "", "payment_channel_key_path": "",

etcd server configuration

The latest Daemon expects an etcd cluster setup already available , in case you wish to set up your own cluster , please go over the documentation below To use embedded etcd server in snet-daemon the configuration file needs to contain the payment_channel_storage_server JSON map with fields:

Field name	Description	Default Value
id	unique name of the etcd server node	storage-1
schema	URL schema used to create client and peer and urls	http
host	host where the etcd server is executed	127.0.0.1
client_port	port to listen clients requests	2379
peer_port	port to listen etcd peers	2380
token	unique initial cluster token	unique-token
cluster	initial cluster configuration for bootstrapping	storage-1=http://127.0.0.1:2380
startup_timeout	time to wait that etcd server is successfully started	1 minute
data_dir	directory where etcd server stores its data	storage-data-dir-1.etcd
log_level	etcd server logging level (error, warning, info, debug)	info
enabled	enable running embedded etcd server	true

The cluster field is a comma-separated list of one or more etcd peer URLs in form of id=host:peer_port.

schema, host, and client_port/peer_port are used together to compose etcd listen-client-urls/listen-peer-urls (see the link below).

Using unique token, etcd can generate unique cluster IDs and member IDs for the clusters even if they otherwise have the exact same configuration. This can protect etcd from cross-cluster-interaction, which might corrupt the clusters.

To disable etcd server log messages set log_level field to error.

For more details see etcd Clustering Guide link.

It is possible to configure snet-daemon to run with or without embedded etcd server using the enabled property.

Config for snet-daemon that runs embedded etcd server:

• enabled field is set to false , ETCD cluster set up is retrieved from the Organization metadata , If you want to set up a local cluster , then payment_channel_storage_server.enabled configuration needs to be set to true.

{
    "payment_channel_storage_server": {
        "id": "storage-1",
        "host" : "127.0.0.1",
        "client_port": 2379,
        "peer_port": 2380,
        "token": "unique-token",
        "cluster": "storage-1=http://127.0.0.1:2380",
        "enabled": true
    }
}

Config for snet-daemon that does not run embedded etcd node:

• enabled field is set to false

{
    "payment_channel_storage_server": {
        "id": "storage-2",
        "host" : "127.0.0.2",
        "client_port": 2379,
        "peer_port": 2380,
        "token": "unique-token",
        "cluster": "storage-1=http://127.0.0.1:2380,storage-2=http://127.0.0.2:2380,storage-3=http://127.0.0.3:2380",
        "enabled": false
    }
}

Documentation

Index

• func IsEtcdServerEnabled() (enabled bool, err error)
• func IsEtcdServerEnabledInVip(vip *viper.Viper) (enabled bool, err error)
• type EtcdClient
  • func NewEtcdClient(metaData *blockchain.OrganizationMetaData) (client *EtcdClient, err error)
  • func NewEtcdClientFromVip(vip *viper.Viper, metaData *blockchain.OrganizationMetaData) (client *EtcdClient, err error)
  • func (client *EtcdClient) Close()
  • func (client *EtcdClient) CompareAndSwap(key string, prevValue string, newValue string) (ok bool, err error)
  • func (client *EtcdClient) CompleteTransaction(_transaction escrow.Transaction, update []escrow.KeyValueData) (ok bool, err error)
  • func (client *EtcdClient) Delete(key string) error
  • func (client *EtcdClient) ExecuteTransaction(request escrow.CASRequest) (ok bool, err error)
  • func (client *EtcdClient) Get(key string) (value string, ok bool, err error)
  • func (client *EtcdClient) GetByKeyPrefix(key string) (values []string, err error)
  • func (client *EtcdClient) NewMutex(key string) (mutex *EtcdClientMutex, err error)
  • func (client *EtcdClient) Put(key string, value string) (err error)
  • func (client *EtcdClient) PutIfAbsent(key string, value string) (ok bool, err error)
  • func (client *EtcdClient) StartTransaction(keys []string) (_transaction escrow.Transaction, err error)
  • func (client *EtcdClient) Transaction(compare []EtcdKeyValue, swap []EtcdKeyValue) (ok bool, err error)
• type EtcdClientConf
  • func GetEtcdClientConf(vip *viper.Viper, metaData *blockchain.OrganizationMetaData) (conf *EtcdClientConf, err error)
• type EtcdClientMutex
  • func (mutex *EtcdClientMutex) Lock(ctx context.Context) (err error)
  • func (mutex *EtcdClientMutex) Unlock(ctx context.Context) (err error)
• type EtcdKeyValue
• type EtcdServer
  • func GetEtcdServer() (server *EtcdServer, err error)
  • func GetEtcdServerFromVip(vip *viper.Viper) (server *EtcdServer, err error)
  • func (server *EtcdServer) Close()
  • func (server *EtcdServer) Start() (err error)
• type EtcdServerConf
  • func GetEtcdServerConf(vip *viper.Viper) (conf *EtcdServerConf, err error)

Functions

func IsEtcdServerEnabled

func IsEtcdServerEnabled() (enabled bool, err error)

IsEtcdServerEnabled checls that etcd server is enabled using conf file

func IsEtcdServerEnabledInVip

func IsEtcdServerEnabledInVip(vip *viper.Viper) (enabled bool, err error)

IsEtcdServerEnabledInVip checls that etcd server is enabled using viper conf

Types

type EtcdClient

type EtcdClient struct {
	// contains filtered or unexported fields
}

EtcdClient struct has some useful methods to wolrk with etcd client

func NewEtcdClient

func NewEtcdClient(metaData *blockchain.OrganizationMetaData) (client *EtcdClient, err error)

NewEtcdClient create new etcd storage client.

func NewEtcdClientFromVip

func NewEtcdClientFromVip(vip *viper.Viper, metaData *blockchain.OrganizationMetaData) (client *EtcdClient, err error)

NewEtcdClientFromVip create new etcd storage client from viper.

func (*EtcdClient) Close

func (client *EtcdClient) Close()

Close closes etcd client

func (*EtcdClient) CompareAndSwap

func (client *EtcdClient) CompareAndSwap(key string, prevValue string, newValue string) (ok bool, err error)

CompareAndSwap uses CAS operation to set a value

func (*EtcdClient) CompleteTransaction

func (client *EtcdClient) CompleteTransaction(_transaction escrow.Transaction, update []escrow.KeyValueData) (
	ok bool, err error)

If there are no Old values in the transaction, to compare, then this method can be used to write in the new values , if the key does not exist then put it in a transaction

func (*EtcdClient) Delete

func (client *EtcdClient) Delete(key string) error

Delete deletes the existing key and value from etcd

func (*EtcdClient) ExecuteTransaction

func (client *EtcdClient) ExecuteTransaction(request escrow.CASRequest) (ok bool, err error)

func (*EtcdClient) Get

func (client *EtcdClient) Get(key string) (value string, ok bool, err error)

Get gets value from etcd by key

func (*EtcdClient) GetByKeyPrefix

func (client *EtcdClient) GetByKeyPrefix(key string) (values []string, err error)

GetByKeyPrefix gets all values which have the same key prefix

func (*EtcdClient) NewMutex

func (client *EtcdClient) NewMutex(key string) (mutex *EtcdClientMutex, err error)

NewMutex Create a mutex for the given key

func (*EtcdClient) Put

func (client *EtcdClient) Put(key string, value string) (err error)

Put puts key and value to etcd

func (*EtcdClient) PutIfAbsent

func (client *EtcdClient) PutIfAbsent(key string, value string) (ok bool, err error)

PutIfAbsent puts value if absent

func (*EtcdClient) StartTransaction

func (client *EtcdClient) StartTransaction(keys []string) (_transaction escrow.Transaction, err error)

func (*EtcdClient) Transaction

func (client *EtcdClient) Transaction(compare []EtcdKeyValue, swap []EtcdKeyValue) (ok bool, err error)

Transaction uses CAS operation to compare and set multiple key values

type EtcdClientConf

type EtcdClientConf struct {
	ConnectionTimeout time.Duration `json:"connection_timeout" mapstructure:"connection_timeout"`
	RequestTimeout    time.Duration `json:"request_timeout" mapstructure:"request_timeout"`
	Endpoints         []string
}

EtcdClientConf config ConnectionTimeout - timeout for failing to establish a connection RequestTimeout - per request timeout Endpoints - cluster endpoints

func GetEtcdClientConf

func GetEtcdClientConf(vip *viper.Viper, metaData *blockchain.OrganizationMetaData) (conf *EtcdClientConf, err error)

GetEtcdClientConf gets EtcdServerConf from viper The DefaultEtcdClientConf is used in case the PAYMENT_CHANNEL_STORAGE_CLIENT field is not set in the configuration file Left Vip, just in case we need to read something from configuration in the future

type EtcdClientMutex

type EtcdClientMutex struct {
	// contains filtered or unexported fields
}

EtcdClientMutex mutex struct for etcd client

func (*EtcdClientMutex) Lock

func (mutex *EtcdClientMutex) Lock(ctx context.Context) (err error)

Lock lock etcd key

func (*EtcdClientMutex) Unlock

func (mutex *EtcdClientMutex) Unlock(ctx context.Context) (err error)

Unlock unlock etcd key

type EtcdKeyValue

type EtcdKeyValue struct {
	// contains filtered or unexported fields
}

EtcdKeyValue contains key and value

type EtcdServer

type EtcdServer struct {
	// contains filtered or unexported fields
}

EtcdServer struct has some useful methods to wolrk with etcd server

func GetEtcdServer

func GetEtcdServer() (server *EtcdServer, err error)

GetEtcdServer returns EtcdServer in case it is defined in the viper config reuturns null if PAYMENT_CHANNEL_STORAGE property is not defined in the config file or the ENABLED field of the PAYMENT_CHANNEL_STORAGE is set to false

func GetEtcdServerFromVip

func GetEtcdServerFromVip(vip *viper.Viper) (server *EtcdServer, err error)

GetEtcdServerFromVip run etcd server using viper config

func (*EtcdServer) Close

func (server *EtcdServer) Close()

Close closes etcd server

func (*EtcdServer) Start

func (server *EtcdServer) Start() (err error)

Start starts etcd server

type EtcdServerConf

type EtcdServerConf struct {
	ID             string
	Scheme         string
	Host           string
	ClientPort     int `json:"client_port" mapstructure:"CLIENT_PORT"`
	PeerPort       int `json:"peer_port" mapstructure:"PEER_PORT"`
	Token          string
	Cluster        string
	StartupTimeout time.Duration `json:"startup_timeout" mapstructure:"startup_timeout"`
	Enabled        bool
	DataDir        string `json:"data_dir" mapstructure:"DATA_DIR"`
	LogLevel       string `json:"log_level" mapstructure:"LOG_LEVEL"`
}

EtcdServerConf contains embedded etcd server config ID - unique name of the etcd server node Scheme - URL schema used to create client and peer and urls Host - host where the etcd server is executed ClientPort - port to listen clients, used together with

Schema and host to compose listen-client-urls (see link below)

PeerPort - port to listen etcd peers, used together with

Schema and host to compose listen-client-urls (see link below)

Token - unique initial cluster token. Using unique token etcd can generate unique

cluster IDs and member IDs for the clusters even if they otherwise have
the exact same configuration. This can protect etcd from
cross-cluster-interaction, which might corrupt the clusters.

StartupTimeout - time to wait the etcd server successfully started Enabled - enable running embedded etcd server For more details see etcd Clustering Guide link: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/clustering.md

func GetEtcdServerConf

func GetEtcdServerConf(vip *viper.Viper) (conf *EtcdServerConf, err error)

GetEtcdServerConf gets EtcdServerConf from viper The DefaultEtcdServerConf is used in case the PAYMENT_CHANNEL_STORAGE_SERVER field is not set in the configuration file