exporter

README

SSV - Exporter

Intro

Exporter node is responsible for exposing data from SSV Network to the Explorer Center, which index the data and provides an API for the Web UI.

The Web UI shows information for a validator, It provides a way for validator to inspect the operators' performance, duties history and more.

Links

• BLOXSSV-157

Design

Exporter node is new type of peer that needs to pull and store data from SSV nodes or smart contract.
As most of that logic already exist in SSV, the exporter is just a new executable, re-using existing code from SSV and have slightly different configuration.

The introduction of exporter requires to do some refactoring in the project structure as follows:

Data Sources

The following information will be stored and served by exporter:

• Operators
  • Name (name: string) --> contract event
  • Public Key (publicKey: string) --> contract event
  • Owner Address (ownerAddress: string) --> contract event
  • Index (index: uint) --> a sequential index
• Validators
  • Public Key (publicKey: string) --> contract event
  • Operators (operators: []) --> contract event
    • Operator Public Key (publicKey: string)
    • IBFT/Node ID (nodeId: uint)
  • Index (index: uint) --> a sequential index
• Duties
  • Validator Public Key (publicKey: string)
  • Epoch (epoch: uint64) --> calculated from Slot
  • Slot (slot: uint64) --> part of the lambda
  • Duty type / role (slot: string) --> part of the lambda
  • Status (status: bool) - failed | success
  • Operators --> signer_ids

Contract Data

Events to listen:

• OperatorAdded
• ValidatorAdded
• OessAdded

Contract Sync

In order to have all the needed data, exporter needs to read all events logs of the specified events.

On start, exporter will first finish syncing data. Once sync is finished, the exporter will be able to serve requests and listen to live events from the contract.

FilterLogs() accepts FilterQuery that enables to query a specific contract by addresses, and to provide a scope of blocks (FromBlock, ToBlock)

A genesis block for the contract can be used as a baseline block (the block to start the sync)

IBFT Data

Interaction with SSV nodes can be done using the existing history sync end-point.

Persistency

A storage for Exporter Node should support persistence of:

• Operators
• Validators
• IBFT (decided)

Database

A Key-Value Store (Badger) is a sufficient storage as the indexing of the data will be done by exporter consumers (Explorer).

Badger is an embedded DB (stored in FS), therefore won't support HA.
In order to achieve HA, one of the following should be the way to go:

• Use some other remote DB (e.g. S3)
• Use a shared volume (K8S)

APIs

Exporter Node provides WebSocket endpoints for reading the collected data.
There are 2 types of end-points:

• stream - exporter pushes live data
  • IBFT data - notify once decided messages arrives
  • Operators / Validators - notify on contract events
• query - consumer request data on demand
  • requested with the corresponding filters

Message Structure

Request holds a filter for making queries of specific data and a type to distinguish between messages:

{
  "type": "operator" | "validator" | "ibft"
  "filter": {
    "from": number,
    "to": number,
    "role": "ATTESTER" | "AGGREGATOR" | "PROPOSER",
    "publicKey": string
  }
}

Response extends the Request with a data section that contains the corresponding results:

{
  "data": Operator[] | Validator[] | DecidedMessage[]
}

In addition, response might reflect an error, see Error Handling:

{
  "type": "error",
  "data": string[]
}

End Points

Query

/query is an API that allows some consumer to request data, by specifying filter.

For example, a request to get all available operators:

{
  "type": "operator",
  "filter": {
    "from": 0
  }
}

Exporter will produce the following response:

{
  "type": "operator",
  "filter": {
    "from": 0
  },
  "data": [
    {
      "publicKey": "...",
      "name": "myOperator",
      "ownerAddress": "...",
      "index": 0
    },
    {
      "publicKey": "...",
      "name": "myOperator",
      "ownerAddress": "...",
      "index": 1
    },
    ...
  ]
}

Error Handling

In case of bad request or some internal error, the response will be of type "error".

Some Examples:

• Bad input (corrupted JSON) produces:

  {
    "type": "error",
    "filter": {
      "from": 0,
      "to": 0
    },
    "data": [
      "could not parse network message"
    ]
  }
  

• Unknown message type results:

  {
    "type": "error",
    "filter": {
      "from": 0,
      "to": 0
    },
    "data": [
      "bad request - unknown message type 'foo'"
    ]
  }
  

Stream

/stream is an API that allows consumers to get live data that is collected by the exporter, which will push the information it receives (validator, operator or duties) from SSV nodes or contract.

For example, exporter will push a message if a new validator was added to the network:

{
  "type": "validator",
  "filter": {
    "from": 2430,
    "to": 2431,
  },
  "data": [
    {
      "publicKey": "...",
      "operators": [...],
      "index": 2341
    }
  ]
}

Usage

Run Locally

Build if you don't have the executable:

make build

Then run with:

make CONFIG_PATH=./config/config.exporter.yaml BUILD_PATH=./bin/ssvnode start-exporter

Run in Docker

make NODES=exporter-node docker-all

Run in Docker (debug)

make DEBUG_NODES=exporter-node-dev docker-debug

Explore API

Use a tool for WebSockets (such as wscat) to interact with the API.

wscat -c ws://ws-exporter.stage.ssv.network/query

Once connection is ready, type your query:

> { "type": "operator", "filter": { "from": 0, "to": 4 } }

The expected results contains a list of desired operators, in our case in index [0, 4]

< { "type": "operator", "filter": { "from": 0, "to": 4}, "data":[...] }

Documentation

Index

• type Exporter
  • func New(opts Options) Exporter
• type Options

Types

type Exporter

type Exporter interface {
	Start() error
	StartEth1(syncOffset *eth1.SyncOffset) error
}

Exporter represents the main interface of this package

func New

func New(opts Options) Exporter

New creates a new Exporter instance

type Options

type Options struct {
	Ctx context.Context

	Logger     *zap.Logger
	ETHNetwork *core.Network

	Eth1Client eth1.Client

	Network network.Network

	DB basedb.IDb

	WS        api.WebSocketServer
	WsAPIPort int
}

Options contains options to create the node