README
¶
Client stats reporting
Specification
The request JSON object is a non-nested object with the following properties. The process refers to which process data is associated with the request.
| Property | Type | Process | Source | Description |
|---|---|---|---|---|
| version | int | Every request | hard-coded | Stats data specification version, current only 1 is accepted |
| timestamp | long | Every request | system time | Unix timestamp in milliseconds |
| process | string (enum) | Every request | hard-coded / cli flag | Enum values: validator, beaconnode, system |
| cpu_process_seconds_total | long | beaconnode, validator | prom: process_cpu_seconds_total | CPU seconds consumed by the process |
| memory_process_bytes | long | beaconnode, validator | prom: process_resident_memory_bytes | Number of bytes allocated to the process |
| client_name | string | beaconnode, validator | hard-coded to "prysm" | Name of the client type. Ex: prysm, lighthouse, nimbus, teku |
| client_version | string | beaconnode, validator | prom: prysm_version (label: version) | Client version. Ex: 1.0.0-beta.0 |
| client_build | int | beaconnode, validator | prom: prysm_version (label: buildDate) | Integer representation of build for easier comparison |
| disk_beaconchain_bytes_total | long | beaconchain | prom: bcnode_disk_beaconchain_bytes_total | The amount of data consumed on disk by the beacon chain's database. |
| network_libp2p_bytes_total_receive | long | beaconchain | (currently unsupported) | The number of bytes received via libp2p traffic |
| network_libp2p_bytes_total_transmit | long | beaconchain | (currently unsupported) | The number of bytes transmitted via libp2p traffic |
| network_peers_connected | int | beaconchain | (currently unsupported) | The number of peers currently connected to the beacon chain |
| sync_eth1_connected | bool | beaconchain | prom: powchain_sync_eth1_connected | Whether or not the beacon chain node is connected to a synced eth1 node |
| sync_eth2_synced | bool | beaconchain | prom: beacon_clock_time_slot (true if this equals prom: beacon_head_slot) | Whether or not the beacon chain node is in sync with the beacon chain network |
| sync_beacon_head_slot | long | beaconchain | prom: beacon_head_slot | The head slot number. |
| sync_eth1_fallback_configured | bool | beaconchain | prom: powchain_sync_eth1_fallback_configured | Whether or not the beacon chain node has a fallback eth1 endpoint configured. |
| sync_eth1_fallback_connected | bool | beaconchain | prom: powchain_sync_eth1_fallback_connected | Whether or not the beacon chain node is connected to a fallback eth1 endpoint. A true value indicates a failed or interrupted connection with the primary eth1 endpoint. |
| slasher_active | bool | beaconchain | (coming soon) | Whether or not slasher functionality is enabled. |
| sync_eth2_fallback_configured | bool | validator | (currently unsupported) | Whether or not the process has a fallback eth2 endpoint configured |
| sync_eth2_fallback_connected | bool | validator | (currently unsupported) | Weather or not the process has connected to the failover eth2 endpoint. A true value indicates a failed or interrupted connection with the primary eth2 endpoint. |
| validator_total | int | validator | prom: validator_statuses (count of all peers) | The number of validating keys in use. |
| validator_active | int | validator | prom: validator_statuses (count of peers w/ "ACTIVE" status label) | The number of validator keys that are currently active. |
| cpu_cores | int | system | (currently unsupported) | The number of CPU cores available on the host machine |
| cpu_threads | int | system | (currently unsupported) | The number of CPU threads available on the host machine |
| cpu_node_system_seconds_total | long | system | (currently unsupported) | Overall CPU seconds observed on the host machine for all processes. |
| cpu_node_user_seconds_total | long | system | (currently unsupported) | ?? |
| cpu_node_iowait_seconds_total | long | system | (currently unsupported) | ?? |
| cpu_node_idle_seconds_total | long | system | (currently unsupported) | ?? |
| memory_node_bytes_total | long | system | (currently unsupported) | ?? |
| memory_node_bytes_free | long | system | (currently unsupported) | ?? |
| memory_node_bytes_cached | long | system | (currently unsupported) | ?? |
| memory_node_bytes_bufferd | long | system | (currently unsupported) | ?? |
| disk_node_bytes_total | long | system | (currently unsupported) | ?? |
| disk_node_bytes_free | long | system | (currently unsupported) | ?? |
| disk_node_io_seconds | long | system | (currently unsupported) | ?? |
| disk_node_reads_total | long | system | (currently unsupported) | ?? |
| disk_node_writes_total | long | system | (currently unsupported) | ?? |
| network_node_bytes_total_receive | long | system | (currently unsupported) | ?? |
| network_node_bytes_total_transmit | long | system | (currently unsupported) | ?? |
| misc_node_boot_ts_system | long | system | (currently unsupported) | ?? |
| misc_os | string | system | (currently unsupported) | Enum values: lin, win, mac, unk |
The client stats reporter will submit a request object for each process type. The report request may submit a list of data or a single JSON object.
Examples
POST https://beaconcha.in/api/v1/stats/$API_KEY/$MACHINE_NAME
Single object payload
{
"version": 1,
"timestamp": 11234567,
"process": "validator",
"cpu_process_seconds_total": 1234567,
"memory_process_bytes": 654321,
"client_name": "lighthouse",
"client_version": "1.1.2",
"client_build": 12,
"sync_eth2_fallback_configured": false,
"sync_eth2_fallback_connected": false,
"validator_total": 3,
"validator_active": 2
}
Multiple object payload
[
{
"version":1,
"timestamp":1618835497239,
"process":"beaconnode",
"cpu_process_seconds_total":6925,
"memory_process_bytes":1175138304,
"client_name":"lighthouse",
"client_version":"1.1.3",
"client_build":42,
"sync_eth2_fallback_configured":false,
"sync_eth2_fallback_connected":false,
"validator_active":1,
"validator_total":1
},
{
"version":1,
"timestamp":1618835497258,
"process":"system",
"cpu_cores":4,
"cpu_threads":8,
"cpu_node_system_seconds_total":1953818,
"cpu_node_user_seconds_total":229215,
"cpu_node_iowait_seconds_total":3761,
"cpu_node_idle_seconds_total":1688619,
"memory_node_bytes_total":33237434368,
"memory_node_bytes_free":500150272,
"memory_node_bytes_cached":13904945152,
"memory_node_bytes_buffers":517832704,
"disk_node_bytes_total":250436972544,
"disk_node_bytes_free":124707479552,
"disk_node_io_seconds":0,
"disk_node_reads_total":3362272,
"disk_node_writes_total":47766864,
"network_node_bytes_total_receive":26546324572,
"network_node_bytes_total_transmit":12057786467,
"misc_node_boot_ts_seconds":1617707420,
"misc_os":"unk"
}
]
Documentation
¶
Index ¶
Constants ¶
const ( ClientName = "prysm" BeaconNodeProcessName = "beaconnode" ValidatorProcessName = "validator" APIVersion = 1 )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type APIMessage ¶
type APIMessage struct {
APIVersion int `json:"version"`
Timestamp int64 `json:"timestamp"` // unix timestamp in milliseconds
ProcessName string `json:"process"` // validator, beaconnode, system
}
APIMessage are common to all requests to the client-stats API Note that there is a "system" type that we do not currently support -- if we did APIMessage would be present on the system messages as well as validator and beaconnode, whereas CommonStats would only be part of beaconnode and validator.
type BeaconNodeStats ¶
type BeaconNodeStats struct {
// TODO(#8850): add support for this after slasher refactor is merged
SlasherActive bool `json:"slasher_active"`
SyncEth1Connected bool `json:"sync_eth1_connected"`
SyncEth2Synced bool `json:"sync_eth2_synced"`
DiskBeaconchainBytesTotal int64 `json:"disk_beaconchain_bytes_total"`
// N/A -- would require significant network code changes at this time
NetworkLibp2pBytesTotalReceive int64 `json:"network_libp2p_bytes_total_receive"`
// N/A -- would require significant network code changes at this time
NetworkLibp2pBytesTotalTransmit int64 `json:"network_libp2p_bytes_total_transmit"`
// p2p_peer_count where label "state" == "Connected"
NetworkPeersConnected int64 `json:"network_peers_connected"`
// beacon_head_slot
SyncBeaconHeadSlot int64 `json:"sync_beacon_head_slot"`
CommonStats `json:",inline"`
}
BeaconNodeStats embeds CommonStats and represents metrics specific to the beacon-node process. This type is used to marshal metrics data to the POST body sent to the metrics collcetor. To make the connection to client-stats clear, BeaconNodeStats is also used by prometheus collection code introduced to support client-stats. Note that some metrics are labeled NA because they are expected to be present with their zero-value when not supported by a client.
type CommonStats ¶
type CommonStats struct {
CPUProcessSecondsTotal int64 `json:"cpu_process_seconds_total"`
MemoryProcessBytes int64 `json:"memory_process_bytes"`
ClientName string `json:"client_name"`
ClientVersion string `json:"client_version"`
ClientBuild int64 `json:"client_build"`
// TODO(#8849): parse the grpc connection string to determine
// if multiple addresses are present
SyncEth2FallbackConfigured bool `json:"sync_eth2_fallback_configured"`
// N/A -- when multiple addresses are provided to grpc, requests are
// load-balanced between the provided endpoints.
// This is different from a "fallback" configuration where
// the second address is treated as a failover.
SyncEth2FallbackConnected bool `json:"sync_eth2_fallback_connected"`
APIMessage `json:",inline"`
}
CommonStats represent generic metrics that are expected on both beaconnode and validator metric types. This type is used for marshaling metrics to the POST body sent to the metrics collcetor. Note that some metrics are labeled NA because they are expected to be present with their zero-value when not supported by a client.
type Scraper ¶
A Scraper polls the data source it has been configured with and interprets the content to produce a client-stats process metric. Scrapers currently exist to produce 'validator' and 'beaconnode' metric types.
func NewBeaconNodeScraper ¶
NewBeaconNodeScraper constructs a Scaper capable of scraping the prometheus endpoint of a beacon-node process and producing the json body for the beaconnode client-stats process type.
func NewValidatorScraper ¶
NewValidatorScraper constructs a Scaper capable of scraping the prometheus endpoint of a validator process and producing the json body for the validator client-stats process type.
type Updater ¶
An Updater can take the io.Reader created by Scraper and send it to a data sink for consumption. An Updater is used for instance to send the scraped data for a beacon-node to a remote client-stats endpoint.
func NewClientStatsHTTPPostUpdater ¶
NewClientStatsHTTPPostUpdater is used when the update endpoint is reachable via an HTTP POST request.
func NewGenericClientStatsUpdater ¶
NewGenericClientStatsUpdater can Update any io.Writer. It is used by the cli to write to stdout when an http endpoint is not provided. The output could be piped into another program or used for debugging.
type ValidatorStats ¶
type ValidatorStats struct {
// N/A -- TODO(#8848): verify whether we can obtain this metric from the validator process
ValidatorTotal int64 `json:"validator_total"`
// N/A -- TODO(#8848): verify whether we can obtain this metric from the validator process
ValidatorActive int64 `json:"validator_active"`
CommonStats `json:",inline"`
}
ValidatorStats embeds CommonStats and represents metrics specific to the validator process. This type is used to marshal metrics data to the POST body sent to the metrics collcetor. Note that some metrics are labeled NA because they are expected to be present with their zero-value when not supported by a client.
Source Files