ethereum_watcher

package module
v0.0.0-...-d1cf31c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 9, 2024 License: Apache-2.0 Imports: 15 Imported by: 0

README

ethereum-watcher

ethereum-watcher is an event listener for the Ethereum Blockchain written in Golang. With ethereum-watcher you can monitor and track current or historic events that occur on the Ethereum Blockchain.

Background

Many applications that interact with the Ethereum Blockchain need to know when specific actions occur on the chain, but cannot directly access the on-chain data. ethereum-watcher acts as an interface between application and chain: gathering specified data from the blockchain so that applications can more seamlessly interact with on-chain events.

Features

  1. Plug-in friendly. You can easily add a plugin to ethereum-watcher to listen to any type of on-chain event.
  2. Fork Tolerance. If a fork occurs, a revert message is sent to the subscriber.

Example Use Cases

  • DefiWatch monitors the state of Ethereum addresses on several DeFi platforms including DDEX, Compound, DyDx, and Maker. It tracks things like loan ROI, current borrows, liquidations, etc. To track all of this on multiple platforms, DefiWatch has to continuously receive updates from their associated smart contracts. This is done using ethereum-watcher instead of spending time dealing with serialization/deserialization messages from the Ethereum Node, so they can focus on their core logic.
  • Profit & Loss calculations on DDEX. DDEX provides their margin trading users with estimated Profit and Loss (P&L) calculations for their margin positions. To update the P&L as timely and accurately as possible, DDEX uses ethereum-watcher to listen to updates from the Ethereum Blockchain. These updates include: onchain price updates, trading actions from users, and more.
  • DDEX also uses an "Eth-Transaction-Watcher" to monitor the on-chain status of trading transactions. DDEX needs to know the latest states of these transactions once they are included in newly mined blocks, so that the platform properly updates trading balances and histories. This is done using the TxReceiptPlugin of ethereum-watcher.

Installation

Run go get github.com/mobazha/ethereum-watcher

Sample Commands

This project is primarily designed as a library to build upon. However, to help others easily understand how ethereum-watcher works and what it is capable of, we prepared some sample commands for you to try out.

display basic help info

docker run hydroprotocolio/ethereum-watcher:master /bin/ethereum-watcher help

ethereum-watcher makes getting updates from Ethereum easier

Usage:
  ethereum-watcher [command]

Available Commands:
  contract-event-listener listen and print events from contract
  help                    Help about any command
  new-block-number        Print number of new block
  usdt-transfer           Show Transfer Event of USDT

Flags:
  -h, --help   help for ethereum-watcher

Use "ethereum-watcher [command] --help" for more information about a command.

print new block numbers

docker run hydroprotocolio/ethereum-watcher:master /bin/ethereum-watcher new-block-number

time="2020-01-07T07:33:17Z" level=info msg="waiting for new block..."
time="2020-01-07T07:33:19Z" level=info msg=">> found new block: 9232152, is removed: false"
time="2020-01-07T07:33:44Z" level=info msg=">> found new block: 9232153, is removed: false"
time="2020-01-07T07:34:03Z" level=info msg=">> found new block: 9232154, is removed: false"
time="2020-01-07T07:34:04Z" level=info msg=">> found new block: 9232155, is removed: false"
time="2020-01-07T07:34:05Z" level=info msg=">> found new block: 9232156, is removed: false"
...

see USDT transfer events

docker run hydroprotocolio/ethereum-watcher:master /bin/ethereum-watcher usdt-transfer

time="2020-01-07T07:34:32Z" level=info msg="See new USDT Transfer at block: 9232158, count:  9"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0x1072efee913229a5e9a3013af6b580099f03ac9c75bfc60013cfa7efac726067"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0xae191608df7688ad6d83ebe8151fc5519ff0e29515b557e15ed3fbcbfadca698"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0xc3bcfcffa4d3318c27b923eccbd6bbac179f9f982d0282bed164c8e5216130cc"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0xe15f2b029c248d1560b10f1879af46b2c8fb2362db9287a8a257042ec2dbb46c"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0x860156ebe4c1f00cc479d4f75b7665e2c6ef66d3b085ba1630e803fa44ce2836"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0x5b2f4b07332096f55c6ca69e3b349fc19e2c6f54f3a7062f4365697ffb6c5d51"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0xb9325afae978b509feee8bc092a090e7a4df750e9e94d10a2bce29e543406c26"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0xe379b9e776cf0920a5cc284e9a0f7d141894078827b28f43905dd45457f886e3"
time="2020-01-07T07:34:32Z" level=info msg="  >> tx: https://etherscan.io/tx/0x0f89d867117a6107de44da523b1ad8fbd630d42d9f809c16b82c6f582a8c97da"

time="2020-01-07T07:34:50Z" level=info msg="See new USDT Transfer at block: 9232159, count: 18"
time="2020-01-07T07:34:50Z" level=info msg="  >> tx: https://etherscan.io/tx/0x506ff84b205f0802f037eefe988c9c431d5cae40d3c2c9354616f99f1751bad9"
time="2020-01-07T07:34:50Z" level=info msg="  >> tx: https://etherscan.io/tx/0xd55a1175801eb9af3be65db23d3a9a32f4494f31473ad92dbd95e1d9f9b57fc3"
time="2020-01-07T07:34:50Z" level=info msg="  >> tx: https://etherscan.io/tx/0x93bba744986c75645590b2c930b43e34e9ab219feb38875b87b327854c37e7eb"
time="2020-01-07T07:34:50Z" level=info msg="  >> tx: https://etherscan.io/tx/0xae22eaa9aef7079ff9bb23fa25f314fcf22e9688ef7b585462d3a2afe33628ef"
...

see specific events that occur within a smart contract. The example shows Transfer & Approve events from Multi-Collateral-DAI

docker run hydroprotocolio/ethereum-watcher:master /bin/ethereum-watcher contract-event-listener \
    --block-backoff 100 \
    --contract 0x6b175474e89094c44da98b954eedeac495271d0f \
    --events 0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef
    
INFO[2020-01-07T18:05:26+08:00] --block-backoff activated, we start from block: 9232741 (= 9232841 - 100)

INFO[2020-01-07T18:05:27+08:00] # of interested events at block(9232741->9232745): 1
INFO[2020-01-07T18:05:27+08:00]   >> tx: https://etherscan.io/tx/0x4e05d731ca1b8bf0e2a85d825751312c722bfa3c9210a8b574c06e1e6c992a75

INFO[2020-01-07T18:05:28+08:00] # of interested events at block(9232746->9232750): 3
INFO[2020-01-07T18:05:28+08:00]   >> tx: https://etherscan.io/tx/0x8645784c881a01d326469ddbf45a151f9eab8bedbfc02f6cbe1ab2f03c03c022
INFO[2020-01-07T18:05:28+08:00]   >> tx: https://etherscan.io/tx/0xac5e9fa166a2e7f13aa104812b9278cc65247beff68a4aca7b382e9547d84ec5
INFO[2020-01-07T18:05:28+08:00]   >> tx: https://etherscan.io/tx/0xac5e9fa166a2e7f13aa104812b9278cc65247beff68a4aca7b382e9547d84ec5

INFO[2020-01-07T18:05:29+08:00] # of interested events at block(9232751->9232755): 2
INFO[2020-01-07T18:05:29+08:00]   >> tx: https://etherscan.io/tx/0x280eb9556f6286bf707e859fc6c08bdd1e2ed8f2bdd360883c207f0da0122b01
INFO[2020-01-07T18:05:29+08:00]   >> tx: https://etherscan.io/tx/0xcc5ee10d5ac8f55f51f74b23186052c042ebc5dda61578c1a1038d0b30b6fd91
...

Here the flag --block-backoff signals for ethereum-watcher to use historic tracking from 100 blocks ago.

Usage

To effectively use ethereum-watcher, you will be interacting with two primary structs:

  • Watcher
  • ReceiptLogWatcher

Watcher

Watcher is an HTTP client which continuously polls newly mined blocks on the Ethereum Blockchain. We can incorporate various kinds of "plugins" into Watcher, which will poll for specific types of events and data, such as:

  • BlockPlugin
  • TransactionPlugin
  • TransactionReceiptPlugin
  • ReceiptLogPlugin

Once the Watcher sees a new block, it will parse the info and feed the data into the registered plugins. You can see some of the code examples below.

The plugins are designed to be easily modifiable so that you can create your own plugin based on the provided ones. For example, the code shown below registers an ERC20TransferPlugin to show new ERC20 Transfer Events. This plugin simply parses some receipt info from a different TransactionReceiptPlugin. So if you want to show more info than what the ERC20TransferPlugin shows, like the gas used in the transaction, you can easily create a BetterERC20TransferPlugin showing that.

Watcher Examples
Print number of newly mined blocks
package main

import (
	"context"
	"fmt"
	"github.com/mobazha/ethereum-watcher/plugin"
	"github.com/mobazha/ethereum-watcher/structs"
)

func main() {
	api := "https://mainnet.infura.io/v3/19d753b2600445e292d54b1ef58d4df4"
	w := NewHttpBasedEthWatcher(context.Background(), api)

	// we use BlockPlugin here
	w.RegisterBlockPlugin(plugin.NewBlockNumPlugin(func(i uint64, b bool) {
		fmt.Println(">>", i, b)
	}))

	w.RunTillExit()
}
Listen for new ERC20 Transfer Events
package main

import (
	"context"
	"fmt"
	"github.com/mobazha/ethereum-watcher/plugin"
	"github.com/mobazha/ethereum-watcher/structs"
	"github.com/sirupsen/logrus"
)

func main() {
	api := "https://mainnet.infura.io/v3/19d753b2600445e292d54b1ef58d4df4"
	w := NewHttpBasedEthWatcher(context.Background(), api)

	// we use TxReceiptPlugin here
	w.RegisterTxReceiptPlugin(plugin.NewERC20TransferPlugin(
		func(token, from, to string, amount decimal.Decimal, isRemove bool) {

			logrus.Infof("New ERC20 Transfer >> token(%s), %s -> %s, amount: %s, isRemoved: %t",
				token, from, to, amount, isRemove)

		},
	))

	w.RunTillExit()
}

ReceiptLogWatcher

Watcher is polling for blocks one by one, so what if we want to query certain events from the latest 10000 blocks? Watcher can do that but fetching blocks one by one can be slow. ReceiptLogWatcher to the rescue!

ReceiptLogWatcher makes use of the eth_getLogs to query for logs in a batch. Check out the code below to see how to use it.

Example of ReceiptLogWatcher
package main

import (
	"context"
	"github.com/mobazha/ethereum-watcher/blockchain"
	"github.com/sirupsen/logrus"
)

func main() {
	api := "https://mainnet.infura.io/v3/19d753b2600445e292d54b1ef58d4df4"
	usdtContractAdx := "0xdac17f958d2ee523a2206206994597c13d831ec7"
  
	// ERC20 Transfer Event
	topicsInterestedIn := []string{"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"}

	handler := func(from, to int, receiptLogs []blockchain.IReceiptLog, isUpToHighestBlock bool) error {
		logrus.Infof("USDT Transfer count: %d, %d -> %d", len(receiptLogs), from, to)
		return nil
	}

	// query for USDT Transfer Events
	receiptLogWatcher := NewReceiptLogWatcher(
		context.TODO(),
		api,
		-1,
		usdtContractAdx,
		topicsInterestedIn,
		handler,
		ReceiptLogWatcherConfig{
			StepSizeForBigLag:               5,
			IntervalForPollingNewBlockInSec: 5,
			RPCMaxRetry:                     3,
			ReturnForBlockWithNoReceiptLog:  true,
		},
	)

	receiptLogWatcher.Run()
}

License

Apache 2.0 License

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AbstractWatcher 

type AbstractWatcher struct {
	Ctx context.Context

	NewBlockChan        chan *structs.RemovableBlock
	NewTxAndReceiptChan chan *structs.RemovableTxAndReceipt
	NewReceiptLogChan   chan *structs.RemovableReceiptLog

	SyncedBlocks         *list.List[*types.Block]
	SyncedTxAndReceipts  *list.List[*structs.TxAndReceipt]
	MaxSyncedBlockToKeep int

	BlockPlugins      []plugin.IBlockPlugin
	TxPlugins         []plugin.ITxPlugin
	TxReceiptPlugins  []plugin.ITxReceiptPlugin
	ReceiptLogPlugins []plugin.IReceiptLogPlugin

	ReceiptCatchUpFromBlock uint64
	// contains filtered or unexported fields
}

func NewHttpBasedEthWatcher 

func NewHttpBasedEthWatcher(ctx context.Context, api string) (*AbstractWatcher, error)

func (*AbstractWatcher) FoundFork 

func (watcher *AbstractWatcher) FoundFork(newBlock *types.Block) bool

func (*AbstractWatcher) LatestSyncedBlockNum 

func (watcher *AbstractWatcher) LatestSyncedBlockNum() uint64

func (*AbstractWatcher) RegisterBlockPlugin 

func (watcher *AbstractWatcher) RegisterBlockPlugin(plugin plugin.IBlockPlugin)

func (*AbstractWatcher) RegisterReceiptLogPlugin 

func (watcher *AbstractWatcher) RegisterReceiptLogPlugin(plugin plugin.IReceiptLogPlugin)

func (*AbstractWatcher) RegisterTxPlugin 

func (watcher *AbstractWatcher) RegisterTxPlugin(plugin plugin.ITxPlugin)

func (*AbstractWatcher) RegisterTxReceiptPlugin 

func (watcher *AbstractWatcher) RegisterTxReceiptPlugin(plugin plugin.ITxReceiptPlugin)

func (*AbstractWatcher) RunTillExit 

func (watcher *AbstractWatcher) RunTillExit() error

start sync from latest block

func (*AbstractWatcher) RunTillExitFromBlock 

func (watcher *AbstractWatcher) RunTillExitFromBlock(startBlockNum uint64) error

start sync from given block 0 means start from latest block

func (*AbstractWatcher) SetSleepSecondsForNewBlock 

func (watcher *AbstractWatcher) SetSleepSecondsForNewBlock(sec int)

type ReceiptLogWatcher 

type ReceiptLogWatcher struct {
	// contains filtered or unexported fields
}

func NewReceiptLogWatcher 

func NewReceiptLogWatcher(
	ctx context.Context,
	api string,
	startBlockNum int,
	contracts []common.Address,
	interestedTopics [][]common.Hash,
	handler func(from, to int, receiptLogs []types.Log, isUpToHighestBlock bool) error,
	configs ...ReceiptLogWatcherConfig,
) *ReceiptLogWatcher

func (*ReceiptLogWatcher) AddInterestedTopics 

func (w *ReceiptLogWatcher) AddInterestedTopics(contracts []common.Address, position int, topics []common.Hash)

AddInterestedTopics add more topics to match indexed topic in given position (0 based)

func (*ReceiptLogWatcher) GetHighestSyncedBlockNum 

func (w *ReceiptLogWatcher) GetHighestSyncedBlockNum() int

func (*ReceiptLogWatcher) GetHighestSyncedBlockNumAndLogIndex 

func (w *ReceiptLogWatcher) GetHighestSyncedBlockNumAndLogIndex() (int, int)

func (*ReceiptLogWatcher) IsRunning 

func (w *ReceiptLogWatcher) IsRunning() bool

func (*ReceiptLogWatcher) RemoveInterestedTopics 

func (w *ReceiptLogWatcher) RemoveInterestedTopics(position int, topics []common.Hash)

RemoveInterestedTopics remove topics to match indexed topic in given position (0 based)

func (*ReceiptLogWatcher) Run 

func (w *ReceiptLogWatcher) Run() error

type ReceiptLogWatcherConfig 

type ReceiptLogWatcherConfig struct {
	StepSizeForBigLag               int
	ReturnForBlockWithNoReceiptLog  bool
	IntervalForPollingNewBlockInSec int
	RPCMaxRetry                     int
	LagToHighestBlock               int
	StartSyncAfterLogIndex          int
}

type SyncSignal 

type SyncSignal struct {
	// contains filtered or unexported fields
}

func (*SyncSignal) Done 

func (s *SyncSignal) Done()

func (*SyncSignal) Permit 

func (s *SyncSignal) Permit()

func (*SyncSignal) WaitDone 

func (s *SyncSignal) WaitDone()

func (*SyncSignal) WaitPermission 

func (s *SyncSignal) WaitPermission()

Source Files

View all Source files

Directories

Path Synopsis
pkg
container/list
Package list implements a doubly linked list.
 Package list implements a doubly linked list.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.