README
¶
Rosetta Komodo
Overview
rosetta-komodo provides an implementation of the Rosetta API for the
Komodo network in Golang. If you haven't heard of the Rosetta API, you can find more
information here.
Usage
As specified in the Rosetta API Principles,
all Rosetta implementations must be deployable via Docker and support running via either an
online or offline mode.
YOU MUST INSTALL DOCKER FOR THE FOLLOWING INSTRUCTIONS TO WORK. YOU CAN DOWNLOAD DOCKER HERE.
Install
Running the following commands will create a Docker image called rosetta-komodo:latest.
From GitHub
To download the pre-built Docker image from the latest release, run:
curl -sSfL https://raw.githubusercontent.com/DeckerSU/rosetta-komodo/rosetta-komodo/install.sh | sh -s
Do not try to install rosetta-komodo using GitHub Packages!
From Source
After cloning this repository, run:
make build-local
Run
Running the following commands will start a Docker container in
detached mode with
a data directory at <working directory>/komodo-data and the Rosetta API accessible
at port 8080. Please make sure that <working directory>/komodo-data has nobody:nogroup ownership.
You can also use a named volume which will be created with the correct ownership using: -v "komodo-data:/data".
Mainnet:Online
# create <working directory>/komodo-data with correct ownership
docker run --rm -v "$(pwd)/komodo-data:/data" ubuntu:18.04 bash -c 'chown -R nobody:nogroup /data'
# start rosetta-komodo
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/komodo-data:/data" -e "MODE=ONLINE" -e "NETWORK=MAINNET" -e "PORT=8080" -p 8080:8080 -p 7771:7771 rosetta-komodo:latest
If you cloned the repository, you can run make run-mainnet-online.
Adding options to the komodod conf file
The komodod configuration file can be extended by setting the docker command to /app/rosetta-komodo and using the optional -extend-kmd-conf="" switch. The value of -extend-kmd-conf="" will be appended to /app/komodo-${NETWORK}.conf, newlines can be set as "\n".
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/komodo-data:/data" -e "MODE=ONLINE" -e "NETWORK=MAINNET" -e "PORT=8080" -p 8080:8080 -p 7771:7771 rosetta-komodo:latest /app/rosetta-komodo -extend-kmd-conf="reindexfast=1\ndebug=rpc\ndebug=net"
# this command line would append the following to /app/komodo-mainnet.conf
reindexfast=1
debug=rpc
debug=net
Mainnet:Offline
docker run -d --rm -e "MODE=OFFLINE" -e "NETWORK=MAINNET" -e "PORT=8081" -p 8081:8081 rosetta-komodo:latest
If you cloned the repository, you can run make run-mainnet-offline.
Testnet:Online
# create <working directory>/komodo-data with correct ownership
docker run --rm -v "$(pwd)/komodo-data:/data" ubuntu:18.04 bash -c 'chown -R nobody:nogroup /data'
# start rosetta-komodo
docker run -d --rm --ulimit "nofile=100000:100000" -v "$(pwd)/komodo-data:/data" -e "MODE=ONLINE" -e "NETWORK=TESTNET" -e "PORT=8080" -p 8080:8080 -p 19033:19033 rosetta-komodo:latest
If you cloned the repository, you can run make run-testnet-online.
Testnet:Offline
docker run -d --rm -e "MODE=OFFLINE" -e "NETWORK=TESTNET" -e "PORT=8081" -p 8081:8081 rosetta-komodo:latest
If you cloned the repository, you can run make run-testnet-offline.
Network Settings
To increase the load rosetta-komodo can handle, it is recommended to tune your OS
settings to allow for more connections. On a linux-based OS, you can run the following
commands (source):
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_max_syn_backlog=10000
sysctl -w net.core.somaxconn=10000
sysctl -p (when done)
We have not tested rosetta-komodo with net.ipv4.tcp_tw_recycle and do not recommend
enabling it.
You should also modify your open file settings to 100000. This can be done on a linux-based OS
with the command: ulimit -n 100000.
Memory-Mapped Files
rosetta-komodo uses memory-mapped files to
persist data in the indexer. As a result, you must run rosetta-komodo on a 64-bit
architecture (the virtual address space easily exceeds 100s of GBs).
If you receive a kernel OOM, you may need to increase the allocated size of swap space on your OS. There is a great tutorial for how to do this on Linux here.
Architecture
rosetta-komodo uses the syncer, storage, parser, and server package
from rosetta-sdk-go instead
of a new Komodo-specific implementation of packages of similar functionality. Below
you can find a high-level overview of how everything fits together:
+------------------------------------------------------------------+
| |
| +--------------------------------------+ |
| | | |
| | indexer | |
| | | |
| | +--------+ | |
+-------------------+ pruner <----------+ | |
| | +--------+ | | |
+-----v----+ | | | |
| komodod | | +------+--------+ | |
+-----+----+ | +--------> block_storage <----+ | |
| | | +---------------+ | | |
| | +---+----+ | | |
+-------------------> syncer | | | |
| +---+----+ | | |
| | +--------------+ | | |
| +--------> coin_storage | | | |
| +------^-------+ | | |
| | | | |
+--------------------------------------+ |
| | |
+-------------------------------------------------------------------------------------------+ |
| | | | |
| +------------------------------------------------------------+ | | |
| | | | |
| | +---------------------+-----------------------+------+ | |
| | | | | | |
| +-------+---------+ +-------+---------+ +-------+-------+ +-----------+----------+ | |
| | account_service | | network_service | | block_service | | construction_service +--------+
| +-----------------+ +-----------------+ +---------------+ +----------------------+ |
| |
| server |
| |
+-------------------------------------------------------------------------------------------+
Optimizations
- Reduce sync time with concurrent block indexing
- Use Zstandard compression to reduce the size of data stored on disk without needing to write a manual byte-level encoding
Concurrent Block Syncing
To speed up indexing, rosetta-komodo uses concurrent block processing
with a "wait free" design (using channels instead of sleeps to signal
which threads are unblocked). This allows rosetta-komodo to fetch
multiple inputs from disk while it waits for inputs that appeared
in recently processed blocks to save to disk.
+----------+
| komodod |
+-----+----+
|
|
+---------+ fetch block data / unpopulated txs |
| block 1 <------------------------------------+
+---------+ |
+--> tx 1 | |
| +---------+ |
| | tx 2 | |
| +----+----+ |
| | |
| | +---------+ |
| | | block 2 <-------------------+
| | +---------+ |
| +-----------> tx 3 +--+ |
| +---------+ | |
+-------------------> tx 4 | | |
| +---------+ | |
| | |
| retrieve previously synced | +---------+ |
| inputs needed for future | | block 3 <--+
| blocks while waiting for | +---------+
| populated blocks to save to +---> tx 5 |
| disk +---------+
+------------------------------------> tx 6 |
| +---------+
|
|
+------+--------+
| coin_storage |
+---------------+
Testing with rosetta-cli
To validate rosetta-komodo, install rosetta-cli
and run one of the following commands:
rosetta-cli check:data --configuration-file rosetta-cli-conf/komodo_testnet.jsonrosetta-cli check:construction --configuration-file rosetta-cli-conf/komodo_testnet.json
Development
make depsto install dependenciesmake testto run testsmake lintto lint the source codemake salusto check for security concernsmake build-localto build a Docker image from the local contextmake coverage-localto generate a coverage report
License
This project is available open source under the terms of the Apache 2.0 License.
© 2020 Coinbase
© 2020 Zen Blockchain Foundation
© 2023 Horizen Foundation
© 2023 DCRDEX Developers
© 2024 Komodo Platform
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
contrib
|
|
|
blake2b
Package blake2b implements BLAKE2b cryptographic hash function.
|
Package blake2b implements BLAKE2b cryptographic hash function. |
|
btcec
Package btcec implements support for the elliptic curves needed for bitcoin.
|
Package btcec implements support for the elliptic curves needed for bitcoin. |
|
chaincfg
Package chaincfg defines chain configuration parameters.
|
Package chaincfg defines chain configuration parameters. |
|
chaincfg/chainhash
Package chainhash provides abstracted hash functionality.
|
Package chainhash provides abstracted hash functionality. |
|
txscript
Package txscript implements the bitcoin transaction script language.
|
Package txscript implements the bitcoin transaction script language. |
|
wire
Package wire implements the bitcoin wire protocol.
|
Package wire implements the bitcoin wire protocol. |
|
package komodoutil provides komodo-specific convenience functions and types.
|
package komodoutil provides komodo-specific convenience functions and types. |
|
base58
Package base58 provides an API for working with modified base58 and Base58Check encodings.
|
Package base58 provides an API for working with modified base58 and Base58Check encodings. |
|
mocks
|
|