README
¶
Kwil
The database for Web3.
Kwil is the node software for Kwil Networks. Built with PostgreSQL and CometBFT, Kwil enables byzantine fault tolerant networks to be built on top of relational databases.
Overview
To learn more about high-level Kwil concepts, refer to the Kwil documentation.
To test deploying and using a Kuneiform schema (Kwil's smart contract language) on the Kwil testnet, refer to the Kwil testnet tutorial.
For more information on kwil-db, check out the Kwil node documentation.
Quickstart
Build Instructions
Prerequisites
To build Kwil, you will need to install:
Only Go is required to build directly from the cmd/kwild folder or via go install, although developers may require the other tools.
To run Kwil, PostgreSQL is also required. See the documentation for more information.
Build
The build task will compile kwild and kwil-cli binaries. They will be generated in .build/:
task build
You may also build the individual applications manually:
cd cmd/kwild
go build
Or without even cloning the source repository:
go install github.com/kwilteam/kwil-db/cmd/kwild@v0.9.0
Just replace v0.9.0 with the desired version or latest.
Running kwild
Running kwild requires a PostgreSQL host running. Since the default
configuration of most PostgreSQL packages requires changes for kwild, the
easiest is to run our pre-configured Docker image:
docker run -p 5432:5432 -v kwil-pg-demo:/var/lib/postgresql/data \
--shm-size 256m -e "POSTGRES_HOST_AUTH_METHOD=trust" \
--name kwil-pg-demo kwildb/postgres:latest
The first time this is run, it will pull the kwildb/postgres image from Docker
Hub and create a new persistent Docker volume named kwil-pg-demo. NOTE: This
command requires no authentication with postgres, and should not be used in
production.
task pg may be used to run the above command.
You can then start a single node network using the kwild binary built in the previous section:
# Use the full path to kwild if it is not on your PATH.
kwild start --autogen
With the --autogen flag, the node automatically creates a new random network
and validator key, and the node will begin producing blocks.
For more information on running nodes, and how to run a multi-node network, refer to the Kwil documentation.
Resetting node data
By default, kwild stores all data in ~/.kwild. To reset the data on a deployment, remove the data directory while the node is stopped:
rm -r ~/.kwild
Then delete the PostgreSQL database. If using the Docker image or service, delete the container and it's volume:
docker container rm -f kwil-pg-demo
docker volume rm -f kwil-pg-demo
task pg:clean may be used to run the above commands.
If using a system install of postgres, recreate the database with psql:
psql -U postgres -h 127.0.0.1 -d postgres \
-c "DROP DATABASE IF EXISTS kwild" \
-c "CREATE DATABASE kwild OWNER kwild"
Unified kwild + postgres Quickstart Docker Service
For development purposes, the deployments/compose/kwil folder contains a
Docker Compose service definition that starts both kwild and postgres,
configured so that they will work together out-of-the-box with no additional
configuration changes. Start it by running the following from the
deployments/compose/kwil folder in the repository:
cd contrib/docker/compose/kwil
docker compose up --build -d
With the -d option, the service(s) will be started as background processes. To
stop them or view logs, use the docker container commands or the Docker
Desktop dashboard.
On start, this service definition will create a testnode folder in the same
location as the docker-compose.yml file, and a persistent Docker volume called
kwil_pgkwil for the postgres database cluster files.
This also runs with the --autogen flag, creating a new randomly generated
chain, and is not intended for production use. However, the service definition
may be used as a basis for a customized deployment.
Extensions
Kwil offers an extension system that allows you to extend the functionality of your network (e.g. building network oracles, customizing authentication, running deterministic compute, etc.). To learn more about the types of extensions and how to build them, refer to the extensions directory README.
Contributing
We welcome contributions to kwil-db. To contribute, please read our contributing guidelines.
License
The kwil-db repository (i.e. everything outside of the core directory) is licensed under the Apache License, Version 2.0. See LICENSE for more details.
The kwil Go SDK (i.e. everything inside of the core directory) is licensed under the MIT License. See core/LICENSE.md for more details.
Directories
¶
| Path | Synopsis |
|---|---|
|
_previous
|
|
|
cmd/kwil-admin/nodecfg
Package nodecfg provides functions to assist in the generation of new kwild node configurations.
|
Package nodecfg provides functions to assist in the generation of new kwild node configurations. |
|
cmd/kwild/config
Package config provides types and functions for node configuration loading and generation.
|
Package config provides types and functions for node configuration loading and generation. |
|
cmd/kwild/server
Package server defines the main Kwil server, which includes the blockchain node and the gRPC services that interface with the Kwil dataset engine.
|
Package server defines the main Kwil server, which includes the blockchain node and the gRPC services that interface with the Kwil dataset engine. |
|
common/sql
Package sql defines common type required by SQL database implementations and consumers.
|
Package sql defines common type required by SQL database implementations and consumers. |
|
extensions/auth
Package auth contains any known Authenticator extensions that may be selected at build-time for use in kwild.
|
Package auth contains any known Authenticator extensions that may be selected at build-time for use in kwild. |
|
extensions/consensus
Package consensus is used to apply customized rules for activation of hardforks defined in the genesis file.
|
Package consensus is used to apply customized rules for activation of hardforks defined in the genesis file. |
|
internal/abci/meta
Package meta defines a chain metadata store for the ABCI application.
|
Package meta defines a chain metadata store for the ABCI application. |
|
custom
package custom allows for the creation of a custom-branded CLI that packages together the kwil-cli, kwil-admin, and kwild CLIs.
|
package custom allows for the creation of a custom-branded CLI that packages together the kwil-cli, kwil-admin, and kwild CLIs. |
|
node/conf
Package conf is used to merge multiple node config sources.
|
Package conf is used to merge multiple node config sources. |
|
shared/display
Package display provides interfaces and functions to format the command line output and print.
|
Package display provides interfaces and functions to format the command line output and print. |
|
cmd
|
|
|
core
module
|
|
|
auth
Package auth contains any known Authenticator extensions that may be selected at build-time for use in kwild.
|
Package auth contains any known Authenticator extensions that may be selected at build-time for use in kwild. |
|
consensus
Package consensus is used to apply customized rules for activation of hardforks defined in the genesis file.
|
Package consensus is used to apply customized rules for activation of hardforks defined in the genesis file. |
|
listeners
package listeners provides the interface and registration for custom event-driven listeners.
|
package listeners provides the interface and registration for custom event-driven listeners. |
|
listeners/eth_deposits
package ethdeposits implements an listener that listens to Ethereum events and triggers the creation of deposit events in Kwil.
|
package ethdeposits implements an listener that listens to Ethereum events and triggers the creation of deposit events in Kwil. |
|
precompiles
package actions allows custom actions to be registered with the engine.
|
package actions allows custom actions to be registered with the engine. |
|
resolutions
package resolutions contains the interface and registration for resolution types.
|
package resolutions contains the interface and registration for resolution types. |
|
resolutions/credit
package credit implements a credit resolution, allowing accounts to be credited with a given amount.
|
package credit implements a credit resolution, allowing accounts to be credited with a given amount. |
|
admin
package adminclient provides a client for the Kwil admin service.
|
package adminclient provides a client for the Kwil admin service. |
|
engine/interpreter
package interpreter provides a basic interpreter for Kuneiform procedures.
|
package interpreter provides a basic interpreter for Kuneiform procedures. |
|
engine/parse
package parse contains logic for parsing SQL, DDL, and Actions, and SQL.
|
package parse contains logic for parsing SQL, DDL, and Actions, and SQL. |
|
engine/pg_generate
pggenerate package is responsible for generating the Postgres-compatible SQL from the AST.
|
pggenerate package is responsible for generating the Postgres-compatible SQL from the AST. |
|
exts/erc20reward
This file contains reward extension related types and database operations.
|
This file contains reward extension related types and database operations. |
|
exts/erc20reward/meta
Package meta is the reward_meta extension.
|
Package meta is the reward_meta extension. |
|
exts/evm-sync/chains
package chains tracks the EVM chains that are supported by the node.
|
package chains tracks the EVM chains that are supported by the node. |
|
exts/ordered-sync
package orderedsync is a general purpose extension that synchronizes data from systems where absolute order is guaranteed (e.g.
|
package orderedsync is a general purpose extension that synchronizes data from systems where absolute order is guaranteed (e.g. |
|
exts/poll
package poll implements a basic polling mechanism for Kwil event listeners
|
package poll implements a basic polling mechanism for Kwil event listeners |
|
meta
Package meta defines a chain metadata store for the ABCI application.
|
Package meta defines a chain metadata store for the ABCI application. |
|
migrations
package migrations implements a long-running migrations protocol for Kwil.
|
package migrations implements a long-running migrations protocol for Kwil. |
|
pg
Package pg defines the primary PostgreSQL-powered DB and Pool types used to support Kwil DB.
|
Package pg defines the primary PostgreSQL-powered DB and Pool types used to support Kwil DB. |
|
store/memstore
Package memstore provides a memory-backed block store, which is only suitable for testing where a disk-based store or third party dependencies are not desired.
|
Package memstore provides a memory-backed block store, which is only suitable for testing where a disk-based store or third party dependencies are not desired. |
|
txapp
package tx_router routes transactions to the appropriate module(s)
|
package tx_router routes transactions to the appropriate module(s) |
|
types/sql
Package sql defines common type required by SQL database implementations and consumers.
|
Package sql defines common type required by SQL database implementations and consumers. |
|
utils/syncmap
Package syncmap provides a map that is safe for concurrent use.
|
Package syncmap provides a map that is safe for concurrent use. |
|
utils/url
package url provides url fuctionalities to provide consistent parsing for Kwil clients.
|
package url provides url fuctionalities to provide consistent parsing for Kwil clients. |
|
versioning
package versioning provides standard schema versioning for Kwil databases.
|
package versioning provides standard schema versioning for Kwil databases. |
|
voting
package events is used to track events that need to be included in a Kwil block.
|
package events is used to track events that need to be included in a Kwil block. |
|
parse
module
|
|
|
test
module
|
|
|
package testing provides tools for testing Kuneiform schemas.
|
package testing provides tools for testing Kuneiform schemas. |