weave

README

Iov Weave

IOV Weave is a framework for quickly building your custom ABCI application to run a blockchain on top of the best-of-class BFT Proof-of-stake Tendermint consensus engine. It provides much commonly used functionality that can quickly be imported in your custom chain, as well as a simple framework for adding the custom functionality unique to your project.

Note: Requires Go 1.9+

It is inspired by the routing and middleware model of many web application frameworks, and informed by years of wrestling with blockchain state machines. More directly, it is based on the official cosmos-sdk, both the 0.8 release as well as the future 0.9 rewrite. Naturally, as I was the main author of 0.8.

While both of those are extremely powerful and flexible and contain advanced features, they have a steep learning curve for novice users. Thus, this library aims to favor simplicity over power when there is a choice. If you hit limitations in the design of this library (such as maintaining multiple merkle stores in one app), I highly advise you to use the official cosmos sdk.

On the other hand, if you want to try out tendermint, or have a design that doesn't require an advanced setup, you should try this library and give feedback, especially on ease-of-use. The end goal is to make blockchain development almost as productive as web development (in golang), by providing defaults and best practices for many choices, while allowing extreme flexibility in business logic and data modelling.

For more details on the design goals, see the Design Document

Prerequisites

• golang 1.10+

Instructions

First, make sure you have set up the requirements. If you have a solid go and node developer setup, you may skip this, but good to go through it to be sure.

Once you are set up, you should be able to run something like the following to compile both mycoind (sample app) and tendermint (a BFT consensus engine):

go get github.com/iov-one/weave
cd $GOPATH/src/github.com/iov-one/weave
make deps
make install
# test it built properly
tendermint version
# 0.29.1-xxxxx
mycoind version
# v0.11.1

Note that this app relies on a separate tendermint process to drive it. It is helpful to first read a primer on tendermint as well as the documentation on the tendermint cli commands.

Once it compiles, I highly suggest going through the tutorials on readthedocs

Compatibility Charts

Weave	Tendermint
v0.11.1	v0.29.1
v0.11.0	v0.27.4
v0.10.x	v0.27.4
v0.9.3	v0.25.0

Weave	Protobuf compatible to previous version	Comments
v0.12.0	❓	Currency no longer supports supplying SigFigs nor returns them
v0.11.1	✔
v0.11.0	✔
v0.10.2	✔
v0.10.1	❌
v0.10.0	❌
v0.9.3	✔

Protobuf Documentation

We generate documentation from the *.proto files to keep it up to date.

You can view the documentation for all packages used in the bns app.

Or generate it yourself:

make prototools
make protodocs
open ./docs/proto/index.html

History

The original version, until v0.6.0 was released under confio/weave. The original author, Ethan Frey, had previously worked on the Cosmos SDK and wanted to make a simpler framework he could use to start building demo apps, while the main sdk matured. Thus, confio/weave was born the first few months of 2018. This framework was designed to be open source and shared, but the only real usage and development was by IOV, so it was donated to that organization in August 2018 to be developed further for their BNS blockchain, as well as a companion to iov-core client libraries that deprecated confio/weave-js

Documentation

Overview

Package weave defines all common interfaces to weave together the various subpackages, as well as implementations of some of the simpler components (when interfaces would be too much overhead).

We pass context through context.Context between app, middleware, and handlers. To do so, weave defines some common keys to store info, such as block height and chain id. Each extension, such as auth, may add its own keys to enrich the context with specific data.

There should exist two functions for every XYZ of type T that we want to support in Context:

WithXYZ(Context, T) Context
GetXYZ(Context) (val T, ok bool)

WithXYZ may error/panic if the value was previously set to avoid lower-level modules overwriting the value (eg. height, header)

Index

• Constants
• Variables
• func BlockTime(ctx Context) (time.Time, bool)
• func CheckOrError(result CheckResult, err error, debug bool) abci.ResponseCheckTx
• func CheckTxError(err error, debug bool) abci.ResponseCheckTx
• func DeliverOrError(result DeliverResult, err error, debug bool) abci.ResponseDeliverTx
• func DeliverTxError(err error, debug bool) abci.ResponseDeliverTx
• func GetChainID(ctx Context) string
• func GetHeader(ctx Context) (abci.Header, bool)
• func GetHeight(ctx Context) (int64, bool)
• func GetLogger(ctx Context) log.Logger
• func GetPath(tx Tx) string
• type Address
  • func NewAddress(data []byte) Address
  • func (a Address) Equals(b Address) bool
  • func (a Address) MarshalJSON() ([]byte, error)
  • func (a Address) String() string
  • func (a *Address) UnmarshalJSON(raw []byte) error
  • func (a Address) Validate() error
• type Batch
• type CacheableKVStore
• type CheckResult
  • func NewCheck(gasAllocated int64, log string) CheckResult
  • func (c CheckResult) ToABCI() abci.ResponseCheckTx
• type Checker
• type CommitID
• type CommitKVStore
• type Condition
  • func NewCondition(ext, typ string, data []byte) Condition
  • func (c Condition) Address() Address
  • func (a Condition) Equals(b Condition) bool
  • func (c Condition) MarshalJSON() ([]byte, error)
  • func (c Condition) Parse() (string, string, []byte, error)
  • func (c Condition) String() string
  • func (c *Condition) UnmarshalJSON(raw []byte) error
  • func (c Condition) Validate() error
• type Context
  • func WithBlockTime(ctx Context, t time.Time) Context
  • func WithChainID(ctx Context, chainID string) Context
  • func WithHeader(ctx Context, header abci.Header) Context
  • func WithHeight(ctx Context, height int64) Context
  • func WithLogInfo(ctx Context, keyvals ...interface{}) Context
  • func WithLogger(ctx Context, logger log.Logger) Context
• type Decorator
• type DeliverResult
  • func (d DeliverResult) ToABCI() abci.ResponseDeliverTx
• type Deliverer
• type Handler
• type Initializer
• type Iterator
• type KVCacheWrap
• type KVStore
• type Marshaller
• type Model
  • func Pair(key, value []byte) Model
• type Msg
  • func ExtractMsgFromSum(sum interface{}) (Msg, error)
• type Options
  • func (o Options) ReadOptions(key string, obj interface{}) error
• type Persistent
• type QueryHandler
• type QueryRegister
• type QueryRouter
  • func NewQueryRouter() QueryRouter
  • func (r QueryRouter) Handler(path string) QueryHandler
  • func (r QueryRouter) Register(path string, h QueryHandler)
  • func (r QueryRouter) RegisterAll(qr ...QueryRegister)
• type ReadOnlyKVStore
• type Registry
• type SetDeleter
• type TickResult
• type Ticker
• type Tx
• type TxDecoder

Constants

const (
	// KeyQueryMod means to query for exact match (key)
	KeyQueryMod = ""
	// PrefixQueryMod means to query for anything with this prefix
	PrefixQueryMod = "prefix"
	// RangeQueryMod means to expect complex range query
	// TODO: implement
	RangeQueryMod = "range"
)

const Fix = 0

Fix is the patch number (updated on bugfix releases)

const Maj = 0

Maj is the major version number (updated on breaking release)

const Min = 7

Min is the minor version number (updated on minor releases)

Variables

var (
	// DefaultLogger is used for all context that have not
	// set anything themselves
	DefaultLogger = log.NewNopLogger()

	// IsValidChainID is the RegExp to ensure valid chain IDs
	IsValidChainID = regexp.MustCompile(`^[a-zA-Z0-9_\-]{6,20}$`).MatchString
)

var (
	// AddressLength is the length of all addresses
	// You can modify it in init() before any addresses are calculated,
	// but it must not change during the lifetime of the kvstore
	AddressLength = 20
)

var Version = "please set in makefile"

Version should be set by build flags: `git describe --tags`

Functions

func BlockTime

func BlockTime(ctx Context) (time.Time, bool)

BlockTime returns current block wall clock time as declared in the context. Zero time and false is returned if a block time is not present in the context.

func CheckOrError

func CheckOrError(result CheckResult, err error, debug bool) abci.ResponseCheckTx

CheckOrError returns an abci response for CheckTx, converting the error message if present, or using the successful CheckResult

func CheckTxError

func CheckTxError(err error, debug bool) abci.ResponseCheckTx

CheckTxError converts any error into a abci.ResponseCheckTx, preserving as much info as possible. When in debug mode always the full error information is returned.

func DeliverOrError

func DeliverOrError(result DeliverResult, err error, debug bool) abci.ResponseDeliverTx

DeliverOrError returns an abci response for DeliverTx, converting the error message if present, or using the successful DeliverResult

func DeliverTxError

func DeliverTxError(err error, debug bool) abci.ResponseDeliverTx

DeliverTxError converts any error into a abci.ResponseDeliverTx, preserving as much info as possible. When in debug mode always the full error information is returned.

func GetChainID

func GetChainID(ctx Context) string

GetChainID returns the current chain id panics if chain id not already set (should never happen)

func GetHeader

func GetHeader(ctx Context) (abci.Header, bool)

GetHeader returns the current block header ok is false if no header set in this Context

func GetHeight

func GetHeight(ctx Context) (int64, bool)

GetHeight returns the current block height ok is false if no height set in this Context

func GetLogger

func GetLogger(ctx Context) log.Logger

GetLogger returns the currently set logger, or DefaultLogger if none was set

func GetPath

func GetPath(tx Tx) string

GetPath returns the path of the message, or (missing) if no message

Types

type Address

type Address []byte

Address represents a collision-free, one-way digest of a Condition

It will be of size AddressLength

func NewAddress

func NewAddress(data []byte) Address

NewAddress hashes and truncates into the proper size

func (Address) Equals

func (a Address) Equals(b Address) bool

Equals checks if two addresses are the same

func (Address) MarshalJSON

func (a Address) MarshalJSON() ([]byte, error)

MarshalJSON provides a hex representation for JSON, to override the standard base64 []byte encoding

func (Address) String

func (a Address) String() string

String returns a human readable string. Currently hex, may move to bech32

func (*Address) UnmarshalJSON

func (a *Address) UnmarshalJSON(raw []byte) error

func (Address) Validate

func (a Address) Validate() error

Validate returns an error if the address is not the valid size

type Batch

type Batch interface {
	SetDeleter
	Write()
}

Batch can write multiple ops atomically to an underlying KVStore

type CacheableKVStore

type CacheableKVStore interface {
	KVStore
	CacheWrap() KVCacheWrap
}

CacheableKVStore is a KVStore that supports CacheWrapping

CacheWrap() should not return a Committer, since Commit() on cache-wraps make no sense.

type CheckResult

type CheckResult struct {
	// Data is a machine-parseable return value, like id of created entity
	Data []byte
	// Log is human-readable informational string
	Log string
	// RequiredFee can set an custom fee that must be paid for this transaction to be allowed to run.
	// This may enforced by a decorator, such as cash.DynamicFeeDecorator
	RequiredFee coin.Coin
	// GasAllocated is the maximum units of work we allow this tx to perform
	GasAllocated int64
	// GasPayment is the total fees for this tx (or other source of payment)
	//TODO: Implement when tendermint implements this properly
	GasPayment int64
}

CheckResult captures any non-error abci result to make sure people use error for error cases

func NewCheck

func NewCheck(gasAllocated int64, log string) CheckResult

NewCheck sets the gas used and the response data but no more info these are the most common info needed to be set by the Handler

func (CheckResult) ToABCI

func (c CheckResult) ToABCI() abci.ResponseCheckTx

ToABCI converts our internal type into an abci response

type Checker

type Checker interface {
	Check(ctx Context, store KVStore, tx Tx) (CheckResult, error)
}

Checker is a subset of Handler to verify the validity of a transaction. It is its own interface to allow better type controls in the next arguments in Decorator

type CommitID

type CommitID struct {
	Version int64
	Hash    []byte
}

CommitID contains the tree version number and its merkle root.

type CommitKVStore

type CommitKVStore interface {
	// Get returns the value at last committed state
	// returns nil iff key doesn't exist. Panics on nil key.
	Get(key []byte) []byte

	// Get a CacheWrap to perform actions
	// TODO: add Batch to atomic writes and efficiency
	// invisibly inside this CacheWrap???
	CacheWrap() KVCacheWrap

	// Commit the next version to disk, and returns info
	Commit() CommitID

	// LoadLatestVersion loads the latest persisted version.
	// If there was a crash during the last commit, it is guaranteed
	// to return a stable state, even if older.
	LoadLatestVersion() error

	// LatestVersion returns info on the latest version saved to disk
	LatestVersion() CommitID
}

CommitKVStore is a root store that can make atomic commits to disk. We modify it in batch by getting a CacheWrap() and then Write(). Commit() will persist all changes to disk

This store should also be able to return merkle proofs for any committed state.

type Condition

type Condition []byte

Condition is a specially formatted array, containing information on who can authorize an action. It is of the format:

sprintf("%s/%s/%s", extension, type, data)

func NewCondition

func NewCondition(ext, typ string, data []byte) Condition

func (Condition) Address

func (c Condition) Address() Address

Address will convert a Condition into an Address

func (Condition) Equals

func (a Condition) Equals(b Condition) bool

Equals checks if two permissions are the same

func (Condition) MarshalJSON

func (c Condition) MarshalJSON() ([]byte, error)

func (Condition) Parse

func (c Condition) Parse() (string, string, []byte, error)

Parse will extract the sections from the Condition bytes and verify it is properly formatted

func (Condition) String

func (c Condition) String() string

String returns a human readable string. We keep the extension and type in ascii and hex-encode the binary data

func (*Condition) UnmarshalJSON

func (c *Condition) UnmarshalJSON(raw []byte) error

func (Condition) Validate

func (c Condition) Validate() error

Validate returns an error if the Condition is not the proper format

type Context

type Context = context.Context

Context is just an alias for the standard implementation. We use functions to extend it to our domain

func WithBlockTime

func WithBlockTime(ctx Context, t time.Time) Context

WithBlockTime sets the block time for the context. Block time is always represented in UTC.

func WithChainID

func WithChainID(ctx Context, chainID string) Context

WithChainID sets the chain id for the Context. panics if called with chain id already set

func WithHeader

func WithHeader(ctx Context, header abci.Header) Context

WithHeader sets the block header for the Context. panics if called with header already set

func WithHeight

func WithHeight(ctx Context, height int64) Context

WithHeight sets the block height for the Context. panics if called with height already set

func WithLogInfo

func WithLogInfo(ctx Context, keyvals ...interface{}) Context

WithLogInfo accepts keyvalue pairs, and returns another context like this, after passing all the keyvals to the Logger

func WithLogger

func WithLogger(ctx Context, logger log.Logger) Context

WithLogger sets the logger for this Context

type Decorator

type Decorator interface {
	Check(ctx Context, store KVStore, tx Tx, next Checker) (CheckResult, error)
	Deliver(ctx Context, store KVStore, tx Tx, next Deliverer) (DeliverResult, error)
}

Decorator wraps a Handler to provide common functionality like authentication, or fee-handling, to many Handlers

type DeliverResult

type DeliverResult struct {
	// Data is a machine-parseable return value, like id of created entity
	Data []byte
	// Log is human-readable informational string
	Log string
	// RequiredFee can set an custom fee that must be paid for this transaction to be allowed to run.
	// This may enforced by a decorator, such as cash.DynamicFeeDecorator
	RequiredFee coin.Coin
	// Diff, if present, will apply to the Validator set in tendermint next block
	Diff []abci.ValidatorUpdate
	// Tags, if present, will be used by tendermint to index and search the transaction history
	Tags []common.KVPair
	// GasUsed is currently unused field until effects in tendermint are clear
	GasUsed int64
}

DeliverResult captures any non-error abci result to make sure people use error for error cases

func (DeliverResult) ToABCI

func (d DeliverResult) ToABCI() abci.ResponseDeliverTx

ToABCI converts our internal type into an abci response

type Deliverer

type Deliverer interface {
	Deliver(ctx Context, store KVStore, tx Tx) (DeliverResult, error)
}

Deliverer is a subset of Handler to execute a transaction. It is its own interface to allow better type controls in the next arguments in Decorator

type Handler

type Handler interface {
	Checker
	Deliverer
}

Handler is a core engine that can process a few specific messages This could represent "coin transfer", or "bonding stake to a validator"

type Initializer

type Initializer interface {
	FromGenesis(Options, KVStore) error
}

Initializer implementations are used to initialize extensions from genesis file contents

type Iterator

type Iterator interface {
	// Valid returns whether the current position is valid.
	// Once invalid, an Iterator is forever invalid.
	Valid() bool

	// Next moves the iterator to the next sequential key in the database, as
	// defined by order of iteration.
	//
	// If Valid returns false, this method will panic.
	Next()

	// Key returns the key of the cursor.
	// If Valid returns false, this method will panic.
	// CONTRACT: key readonly []byte
	Key() (key []byte)

	// Value returns the value of the cursor.
	// If Valid returns false, this method will panic.
	// CONTRACT: value readonly []byte
	Value() (value []byte)

	// Close releases the Iterator.
	Close()
}

Iterator allows us to access a set of items within a range of keys. These may all be preloaded, or loaded on demand.

Usage:

var itr Iterator = ...
defer itr.Close()

for ; itr.Valid(); itr.Next() {
  k, v := itr.Key(); itr.Value()
  // ...
}

type KVCacheWrap

type KVCacheWrap interface {
	// CacheableKVStore allows us to use this Cache recursively
	CacheableKVStore

	// Write syncs with the underlying store.
	Write()

	// Discard invalidates this CacheWrap and releases all data
	Discard()
}

KVCacheWrap allows us to maintain a scratch-pad of uncommitted data that we can view with all queries.

At the end, call Write to use the cached data, or Discard to drop it.

type KVStore

type KVStore interface {
	ReadOnlyKVStore
	SetDeleter
	// NewBatch returns a batch that can write multiple ops atomically
	NewBatch() Batch
}

KVStore is a simple interface to get/set data

For simplicity, we require all backing stores to implement this interface. They *may* implement other methods as well, but at least these are required.

type Marshaller

type Marshaller interface {
	Marshal() ([]byte, error)
}

Marshaller is anything that can be represented in binary

Marshall may validate the data before serializing it and unless you previously validated the struct, errors should be expected.

type Model

type Model struct {
	Key   []byte
	Value []byte
}

Model groups together key and value to return

func Pair

func Pair(key, value []byte) Model

Pair constructs a model from a key-value pair

type Msg

type Msg interface {
	Persistent

	// Return the message path.
	// This is used by the Router to locate the proper Handler.
	// Msg should be created alongside the Handler that corresponds to them.
	//
	// Multiple types may have the same value, and will end up at the
	// same Handler.
	//
	// Must be alphanumeric [0-9A-Za-z_\-]+
	Path() string
}

Msg is message for the blockchain to take an action (Make a state transition). It is just the request, and must be validated by the Handlers. All authentication information is in the wrapping Tx.

func ExtractMsgFromSum

func ExtractMsgFromSum(sum interface{}) (Msg, error)

ExtractMsgFromSum will find a weave message from a tx sum type if it exists. Assuming you define your Tx with protobuf, this will help you implement GetMsg()

ExtractMsgFromSum(tx.GetSum())

To work, this requires sum to be a pointer to a struct with one field, and that field can be cast to a Msg. Returns an error if it cannot succeed.

type Options

type Options map[string]json.RawMessage

Options are the app options Each extension can look up it's key and parse the json as desired

func (Options) ReadOptions

func (o Options) ReadOptions(key string, obj interface{}) error

ReadOptions reads the values stored under a given key, and parses the json into the given obj. Returns an error if it cannot parse. Noop and no error if key is missing

type Persistent

type Persistent interface {
	Marshaller
	Unmarshal([]byte) error
}

Persistent supports Marshal and Unmarshal

This is separated from Marshal, as this almost always requires a pointer, and functions that only need to marshal bytes can use the Marshaller interface to access non-pointers.

As with Marshaller, this may do internal validation on the data and errors should be expected.

type QueryHandler

type QueryHandler interface {
	Query(db ReadOnlyKVStore, mod string, data []byte) ([]Model, error)
}

QueryHandler is anything that can process ABCI queries

type QueryRegister

type QueryRegister func(QueryRouter)

QueryRegister is a function that adds some handlers to this router

type QueryRouter

type QueryRouter struct {
	// contains filtered or unexported fields
}

QueryRouter allows us to register many query handlers to different paths and then direct each query to the proper handler.

Minimal interface modeled after net/http.ServeMux

func NewQueryRouter

func NewQueryRouter() QueryRouter

NewQueryRouter initializes a QueryRouter with no routes

func (QueryRouter) Handler

func (r QueryRouter) Handler(path string) QueryHandler

Handler returns the registered Handler for this path. If no path is found, returns a noSuchPath Handler Always returns a non-nil Handler

func (QueryRouter) Register

func (r QueryRouter) Register(path string, h QueryHandler)

Register adds a new Handler for the given path. panics if another Handler was already registered

func (QueryRouter) RegisterAll

func (r QueryRouter) RegisterAll(qr ...QueryRegister)

RegisterAll registers a number of QueryRegister at once

type ReadOnlyKVStore

type ReadOnlyKVStore interface {
	// Get returns nil iff key doesn't exist. Panics on nil key.
	Get(key []byte) []byte

	// Has checks if a key exists. Panics on nil key.
	Has(key []byte) bool

	// Iterator over a domain of keys in ascending order. End is exclusive.
	// Start must be less than end, or the Iterator is invalid.
	// CONTRACT: No writes may happen within a domain while an iterator exists over it.
	Iterator(start, end []byte) Iterator

	// ReverseIterator over a domain of keys in descending order. End is exclusive.
	// Start must be greater than end, or the Iterator is invalid.
	// CONTRACT: No writes may happen within a domain while an iterator exists over it.
	ReverseIterator(start, end []byte) Iterator
}

ReadOnlyKVStore is a simple interface to query data.

type Registry

type Registry interface {
	Handle(path string, h Handler)
}

Registry is an interface to register your handler, the setup side of a Router

type SetDeleter

type SetDeleter interface {
	Set(key, value []byte) // CONTRACT: key, value readonly []byte
	Delete(key []byte)     // CONTRACT: key readonly []byte
}

SetDeleter is a minimal interface for writing, Unifying KVStore and Batch

type TickResult

type TickResult struct {
	Diff []abci.ValidatorUpdate
}

TickResult allows the Ticker to modify the validator set

type Ticker

type Ticker interface {
	Tick(ctx Context, store KVStore) (TickResult, error)
}

Ticker is a method that is called the beginning of every block, which can be used to perform periodic or delayed tasks

type Tx

type Tx interface {
	Persistent

	// GetMsg returns the action we wish to communicate
	GetMsg() (Msg, error)
}

Tx represent the data sent from the user to the chain. It includes the actual message, along with information needed to authenticate the sender (cryptographic signatures), and anything else needed to pass through middleware.

Each Application must define their own tx type, which embeds all the middlewares that we wish to use. auth.SignedTx and token.FeeTx are common interfaces that many apps will wish to support.

type TxDecoder

type TxDecoder func(txBytes []byte) (Tx, error)

TxDecoder can parse bytes into a Tx