bcs

Documentation

Overview

bcs (binary canonical serialization) or lcs (libra canonical serialization) is developed from the shuttered libra/diem block chain project.

bcs defines how a struct in rust-lang can be serialized into bytes, and supports features that are unavaibable in golang, such as tagged union or enum.

By "canonical", it means the serialization is deterministic and unique. On many move-lang based blockchains, bcs is the serialization scheme for the struct in move.

See Marshal and Unmarshal for details.

Index

• Constants
• func Marshal(v any) ([]byte, error)
• func MustMarshal(v any) []byte
• func NewBigIntFromUint64(i uint64) *big.Int
• func ULEB128Decode[T ULEB128SupportedTypes](r io.Reader) (T, int, error)
• func ULEB128Encode[T ULEB128SupportedTypes](input T) []byte
• func Unmarshal(data []byte, v any) (int, error)
• func UnmarshalAll(data []byte, v any) error
• type Decoder
  • func NewDecoder(r io.Reader) *Decoder
  • func (d *Decoder) Decode(v any) (int, error)
• type Encoder
  • func NewEncoder(w io.Writer) *Encoder
  • func (e *Encoder) Encode(v any) error
• type Enum
• type Marshaler
• type Option
  • func (p *Option[T]) MarshalBCS() ([]byte, error)
  • func (p *Option[T]) UnmarshalBCS(r io.Reader) (int, error)
• type ULEB128SupportedTypes
• type Uint128
  • func NewUint128(s string) (*Uint128, error)
  • func NewUint128FromBigInt(bigI *big.Int) (*Uint128, error)
  • func NewUint128FromUint64(lo, hi uint64) *Uint128
  • func (i Uint128) Big() *big.Int
  • func (i *Uint128) Cmp(j *Uint128) int
  • func (i Uint128) MarshalBCS() ([]byte, error)
  • func (i Uint128) MarshalJSON() ([]byte, error)
  • func (i *Uint128) SetBigInt(bigI *big.Int) error
  • func (u Uint128) String() string
  • func (i *Uint128) UnmarshalBCS(r io.Reader) (int, error)
  • func (i *Uint128) UnmarshalJSON(data []byte) error
  • func (i *Uint128) UnmarshalText(data []byte) error
• type Unmarshaler

Examples

• Enum

Constants

const MaxUleb128Length = 10

MaxUleb128Length is the max possible number of bytes for an ULEB128 encoded integer. Go's widest integer is uint64, so the length is 10.

Functions

func Marshal

func Marshal(v any) ([]byte, error)

Marshal a value into bcs bytes.

Many constructs supported by bcs don't exist in golang or move-lang.

• Enum is used to simulate the effects of rust enum.
• Use tag `optional` to indicate an optional value in rust. the field must be pointer or interface.
• Use tag `-` to ignore fields.
• Unexported fields are ignored.

Note that bcs doesn't have schema, and field names are irrelevant. The fields of struct are serialized in the order that they are defined.

Pointers are serialized as the type they point to. Nil pointers will be serialized as zero value of the type they point to unless it's marked as `optional`.

Arrays are serialized as fixed length vector (or serialize the each object individually without prefixing the length of the array).

Vanilla maps are not supported, however, the code will error if map is encountered to call out they are not supported and either ignore or provide a customized marshal function.

Channels, functions are silently ignored.

During marshalling process, how v is marshalled depends on if v implemented Marshaler or Enum

1. if Marshaler, use "MarshalBCS" method.
2. if not Marshaler but Enum, use specialization for Enum.
3. otherwise standard process.

func MustMarshal

func MustMarshal(v any) []byte

MustMarshal Marshal v, and panics if error.

func NewBigIntFromUint64

func NewBigIntFromUint64(i uint64) *big.Int

func ULEB128Decode

func ULEB128Decode[T ULEB128SupportedTypes](r io.Reader) (T, int, error)

ULEB128Decode decodes io.Reader into an integer, returns the resulted value, the number of byte read, and a possible error.

binary.ReadUvarint is not used here because

• it doesn't support returning the number of bytes read.
• it accepts only io.ByteReader, but the recommended way of creating one from bufio.NewReader will read more than 1 byte at the to fill the buffer.

func ULEB128Encode

func ULEB128Encode[T ULEB128SupportedTypes](input T) []byte

ULEB128Encode converts an integer into []byte (see wikipedia and bcs)

This reuses binary.PutUvarint in standard library.

func Unmarshal

func Unmarshal(data []byte, v any) (int, error)

Unmarshal unmarshals the bcs serialized data into v.

Refer to notes in Marshal for details how data serialized/deserialized.

During the unmarshalling process

1. if Unmarshaler, use "UnmarshalBCS" method.
2. if not Unmarshaler but Enum, use the specialization for Enum.
3. otherwise standard process.

func UnmarshalAll

func UnmarshalAll(data []byte, v any) error

UnmarshalAll is like Unmarshal, but will additionally error if the input bytes are not completely consumed by the call to Unmarshal.

This is useful for ensuring that a particular set of bytes *completely* represents the data it is decoded to (i.e. no bytes are left over after decoding).

Types

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder takes an io.Reader and decodes value from it.

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

NewDecoder creates a new Decoder from an io.Reader

func (*Decoder) Decode

func (d *Decoder) Decode(v any) (int, error)

DecodeWithSize decodes a value from the decoder, and returns the number of bytes it consumed from the decoder.

• If the value is Unmarshaler, the corresponding UnmarshalBCS will be called.
• If the value is Enum, it will be special handled for Enum

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder takes an io.Writer and encodes value into it.

func NewEncoder

func NewEncoder(w io.Writer) *Encoder

NewEncoder creates a new Encoder from an io.Writer

func (*Encoder) Encode

func (e *Encoder) Encode(v any) error

Encode a value v into the encoder.

• If the value is Marshaler, the corresponding MarshalBCS implementation will be called.
• If the value is Enum, it will be special handled for Enum.

type Enum

type Enum interface {
	// IsBcsEnum doesn't do anything. Its function is to indicate this is an enum for bcs de/serialization.
	IsBcsEnum()
}

Enum emulates the rust enum, contains only one method, IsBcsEnum, to indicate this is an enum in bcs.

All the fields of the enum type must be pointers or interfaces except those ignored by "-". The index of the field in the field list is the integer value of the enum.

type AEnum struct {
  V0 *uint8 // variant 0
  V1 *uint16 `bcs:"-"` // ignored, so variant 1 is invalid
  V2 int `bcs:"-"` // cannot be set to nil, so variant 2 is invalid
  V3 *uint8 // variant 3
  v4 uint32 // Unexported, so ignored.
}
// IsBcsEnum doesn't do anything besides indicating this is an Enum for BCS.
func (a AEnum) IsBcsEnum() {}

If there are mulitple non-nil fields when marshalling, the first one encountered will be serialized.

The method IsBcsEnum doesn't actually do anything besides acting as an indicator.

Example

package main

import (
	"fmt"

	"github.com/fardream/go-bcs/bcs"
)

type AnotherStruct struct {
	S string
}

type EnumExample struct {
	V0 *uint8
	V1 *uint16 `bcs:"-"`
	V2 *uint32
	v3 uint8
	V4 *AnotherStruct
}

// IsBcsEnum tells this is an enum
func (e EnumExample) IsBcsEnum() {}

func main() {
	// an enum
	v0 := &EnumExample{
		V0: new(uint8),
	}
	*v0.V0 = 42

	v0m, err := bcs.Marshal(v0)
	if err != nil {
		panic(err)
	}
	fmt.Println("v0:", v0m)

	// first value will be picked up
	v1 := &EnumExample{
		V0: new(uint8),
		V2: new(uint32),
	}
	*v1.V0 = 0
	*v1.V2 = 10
	v1m, err := bcs.Marshal(v1)
	if err != nil {
		panic(err)
	}
	fmt.Println("v1:", v1m)

	// enum must be set
	v2 := &EnumExample{}

	_, err = bcs.Marshal(v2)
	if err == nil {
		panic(fmt.Errorf("unset enum should error out"))
	}

	// setting V1, which is ignored, and v3, which is unexported, should be ignored
	v3 := &EnumExample{
		V1: new(uint16),
		v3: 90,
		V4: &AnotherStruct{
			S: "abc",
		},
	}

	v3m, err := bcs.Marshal(v3)
	if err != nil {
		panic(err)
	}
	// print [4 3 97 98 99], which is 4 (enum int), 3 (size of the bytes of the string), 97=a, 98=b, 99=c
	fmt.Println("v3:", v3m)
}

Output:

v0: [0 42]
v1: [0 0]
v3: [4 3 97 98 99]

type Marshaler

type Marshaler interface {
	MarshalBCS() ([]byte, error)
}

Marshaler customizes the marshalling behavior for a type

type Option

type Option[T any] struct {
	Some T
	None bool
}

func (*Option[T]) MarshalBCS

func (p *Option[T]) MarshalBCS() ([]byte, error)

func (*Option[T]) UnmarshalBCS

func (p *Option[T]) UnmarshalBCS(r io.Reader) (int, error)

type ULEB128SupportedTypes

type ULEB128SupportedTypes interface {
	~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uint | ~int8 | ~int16 | ~int32 | ~int64 | ~int
}

ULEB128SupportedTypes is a contraint interface that limits the input to ULEB128Encode and ULEB128Decode to signed and unsigned integers.

type Uint128

type Uint128 struct {
	// contains filtered or unexported fields
}

Uint128 is like `u128` in move.

func NewUint128

func NewUint128(s string) (*Uint128, error)

func NewUint128FromBigInt

func NewUint128FromBigInt(bigI *big.Int) (*Uint128, error)

func NewUint128FromUint64

func NewUint128FromUint64(lo, hi uint64) *Uint128

func (Uint128) Big

func (i Uint128) Big() *big.Int

func (*Uint128) Cmp

func (i *Uint128) Cmp(j *Uint128) int

func (Uint128) MarshalBCS

func (i Uint128) MarshalBCS() ([]byte, error)

func (Uint128) MarshalJSON

func (i Uint128) MarshalJSON() ([]byte, error)

func (*Uint128) SetBigInt

func (i *Uint128) SetBigInt(bigI *big.Int) error

func (Uint128) String

func (u Uint128) String() string

func (*Uint128) UnmarshalBCS

func (i *Uint128) UnmarshalBCS(r io.Reader) (int, error)

func (*Uint128) UnmarshalJSON

func (i *Uint128) UnmarshalJSON(data []byte) error

func (*Uint128) UnmarshalText

func (i *Uint128) UnmarshalText(data []byte) error

type Unmarshaler

type Unmarshaler interface {
	UnmarshalBCS(io.Reader) (int, error)
}

Unmarshaler customizes the unmarshalling behavior for a type.

Compared with other Unmarshalers in golang, the Unmarshaler here takes a io.Reader instead of []byte, since it is difficult to delimit the byte streams without unmarshalling. Method [UnmarshalBCS] returns the number of bytes read, and potentially an error.