Documentation ¶
Overview ¶
Amino is an encoding library that can handle interfaces (like protobuf "oneof") well. This is achieved by prefixing bytes before each "concrete type".
A concrete type is some non-interface value (generally a struct) which implements the interface to be (de)serialized. Not all structures need to be registered as concrete types -- only when they will be stored in interface type fields (or interface type slices) do they need to be registered.
Registering types ¶
All interfaces and the concrete types that implement them must be registered.
amino.RegisterInterface((*MyInterface1)(nil), nil) amino.RegisterInterface((*MyInterface2)(nil), nil) amino.RegisterConcrete(MyStruct1{}, "com.tendermint/MyStruct1", nil) amino.RegisterConcrete(MyStruct2{}, "com.tendermint/MyStruct2", nil) amino.RegisterConcrete(&MyStruct3{}, "anythingcangoinhereifitsunique", nil)
Notice that an interface is represented by a nil pointer.
Structures that must be deserialized as pointer values must be registered with a pointer value as well. It's OK to (de)serialize such structures in non-pointer (value) form, but when deserializing such structures into an interface field, they will always be deserialized as pointers.
How it works ¶
All registered concrete types are encoded with leading 4 bytes (called "prefix bytes"), even when it's not held in an interface field/element. In this way, Amino ensures that concrete types (almost) always have the same canonical representation. The first byte of the prefix bytes must not be a zero byte, so there are 2**(8*4)-2**(8*3) possible values.
When there are 4096 types registered at once, the probability of there being a conflict is ~ 0.2%. See https://instacalc.com/51189 for estimation. This is assuming that all registered concrete types have unique natural names (e.g. prefixed by a unique entity name such as "com.tendermint/", and not "mined/grinded" to produce a particular sequence of "prefix bytes").
TODO Update instacalc.com link with 255/256 since 0x00 is an escape.
Do not mine/grind to produce a particular sequence of prefix bytes, and avoid using dependencies that do so.
Since 4 bytes are not sufficient to ensure no conflicts, sometimes it is necessary to prepend more than the 4 prefix bytes for disambiguation. Like the prefix bytes, the disambiguation bytes are also computed from the registered name of the concrete type. There are 3 disambiguation bytes, and in binary form they always precede the prefix bytes. The first byte of the disambiguation bytes must not be a zero byte, so there are 2**(8*3)-2**(8*2) possible values.
// Sample Amino encoded binary bytes with 4 prefix bytes. > [0xBB 0x9C 0x83 0xDD] [...] // Sample Amino encoded binary bytes with 3 disambiguation bytes and 4 // prefix bytes. > 0x00 <0xA8 0xFC 0x54> [0xBB 0x9C 0x83 0xDD] [...]
The prefix bytes never start with a zero byte, so the disambiguation bytes are escaped with 0x00.
Notice that the 4 prefix bytes always immediately precede the binary encoding of the concrete type.
Computing prefix bytes ¶
To compute the disambiguation bytes, we take `hash := sha256(concreteTypeName)`, and drop the leading 0x00 bytes.
> hash := sha256("com.tendermint.consensus/MyConcreteName") > hex.EncodeBytes(hash) // 0x{00 00 A8 FC 54 00 00 00 BB 9C 83 DD ...} (example)
In the example above, hash has two leading 0x00 bytes, so we drop them.
> rest = dropLeadingZeroBytes(hash) // 0x{A8 FC 54 00 00 BB 9C 83 DD ...} > disamb = rest[0:3] > rest = dropLeadingZeroBytes(rest[3:]) > prefix = rest[0:4]
The first 3 bytes are called the "disambiguation bytes" (in angle brackets). The next 4 bytes are called the "prefix bytes" (in square brackets).
> <0xA8 0xFC 0x54> [0xBB 0x9C 9x83 9xDD]
Example ¶
package main import ( "fmt" amino "github.com/tendermint/go-amino" ) func main() { defer func() { if e := recover(); e != nil { fmt.Println("Recovered:", e) } }() type Message interface{} type bcMessage struct { Message string Height int } type bcResponse struct { Status int Message string } type bcStatus struct { Peers int } var cdc = amino.NewCodec() cdc.RegisterInterface((*Message)(nil), nil) cdc.RegisterConcrete(&bcMessage{}, "bcMessage", nil) cdc.RegisterConcrete(&bcResponse{}, "bcResponse", nil) cdc.RegisterConcrete(&bcStatus{}, "bcStatus", nil) var bm = &bcMessage{Message: "ABC", Height: 100} var msg = bm var bz []byte // the marshalled bytes. var err error bz, err = cdc.MarshalBinaryLengthPrefixed(msg) fmt.Printf("Encoded: %X (err: %v)\n", bz, err) var msg2 Message err = cdc.UnmarshalBinaryLengthPrefixed(bz, &msg2) fmt.Printf("Decoded: %v (err: %v)\n", msg2, err) var bm2 = msg2.(*bcMessage) fmt.Printf("Decoded successfully: %v\n", *bm == *bm2) }
Output: Encoded: 0B740613650A034142431064 (err: <nil>) Decoded: &{ABC 100} (err: <nil>) Decoded successfully: true
Index ¶
- Constants
- Variables
- func ByteSliceSize(bz []byte) int
- func DecodeBool(bz []byte) (b bool, n int, err error)
- func DecodeByte(bz []byte) (b byte, n int, err error)
- func DecodeByteSlice(bz []byte) (bz2 []byte, n int, err error)
- func DecodeDisambPrefixBytes(bz []byte) (db DisambBytes, hasDb bool, pb PrefixBytes, hasPb bool, n int, err error)
- func DecodeFloat32(bz []byte) (f float32, n int, err error)
- func DecodeFloat64(bz []byte) (f float64, n int, err error)
- func DecodeInt16(bz []byte) (i int16, n int, err error)
- func DecodeInt32(bz []byte) (i int32, n int, err error)
- func DecodeInt64(bz []byte) (i int64, n int, err error)
- func DecodeInt8(bz []byte) (i int8, n int, err error)
- func DecodeString(bz []byte) (s string, n int, err error)
- func DecodeTime(bz []byte) (t time.Time, n int, err error)
- func DecodeUint16(bz []byte) (u uint16, n int, err error)
- func DecodeUint32(bz []byte) (u uint32, n int, err error)
- func DecodeUint64(bz []byte) (u uint64, n int, err error)
- func DecodeUint8(bz []byte) (u uint8, n int, err error)
- func DecodeUvarint(bz []byte) (u uint64, n int, err error)
- func DecodeVarint(bz []byte) (i int64, n int, err error)
- func DeepCopy(o interface{}) (r interface{})
- func EncodeBool(w io.Writer, b bool) (err error)
- func EncodeByte(w io.Writer, b byte) (err error)
- func EncodeByteSlice(w io.Writer, bz []byte) (err error)
- func EncodeFloat32(w io.Writer, f float32) (err error)
- func EncodeFloat64(w io.Writer, f float64) (err error)
- func EncodeInt16(w io.Writer, i int16) (err error)
- func EncodeInt32(w io.Writer, i int32) (err error)
- func EncodeInt64(w io.Writer, i int64) (err error)
- func EncodeInt8(w io.Writer, i int8) (err error)
- func EncodeString(w io.Writer, s string) (err error)
- func EncodeTime(w io.Writer, t time.Time) (err error)
- func EncodeUint16(w io.Writer, u uint16) (err error)
- func EncodeUint32(w io.Writer, u uint32) (err error)
- func EncodeUint64(w io.Writer, u uint64) (err error)
- func EncodeUint8(w io.Writer, u uint8) (err error)
- func EncodeUvarint(w io.Writer, u uint64) (err error)
- func EncodeVarint(w io.Writer, i int64) (err error)
- func MarshalBinaryBare(o interface{}) ([]byte, error)
- func MarshalBinaryLengthPrefixed(o interface{}) ([]byte, error)
- func MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)
- func MarshalJSON(o interface{}) ([]byte, error)
- func MarshalJSONIndent(o interface{}, prefix, indent string) ([]byte, error)
- func MustMarshalBinaryBare(o interface{}) []byte
- func MustMarshalBinaryLengthPrefixed(o interface{}) []byte
- func MustUnmarshalBinaryBare(bz []byte, ptr interface{})
- func MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})
- func NameToDisfix(name string) (db DisambBytes, pb PrefixBytes)
- func UnmarshalBinaryBare(bz []byte, ptr interface{}) error
- func UnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{}) error
- func UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{}, maxSize int64) (n int64, err error)
- func UnmarshalJSON(bz []byte, ptr interface{}) error
- func UvarintSize(u uint64) int
- func VarintSize(i int64) int
- type Codec
- func (cdc *Codec) MarshalBinaryBare(o interface{}) ([]byte, error)
- func (cdc *Codec) MarshalBinaryLengthPrefixed(o interface{}) ([]byte, error)
- func (cdc *Codec) MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)
- func (cdc *Codec) MarshalJSON(o interface{}) ([]byte, error)
- func (cdc *Codec) MarshalJSONIndent(o interface{}, prefix, indent string) ([]byte, error)
- func (cdc *Codec) MustMarshalBinaryBare(o interface{}) []byte
- func (cdc *Codec) MustMarshalBinaryLengthPrefixed(o interface{}) []byte
- func (cdc *Codec) MustMarshalJSON(o interface{}) []byte
- func (cdc *Codec) MustUnmarshalBinaryBare(bz []byte, ptr interface{})
- func (cdc *Codec) MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})
- func (cdc *Codec) MustUnmarshalJSON(bz []byte, ptr interface{})
- func (cdc *Codec) PrintTypes(out io.Writer) error
- func (cdc *Codec) RegisterConcrete(o interface{}, name string, copts *ConcreteOptions)
- func (cdc *Codec) RegisterInterface(ptr interface{}, iopts *InterfaceOptions)
- func (cdc *Codec) Seal() *Codec
- func (cdc *Codec) UnmarshalBinaryBare(bz []byte, ptr interface{}) error
- func (cdc *Codec) UnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{}) error
- func (cdc *Codec) UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{}, maxSize int64) (n int64, err error)
- func (cdc *Codec) UnmarshalJSON(bz []byte, ptr interface{}) error
- type ConcreteInfo
- type ConcreteOptions
- type DisambBytes
- type DisfixBytes
- type FieldInfo
- type FieldOptions
- type InterfaceInfo
- type InterfaceOptions
- type InvalidTimeErr
- type PrefixBytes
- type StructInfo
- type Typ3
- type TypeInfo
Examples ¶
Constants ¶
const ( // Typ3 types Typ3Varint = Typ3(0) Typ38Byte = Typ3(1) Typ3ByteLength = Typ3(2) //Typ3_Struct = Typ3(3) //Typ3_StructTerm = Typ3(4) Typ3_4Byte = Typ3(5) )
const ( PrefixBytesLen = 4 DisambBytesLen = 3 DisfixBytesLen = PrefixBytesLen + DisambBytesLen )
Lengths
const Version = "0.15.0"
Version
Variables ¶
var ( // ErrNoPointer is thrown when you call a method that expects a pointer, e.g. Unmarshal ErrNoPointer = errors.New("expected a pointer") )
var (
ErrOverflowInt = errors.New("encoded integer value overflows int(32)")
)
Functions ¶
func ByteSliceSize ¶
func DecodeByteSlice ¶ added in v0.9.6
func DecodeDisambPrefixBytes ¶ added in v0.9.6
func DecodeDisambPrefixBytes(bz []byte) (db DisambBytes, hasDb bool, pb PrefixBytes, hasPb bool, n int, err error)
func DecodeFloat32 ¶ added in v0.9.6
NOTE: UNSAFE
func DecodeFloat64 ¶ added in v0.9.6
NOTE: UNSAFE
func DecodeTime ¶ added in v0.9.6
DecodeTime decodes seconds (int64) and nanoseconds (int32) since January 1, 1970 UTC, and returns the corresponding time. If nanoseconds is not in the range [0, 999999999], or if seconds is too large, the behavior is undefined. TODO return error if behavior is undefined.
func DeepCopy ¶ added in v0.11.1
func DeepCopy(o interface{}) (r interface{})
Deeply copies an object. If anything implements `.DeepCopy() <any>` along the way, the result of that function will be used. Otherwise, if it implements `.MarshalAmino() (<any>, error)` and `.UnmarshalAmino(<any>) error`, the pair will be used to copy. If .MarshalAmino() or .UnmarshalAmino() returns an error, this function will panic.
func EncodeFloat32 ¶ added in v0.9.6
NOTE: UNSAFE
func EncodeFloat64 ¶ added in v0.9.6
NOTE: UNSAFE
func EncodeTime ¶ added in v0.9.6
EncodeTime writes the number of seconds (int64) and nanoseconds (int32), with millisecond resolution since January 1, 1970 UTC to the Writer as an UInt64. Milliseconds are used to ease compatibility with Javascript, which does not support finer resolution.
func EncodeUvarint ¶ added in v0.9.6
EncodeUvarint is used to encode golang's int, int32, int64 by default. unless specified differently by the `binary:"fixed32"`, `binary:"fixed64"`, or `binary:"zigzag32"` `binary:"zigzag64"` tags. It matches protobufs varint encoding.
func MarshalBinaryBare ¶ added in v0.11.1
func MarshalBinaryLengthPrefixed ¶ added in v0.13.0
func MarshalBinaryLengthPrefixedWriter ¶ added in v0.13.0
func MarshalJSON ¶ added in v0.7.3
func MarshalJSONIndent ¶ added in v0.11.1
func MustMarshalBinaryBare ¶ added in v0.11.1
func MustMarshalBinaryBare(o interface{}) []byte
func MustMarshalBinaryLengthPrefixed ¶ added in v0.13.0
func MustMarshalBinaryLengthPrefixed(o interface{}) []byte
func MustUnmarshalBinaryBare ¶ added in v0.11.1
func MustUnmarshalBinaryBare(bz []byte, ptr interface{})
func MustUnmarshalBinaryLengthPrefixed ¶ added in v0.13.0
func MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})
func NameToDisfix ¶ added in v0.9.7
func NameToDisfix(name string) (db DisambBytes, pb PrefixBytes)
Return the DisambBytes and the PrefixBytes for a given name.
func UnmarshalBinaryBare ¶ added in v0.11.1
func UnmarshalBinaryLengthPrefixed ¶ added in v0.13.0
func UnmarshalBinaryLengthPrefixedReader ¶ added in v0.13.0
func UnmarshalJSON ¶ added in v0.7.3
func UvarintSize ¶
func VarintSize ¶ added in v0.9.6
Types ¶
type Codec ¶
type Codec struct {
// contains filtered or unexported fields
}
func (*Codec) MarshalBinaryBare ¶ added in v0.9.6
MarshalBinaryBare encodes the object o according to the Amino spec. MarshalBinaryBare doesn't prefix the byte-length of the encoding, so the caller must handle framing.
func (*Codec) MarshalBinaryLengthPrefixed ¶ added in v0.13.0
MarshalBinaryLengthPrefixed encodes the object o according to the Amino spec, but prefixed by a uvarint encoding of the object to encode. Use MarshalBinaryBare if you don't want byte-length prefixing.
For consistency, MarshalBinaryLengthPrefixed will first dereference pointers before encoding. MarshalBinaryLengthPrefixed will panic if o is a nil-pointer, or if o is invalid.
func (*Codec) MarshalBinaryLengthPrefixedWriter ¶ added in v0.13.0
func (cdc *Codec) MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)
MarshalBinaryLengthPrefixedWriter writes the bytes as would be returned from MarshalBinaryLengthPrefixed to the writer w.
func (*Codec) MarshalJSON ¶ added in v0.9.6
func (*Codec) MarshalJSONIndent ¶ added in v0.9.7
MarshalJSONIndent calls json.Indent on the output of cdc.MarshalJSON using the given prefix and indent string.
func (*Codec) MustMarshalBinaryBare ¶ added in v0.9.6
Panics if error.
func (*Codec) MustMarshalBinaryLengthPrefixed ¶ added in v0.13.0
Panics if error.
func (*Codec) MustMarshalJSON ¶ added in v0.13.0
MustMarshalJSON panics if an error occurs. Besides that behaves exactly like MarshalJSON.
func (*Codec) MustUnmarshalBinaryBare ¶ added in v0.9.7
Panics if error.
func (*Codec) MustUnmarshalBinaryLengthPrefixed ¶ added in v0.13.0
Panics if error.
func (*Codec) MustUnmarshalJSON ¶ added in v0.13.0
MustUnmarshalJSON panics if an error occurs. Besides that behaves exactly like UnmarshalJSON.
func (*Codec) PrintTypes ¶ added in v0.11.1
PrintTypes writes all registered types in a markdown-style table. The table's header is:
| Type | Name | Prefix | Notes |
Where Type is the golang type name and Name is the name the type was registered with.
func (*Codec) RegisterConcrete ¶ added in v0.9.6
func (cdc *Codec) RegisterConcrete(o interface{}, name string, copts *ConcreteOptions)
This function should be used to register concrete types that will appear in interface fields/elements to be encoded/decoded by go-amino. Usage: `amino.RegisterConcrete(MyStruct1{}, "com.tendermint/MyStruct1", nil)`
func (*Codec) RegisterInterface ¶ added in v0.9.6
func (cdc *Codec) RegisterInterface(ptr interface{}, iopts *InterfaceOptions)
This function should be used to register all interfaces that will be encoded/decoded by go-amino. Usage: `amino.RegisterInterface((*MyInterface1)(nil), nil)`
func (*Codec) UnmarshalBinaryBare ¶ added in v0.9.6
UnmarshalBinaryBare will panic if ptr is a nil-pointer.
func (*Codec) UnmarshalBinaryLengthPrefixed ¶ added in v0.13.0
Like UnmarshalBinaryBare, but will first decode the byte-length prefix. UnmarshalBinaryLengthPrefixed will panic if ptr is a nil-pointer. Returns an error if not all of bz is consumed.
func (*Codec) UnmarshalBinaryLengthPrefixedReader ¶ added in v0.13.0
func (cdc *Codec) UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{}, maxSize int64) (n int64, err error)
Like UnmarshalBinaryBare, but will first read the byte-length prefix. UnmarshalBinaryLengthPrefixedReader will panic if ptr is a nil-pointer. If maxSize is 0, there is no limit (not recommended).
func (*Codec) UnmarshalJSON ¶ added in v0.9.6
type ConcreteInfo ¶ added in v0.9.6
type ConcreteInfo struct { // These fields are only set when registered (as implementing an interface). Registered bool // Registered with RegisterConcrete(). PointerPreferred bool // Deserialize to pointer type if possible. // NilPreferred bool // Deserialize to nil for empty structs if PointerPreferred. Name string // Registered name. Disamb DisambBytes // Disambiguation bytes derived from name. Prefix PrefixBytes // Prefix bytes derived from name. ConcreteOptions // Registration options. // These fields get set for all concrete types, // even those not manually registered (e.g. are never interface values). IsAminoMarshaler bool // Implements MarshalAmino() (<ReprObject>, error). AminoMarshalReprType reflect.Type // <ReprType> IsAminoUnmarshaler bool // Implements UnmarshalAmino(<ReprObject>) (error). AminoUnmarshalReprType reflect.Type // <ReprType> }
func (ConcreteInfo) GetDisfix ¶ added in v0.9.6
func (cinfo ConcreteInfo) GetDisfix() DisfixBytes
type ConcreteOptions ¶ added in v0.9.6
type ConcreteOptions struct { }
type DisambBytes ¶ added in v0.9.6
type DisambBytes [DisambBytesLen]byte
Prefix types
func (DisambBytes) Bytes ¶ added in v0.9.6
func (db DisambBytes) Bytes() []byte
func (DisambBytes) EqualBytes ¶ added in v0.9.6
func (db DisambBytes) EqualBytes(bz []byte) bool
type DisfixBytes ¶ added in v0.9.6
type DisfixBytes [DisfixBytesLen]byte // Disamb+Prefix
Prefix types
func (DisfixBytes) Bytes ¶ added in v0.9.6
func (df DisfixBytes) Bytes() []byte
func (DisfixBytes) EqualBytes ¶ added in v0.9.6
func (df DisfixBytes) EqualBytes(bz []byte) bool
type FieldInfo ¶ added in v0.9.6
type FieldInfo struct { Name string // Struct field name Type reflect.Type // Struct field type Index int // Struct field index ZeroValue reflect.Value // Could be nil pointer unlike TypeInfo.ZeroValue. UnpackedList bool // True iff this field should be encoded as an unpacked list. FieldOptions // Encoding options }
type FieldOptions ¶ added in v0.9.6
type FieldOptions struct { JSONName string // (JSON) field name JSONOmitEmpty bool // (JSON) omitempty BinFixed64 bool // (Binary) Encode as fixed64 BinFixed32 bool // (Binary) Encode as fixed32 BinFieldNum uint32 // (Binary) max 1<<29-1 Unsafe bool // e.g. if this field is a float. WriteEmpty bool // write empty structs and lists (default false except for pointers) EmptyElements bool // Slice and Array elements are never nil, decode 0x00 as empty struct. }
type InterfaceInfo ¶ added in v0.9.6
type InterfaceInfo struct { Priority []DisfixBytes // Disfix priority. Implementers map[PrefixBytes][]*TypeInfo // Mutated over time. InterfaceOptions }
type InterfaceOptions ¶ added in v0.9.6
type InvalidTimeErr ¶ added in v0.13.0
type InvalidTimeErr string
func (InvalidTimeErr) Error ¶ added in v0.13.0
func (e InvalidTimeErr) Error() string
type PrefixBytes ¶ added in v0.9.6
type PrefixBytes [PrefixBytesLen]byte
Prefix types
func NewPrefixBytes ¶ added in v0.9.6
func NewPrefixBytes(prefixBytes []byte) PrefixBytes
Copy into PrefixBytes
func (PrefixBytes) Bytes ¶ added in v0.9.6
func (pb PrefixBytes) Bytes() []byte
func (PrefixBytes) EqualBytes ¶ added in v0.9.6
func (pb PrefixBytes) EqualBytes(bz []byte) bool
type StructInfo ¶ added in v0.9.6
type StructInfo struct {
Fields []FieldInfo // If a struct.
}
type TypeInfo ¶
type TypeInfo struct { Type reflect.Type // Interface type. PtrToType reflect.Type ZeroValue reflect.Value ZeroProto interface{} InterfaceInfo ConcreteInfo StructInfo }