Documentation ¶
Overview ¶
Package rlp implements the RLP serialization format.
The purpose of RLP (Recursive Linear Prefix) is to encode arbitrarily nested arrays of binary data, and RLP is the main encoding method used to serialize objects in Ethereum. The only purpose of RLP is to encode structure; encoding specific atomic data types (eg. strings, ints, floats) is left up to higher-order protocols. In Ethereum integers must be represented in big endian binary form with no leading zeroes (thus making the integer value zero equivalent to the empty string).
RLP values are distinguished by a type tag. The type tag precedes the value in the input stream and defines the size and kind of the bytes that follow.
Encoding Rules ¶
Package rlp uses reflection and encodes RLP based on the Go type of the value.
If the type implements the Encoder interface, Encode calls EncodeRLP. It does not call EncodeRLP on nil pointer values.
To encode a pointer, the value being pointed to is encoded. A nil pointer to a struct type, slice or array always encodes as an empty RLP list unless the slice or array has element type byte. A nil pointer to any other value encodes as the empty string.
Struct values are encoded as an RLP list of all their encoded public fields. Recursive struct types are supported.
To encode slices and arrays, the elements are encoded as an RLP list of the value's elements. Note that arrays and slices with element type uint8 or byte are always encoded as an RLP string.
A Go string is encoded as an RLP string.
An unsigned integer value is encoded as an RLP string. Zero always encodes as an empty RLP string. big.Int values are treated as integers. Signed integers (int, int8, int16, ...) are not supported and will return an error when encoding.
Boolean values are encoded as the unsigned integers zero (false) and one (true).
An interface value encodes as the value contained in the interface.
Floating point numbers, maps, channels and functions are not supported.
Decoding Rules ¶
Decoding uses the following type-dependent rules:
If the type implements the Decoder interface, DecodeRLP is called.
To decode into a pointer, the value will be decoded as the element type of the pointer. If the pointer is nil, a new value of the pointer's element type is allocated. If the pointer is non-nil, the existing value will be reused. Note that package rlp never leaves a pointer-type struct field as nil unless one of the "nil" struct tags is present.
To decode into a struct, decoding expects the input to be an RLP list. The decoded elements of the list are assigned to each public field in the order given by the struct's definition. The input list must contain an element for each decoded field. Decoding returns an error if there are too few or too many elements for the struct.
To decode into a slice, the input must be a list and the resulting slice will contain the input elements in order. For byte slices, the input must be an RLP string. Array types decode similarly, with the additional restriction that the number of input elements (or bytes) must match the array's defined length.
To decode into a Go string, the input must be an RLP string. The input bytes are taken as-is and will not necessarily be valid UTF-8.
To decode into an unsigned integer type, the input must also be an RLP string. The bytes are interpreted as a big endian representation of the integer. If the RLP string is larger than the bit size of the type, decoding will return an error. Decode also supports *big.Int. There is no size limit for big integers.
To decode into a boolean, the input must contain an unsigned integer of value zero (false) or one (true).
To decode into an interface value, one of these types is stored in the value:
[]interface{}, for RLP lists []byte, for RLP strings
Non-empty interface types are not supported when decoding. Signed integers, floating point numbers, maps, channels and functions cannot be decoded into.
Struct Tags ¶
As with other encoding packages, the "-" tag ignores fields.
type StructWithIgnoredField struct{ Ignored uint `rlp:"-"` Field uint }
Go struct values encode/decode as RLP lists. There are two ways of influencing the mapping of fields to list elements. The "tail" tag, which may only be used on the last exported struct field, allows slurping up any excess list elements into a slice.
type StructWithTail struct{ Field uint Tail []string `rlp:"tail"` }
The "optional" tag says that the field may be omitted if it is zero-valued. If this tag is used on a struct field, all subsequent public fields must also be declared optional.
When encoding a struct with optional fields, the output RLP list contains all values up to the last non-zero optional field.
When decoding into a struct, optional fields may be omitted from the end of the input list. For the example below, this means input lists of one, two, or three elements are accepted.
type StructWithOptionalFields struct{ Required uint Optional1 uint `rlp:"optional"` Optional2 uint `rlp:"optional"` }
The "nil", "nilList" and "nilString" tags apply to pointer-typed fields only, and change the decoding rules for the field type. For regular pointer fields without the "nil" tag, input values must always match the required input length exactly and the decoder does not produce nil values. When the "nil" tag is set, input values of size zero decode as a nil pointer. This is especially useful for recursive types.
type StructWithNilField struct { Field *[3]byte `rlp:"nil"` }
In the example above, Field allows two possible input sizes. For input 0xC180 (a list containing an empty string) Field is set to nil after decoding. For input 0xC483000000 (a list containing a 3-byte string), Field is set to a non-nil array pointer.
RLP supports two kinds of empty values: empty lists and empty strings. When using the "nil" tag, the kind of empty value allowed for a type is chosen automatically. A field whose Go type is a pointer to an unsigned integer, string, boolean or byte array/slice expects an empty RLP string. Any other pointer field type encodes/decodes as an empty RLP list.
The choice of null value can be made explicit with the "nilList" and "nilString" struct tags. Using these tags encodes/decodes a Go nil pointer value as the empty RLP value kind defined by the tag.
Index ¶
- Variables
- func AppendUint64(b []byte, i uint64) []byte
- func BytesSize(b []byte) uint64
- func CountValues(b []byte) (int, error)
- func Decode(r io.Reader, val interface{}) error
- func DecodeBytes(b []byte, val interface{}) error
- func Encode(w io.Writer, val interface{}) error
- func EncodeToBytes(val interface{}) ([]byte, error)
- func EncodeToReader(val interface{}) (size int, r io.Reader, err error)
- func IntSize(x uint64) int
- func ListSize(contentSize uint64) uint64
- func NewListIterator(data RawValue) (*listIterator, error)
- func SplitList(b []byte) (content, rest []byte, err error)
- func SplitString(b []byte) (content, rest []byte, err error)
- func SplitUint64(b []byte) (x uint64, rest []byte, err error)
- func StringSize(s string) uint64
- type ByteReader
- type Decoder
- type Encoder
- type EncoderBuffer
- func (w *EncoderBuffer) AppendToBytes(dst []byte) []byte
- func (w *EncoderBuffer) Flush() error
- func (w EncoderBuffer) List() int
- func (w EncoderBuffer) ListEnd(index int)
- func (w *EncoderBuffer) Reset(dst io.Writer)
- func (w *EncoderBuffer) ToBytes() []byte
- func (w EncoderBuffer) Write(b []byte) (int, error)
- func (w EncoderBuffer) WriteBigInt(i *big.Int)
- func (w EncoderBuffer) WriteBool(b bool)
- func (w EncoderBuffer) WriteBytes(b []byte)
- func (w EncoderBuffer) WriteString(s string)
- func (w EncoderBuffer) WriteUint256(i *uint256.Int)
- func (w EncoderBuffer) WriteUint64(i uint64)
- type Kind
- type RawValue
- type Stream
- func (s *Stream) BigInt() (*big.Int, error)
- func (s *Stream) Bool() (bool, error)
- func (s *Stream) Bytes() ([]byte, error)
- func (s *Stream) Decode(val interface{}) error
- func (s *Stream) Kind() (kind Kind, size uint64, err error)
- func (s *Stream) List() (size uint64, err error)
- func (s *Stream) ListEnd() error
- func (s *Stream) MoreDataInList() bool
- func (s *Stream) Raw() ([]byte, error)
- func (s *Stream) ReadBytes(b []byte) error
- func (s *Stream) ReadUint256(dst *uint256.Int) error
- func (s *Stream) Reset(r io.Reader, inputLimit uint64)
- func (s *Stream) Uint() (uint64, error)deprecated
- func (s *Stream) Uint16() (uint16, error)
- func (s *Stream) Uint32() (uint32, error)
- func (s *Stream) Uint64() (uint64, error)
- func (s *Stream) Uint8() (uint8, error)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrExpectedString = errors.New("rlp: expected String or Byte") ErrExpectedList = errors.New("rlp: expected List") ErrCanonInt = errors.New("rlp: non-canonical integer format") ErrCanonSize = errors.New("rlp: non-canonical size information") ErrElemTooLarge = errors.New("rlp: element is larger than containing list") ErrValueTooLarge = errors.New("rlp: value size exceeds available input length") ErrMoreThanOneValue = errors.New("rlp: input contains more than one value") )
var ( // EmptyString is the encoding of an empty string. EmptyString = []byte{0x80} // EmptyList is the encoding of an empty list. EmptyList = []byte{0xC0} )
var EOL = errors.New("rlp: end of list")
EOL is returned when the end of the current list has been reached during streaming.
var ErrNegativeBigInt = errors.New("rlp: cannot encode negative big.Int")
Functions ¶
func AppendUint64 ¶
AppendUint64 appends the RLP encoding of i to b, and returns the resulting slice.
func CountValues ¶
CountValues counts the number of encoded values in b.
func Decode ¶
Decode parses RLP-encoded data from r and stores the result in the value pointed to by val. Please see package-level documentation for the decoding rules. Val must be a non-nil pointer.
If r does not implement ByteReader, Decode will do its own buffering.
Note that Decode does not set an input limit for all readers and may be vulnerable to panics cause by huge value sizes. If you need an input limit, use
NewStream(r, limit).Decode(val)
Example ¶
input, _ := hex.DecodeString("C90A1486666F6F626172") type example struct { A, B uint String string } var s example err := Decode(bytes.NewReader(input), &s) if err != nil { fmt.Printf("Error: %v\n", err) } else { fmt.Printf("Decoded value: %#v\n", s) }
Output: Decoded value: rlp.example{A:0xa, B:0x14, String:"foobar"}
Example (StructTagNil) ¶
// In this example, we'll use the "nil" struct tag to change // how a pointer-typed field is decoded. The input contains an RLP // list of one element, an empty string. input := []byte{0xC1, 0x80} // This type uses the normal rules. // The empty input string is decoded as a pointer to an empty Go string. var normalRules struct { String *string } Decode(bytes.NewReader(input), &normalRules) fmt.Printf("normal: String = %q\n", *normalRules.String) // This type uses the struct tag. // The empty input string is decoded as a nil pointer. var withEmptyOK struct { String *string `rlp:"nil"` } Decode(bytes.NewReader(input), &withEmptyOK) fmt.Printf("with nil tag: String = %v\n", withEmptyOK.String)
Output: normal: String = "" with nil tag: String = <nil>
Example (StructTagTail) ¶
package main import ( "bytes" "fmt" ) type structWithTail struct { A, B uint C []uint `rlp:"tail"` } func main() { // In this example, the "tail" struct tag is used to decode lists of // differing length into a struct. var val structWithTail err := Decode(bytes.NewReader([]byte{0xC4, 0x01, 0x02, 0x03, 0x04}), &val) fmt.Printf("with 4 elements: err=%v val=%v\n", err, val) err = Decode(bytes.NewReader([]byte{0xC6, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06}), &val) fmt.Printf("with 6 elements: err=%v val=%v\n", err, val) // Note that at least two list elements must be present to // fill fields A and B: err = Decode(bytes.NewReader([]byte{0xC1, 0x01}), &val) fmt.Printf("with 1 element: err=%q\n", err) }
Output: with 4 elements: err=<nil> val={1 2 [3 4]} with 6 elements: err=<nil> val={1 2 [3 4 5 6]} with 1 element: err="rlp: too few elements for rlp.structWithTail"
func DecodeBytes ¶
DecodeBytes parses RLP data from b into val. Please see package-level documentation for the decoding rules. The input must contain exactly one value and no trailing data.
func Encode ¶
Encode writes the RLP encoding of val to w. Note that Encode may perform many small writes in some cases. Consider making w buffered.
Please see package-level documentation of encoding rules.
func EncodeToBytes ¶
EncodeToBytes returns the RLP encoding of val. Please see package-level documentation for the encoding rules.
func EncodeToReader ¶
EncodeToReader returns a reader from which the RLP encoding of val can be read. The returned size is the total size of the encoded data.
Please see the documentation of Encode for the encoding rules.
func IntSize ¶
IntSize returns the encoded size of the integer x. Note: The return type of this function is 'int' for backwards-compatibility reasons. The result is always positive.
func NewListIterator ¶
NewListIterator creates an iterator for the (list) represented by data TODO: Consider removing this implementation, as it is no longer used.
func SplitList ¶
SplitList splits b into the content of a list and any remaining bytes after the list.
func SplitString ¶
SplitString splits b into the content of an RLP string and any remaining bytes after the string.
func SplitUint64 ¶
SplitUint64 decodes an integer at the beginning of b. It also returns the remaining data after the integer in 'rest'.
Types ¶
type ByteReader ¶
type ByteReader interface { io.Reader io.ByteReader }
ByteReader must be implemented by any input reader for a Stream. It is implemented by e.g. bufio.Reader and bytes.Reader.
type Decoder ¶
Decoder is implemented by types that require custom RLP decoding rules or need to decode into private fields.
The DecodeRLP method should read one value from the given Stream. It is not forbidden to read less or more, but it might be confusing.
type Encoder ¶
type Encoder interface { // EncodeRLP should write the RLP encoding of its receiver to w. // If the implementation is a pointer method, it may also be // called for nil pointers. // // Implementations should generate valid RLP. The data written is // not verified at the moment, but a future version might. It is // recommended to write only a single value but writing multiple // values or no value at all is also permitted. EncodeRLP(io.Writer) error }
Encoder is implemented by types that require custom encoding rules or want to encode private fields.
Example ¶
package main import ( "fmt" "io" "github.com/yanikitat/op-geth/rlp" ) type MyCoolType struct { Name string a, b uint } // EncodeRLP writes x as RLP list [a, b] that omits the Name field. func (x *MyCoolType) EncodeRLP(w io.Writer) (err error) { return rlp.Encode(w, []uint{x.a, x.b}) } func main() { var t *MyCoolType // t is nil pointer to MyCoolType bytes, _ := rlp.EncodeToBytes(t) fmt.Printf("%v → %X\n", t, bytes) t = &MyCoolType{Name: "foobar", a: 5, b: 6} bytes, _ = rlp.EncodeToBytes(t) fmt.Printf("%v → %X\n", t, bytes) }
Output: <nil> → C0 &{foobar 5 6} → C20506
type EncoderBuffer ¶
type EncoderBuffer struct {
// contains filtered or unexported fields
}
EncoderBuffer is a buffer for incremental encoding.
The zero value is NOT ready for use. To get a usable buffer, create it using NewEncoderBuffer or call Reset.
Example ¶
package main import ( "bytes" "fmt" "github.com/yanikitat/op-geth/rlp" ) func main() { var w bytes.Buffer // Encode [4, [5, 6]] to w. buf := rlp.NewEncoderBuffer(&w) l1 := buf.List() buf.WriteUint64(4) l2 := buf.List() buf.WriteUint64(5) buf.WriteUint64(6) buf.ListEnd(l2) buf.ListEnd(l1) if err := buf.Flush(); err != nil { panic(err) } fmt.Printf("%X\n", w.Bytes()) }
Output: C404C20506
func NewEncoderBuffer ¶
func NewEncoderBuffer(dst io.Writer) EncoderBuffer
NewEncoderBuffer creates an encoder buffer.
func (*EncoderBuffer) AppendToBytes ¶
func (w *EncoderBuffer) AppendToBytes(dst []byte) []byte
AppendToBytes appends the encoded bytes to dst.
func (*EncoderBuffer) Flush ¶
func (w *EncoderBuffer) Flush() error
Flush writes encoded RLP data to the output writer. This can only be called once. If you want to re-use the buffer after Flush, you must call Reset.
func (EncoderBuffer) List ¶
func (w EncoderBuffer) List() int
List starts a list. It returns an internal index. Call EndList with this index after encoding the content to finish the list.
func (EncoderBuffer) ListEnd ¶
func (w EncoderBuffer) ListEnd(index int)
ListEnd finishes the given list.
func (*EncoderBuffer) Reset ¶
func (w *EncoderBuffer) Reset(dst io.Writer)
Reset truncates the buffer and sets the output destination.
func (*EncoderBuffer) ToBytes ¶
func (w *EncoderBuffer) ToBytes() []byte
ToBytes returns the encoded bytes.
func (EncoderBuffer) Write ¶
func (w EncoderBuffer) Write(b []byte) (int, error)
Write appends b directly to the encoder output.
func (EncoderBuffer) WriteBigInt ¶
func (w EncoderBuffer) WriteBigInt(i *big.Int)
WriteBigInt encodes a big.Int as an RLP string. Note: Unlike with Encode, the sign of i is ignored.
func (EncoderBuffer) WriteBool ¶
func (w EncoderBuffer) WriteBool(b bool)
WriteBool writes b as the integer 0 (false) or 1 (true).
func (EncoderBuffer) WriteBytes ¶
func (w EncoderBuffer) WriteBytes(b []byte)
WriteBytes encodes b as an RLP string.
func (EncoderBuffer) WriteString ¶
func (w EncoderBuffer) WriteString(s string)
WriteString encodes s as an RLP string.
func (EncoderBuffer) WriteUint256 ¶
func (w EncoderBuffer) WriteUint256(i *uint256.Int)
WriteUint256 encodes uint256.Int as an RLP string.
func (EncoderBuffer) WriteUint64 ¶
func (w EncoderBuffer) WriteUint64(i uint64)
WriteUint64 encodes an unsigned integer.
type Kind ¶
type Kind int8
Kind represents the kind of value contained in an RLP stream.
type RawValue ¶
type RawValue []byte
RawValue represents an encoded RLP value and can be used to delay RLP decoding or to precompute an encoding. Note that the decoder does not verify whether the content of RawValues is valid RLP.
type Stream ¶
type Stream struct {
// contains filtered or unexported fields
}
Stream can be used for piecemeal decoding of an input stream. This is useful if the input is very large or if the decoding rules for a type depend on the input structure. Stream does not keep an internal buffer. After decoding a value, the input reader will be positioned just before the type information for the next value.
When decoding a list and the input position reaches the declared length of the list, all operations will return error EOL. The end of the list must be acknowledged using ListEnd to continue reading the enclosing list.
Stream is not safe for concurrent use.
Example ¶
input, _ := hex.DecodeString("C90A1486666F6F626172") s := NewStream(bytes.NewReader(input), 0) // Check what kind of value lies ahead kind, size, _ := s.Kind() fmt.Printf("Kind: %v size:%d\n", kind, size) // Enter the list if _, err := s.List(); err != nil { fmt.Printf("List error: %v\n", err) return } // Decode elements fmt.Println(s.Uint()) fmt.Println(s.Uint()) fmt.Println(s.Bytes()) // Acknowledge end of list if err := s.ListEnd(); err != nil { fmt.Printf("ListEnd error: %v\n", err) }
Output: Kind: List size:9 10 <nil> 20 <nil> [102 111 111 98 97 114] <nil>
func NewListStream ¶
NewListStream creates a new stream that pretends to be positioned at an encoded list of the given length.
func NewStream ¶
NewStream creates a new decoding stream reading from r.
If r implements the ByteReader interface, Stream will not introduce any buffering.
For non-toplevel values, Stream returns ErrElemTooLarge for values that do not fit into the enclosing list.
Stream supports an optional input limit. If a limit is set, the size of any toplevel value will be checked against the remaining input length. Stream operations that encounter a value exceeding the remaining input length will return ErrValueTooLarge. The limit can be set by passing a non-zero value for inputLimit.
If r is a bytes.Reader or strings.Reader, the input limit is set to the length of r's underlying data unless an explicit limit is provided.
func (*Stream) Bool ¶
Bool reads an RLP string of up to 1 byte and returns its contents as a boolean. If the input does not contain an RLP string, the returned error will be ErrExpectedString.
func (*Stream) Bytes ¶
Bytes reads an RLP string and returns its contents as a byte slice. If the input does not contain an RLP string, the returned error will be ErrExpectedString.
func (*Stream) Decode ¶
Decode decodes a value and stores the result in the value pointed to by val. Please see the documentation for the Decode function to learn about the decoding rules.
func (*Stream) Kind ¶
Kind returns the kind and size of the next value in the input stream.
The returned size is the number of bytes that make up the value. For kind == Byte, the size is zero because the value is contained in the type tag.
The first call to Kind will read size information from the input reader and leave it positioned at the start of the actual bytes of the value. Subsequent calls to Kind (until the value is decoded) will not advance the input reader and return cached information.
func (*Stream) List ¶
List starts decoding an RLP list. If the input does not contain a list, the returned error will be ErrExpectedList. When the list's end has been reached, any Stream operation will return EOL.
func (*Stream) ListEnd ¶
ListEnd returns to the enclosing list. The input reader must be positioned at the end of a list.
func (*Stream) MoreDataInList ¶
MoreDataInList reports whether the current list context contains more data to be read.
func (*Stream) ReadBytes ¶
ReadBytes decodes the next RLP value and stores the result in b. The value size must match len(b) exactly.
func (*Stream) ReadUint256 ¶
ReadUint256 decodes the next value as a uint256.
func (*Stream) Reset ¶
Reset discards any information about the current decoding context and starts reading from r. This method is meant to facilitate reuse of a preallocated Stream across many decoding operations.
If r does not also implement ByteReader, Stream will do its own buffering.