typegen

package module
v0.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 30, 2022 License: MIT Imports: 20 Imported by: 7

README

cbor-gen

DAOT Labs' fork of whyrusleeping/cbor-gen.

Some basic utilities to generate fast path cbor codecs for your types.

Usage

See testgen/main.go.

New features of this fork

Adds support for types: uint16, uint32, int8, int16, int32
Sort map keys according to RFC7049 & DAG-CBOR strict ordering

This adds proper RFC7049 map key sorting, to both bare maps and structs in map representation. The other DAG-CBOR codec implementations strictly sort by length first, then bytes, as per https://github.com/ipld/specs/blob/master/block-layer/codecs/dag-cbor.md#strictness. The original cbor-gen implementation will render CBOR that won't produce stable CIDs with a round-trip using another DAG-CBOR encoder.

For more details, see: https://github.com/whyrusleeping/cbor-gen/pull/42

Embedded struct flattening

This fork additionally support an option flattenEmbeddedStruct in WriteTupleEncodersToFile and WriteMapEncodersToFile functions to flatten embedded structs in serialization.

For example, we have following structs from testing/flatten_tuple/types.go:

type EmbeddingStructOne struct {
	SimpleTypeOne
	*SimpleTypeTwo
	Foo          string
	Stuff        *SimpleTypeTwo
	Others       []uint64
	SignedOthers []int64
}

type EmbeddingStructTwo struct {
	SimpleTypeOne
	*EmbeddingStructOne
	Foo          string
	Value        uint64
	Stuff        *SimpleTypeTwo
	Others       []uint64
	SignedOthers []int64
	Test         [][]byte
	Dog          string
	Numbers      []NamedNumber
}

type EmbeddingStructThree struct {
	*EmbeddingStructTwo
	Foo          string
	Value        uint64
	Binary       []byte
	Stuff        *SimpleTypeTwo
	Others       []uint64
	SignedOthers []int64
	Test         [][]byte
	Dog          string
	Numbers      []NamedNumber
	Pizza        *uint64
	PointyPizza  *NamedNumber
	Arrrrrghay   [testing.Thingc]SimpleTypeOne
}

type FlatStruct struct {
	Signed  int64
	Foo     string
	Binary  []byte
	NString NamedString
	Value   uint64
}

type EmbeddedStruct struct {
	Value  uint64
	Foo    string
	Binary []byte
}

type StructValueFieldStruct struct {
	Signed  int64
	EmbeddedStruct EmbeddedStruct
	Binary  []byte
	NString NamedString
	Value   uint64
}

type StructPointerFieldStruct struct {
	Signed  int64
	EmbeddedStruct *EmbeddedStruct
	Binary  []byte
	NString NamedString
	Value   uint64
}

type EmbedByValueStruct struct {
	Signed  int64
	EmbeddedStruct
	Binary  []byte
	NString NamedString
	Value   uint64
}

type EmbedByPointerStruct struct {
	Signed  int64
	*EmbeddedStruct
	Binary  []byte
	NString NamedString
	Value   uint64
}

With the original cbor-gen, StructValueFieldStruct, StructPointerFieldStruct, EmbedByValueStruct and EmbedByPointerStruct will all be serialized into the same byte sequence, which is different from that of FlatStruct.

With this fork and flattenEmbeddedStruct option set to true, StructValueFieldStruct and StructPointerFieldStruct will all be serialized into the same byte sequence as that of FlatStruct, which is different from that of StructValueFieldStruct and StructPointerFieldStruct.

The same flattening applies to recursively embedded structs with arbitrary levels of embedding.

Note: For flattenEmbeddedStruct option to work, all structs recursively embedded by pointer (any levels) in the root struct need to generate their InitNilEmbeddedStruct methods with cbor-gen and flattenEmbeddedStruct option also set to true, although their generated MarshalCBOR and UnmarshalCBOR methods won't be used when marshalling and unmarshalling the root struct. Alternatively, you can write the InitNilEmbeddedStruct methods by hand, see Inniter.InitNilEmbeddedStruct in utils.go for description and testing/flatten_tuple/cbor_gen.go for examples.

type Initter interface {
	// InitNilEmbeddedStruct prepare the struct for marshalling & unmarshalling by
	// recursively initialize the structs embedded by pointer to their zero values.
	InitNilEmbeddedStruct()
}
Specifying struct field order

This fork additionally support an option fieldOrder in WriteTupleEncodersToFile to specify the orders of structs' fields. If specified (not nil), fieldOrder is a string slice of struct field names in the order they will be serialized, the remaining field names not found in the slice will be serialized after those specified in the original order of appearance in the code.

For example, we have the following two structs from testing/flatten_tuple/types.go:

type FlatStruct struct {
	Signed  int64
	Foo     string
	Binary  []byte
	NString NamedString
	Value   uint64
}

type ReorderedFlatStruct struct {
	Foo     string
	Value   uint64
	Binary  []byte
	Signed  int64
	NString NamedString
}

They have the same fields with different orders. If we generate the MarshalCBOR and UnmarshalCBOR with the following options, they will be serialized into and deserialized from the same byte sequence:

cbg.WriteTupleEncodersToFile("testing/flatten_tuple/cbor_gen.go",
    "flatten_tuple", true, nil,
    flatten_tuple.FlatStruct{},
)

cbg.WriteTupleEncodersToFile("testing/flatten_tuple/cbor_gen_reordered.go",
    "flatten_tuple", true, []string{"Signed", "Foo", "Binary", "NString", "Value"},
    flatten_tuple.ReorderedFlatStruct{},
)

License

MIT

Documentation

Index

Constants

View Source 
const (
	MajUnsignedInt = 0
	MajNegativeInt = 1
	MajByteString  = 2
	MajTextString  = 3
	MajArray       = 4
	MajMap         = 5
	MajTag         = 6
	MajOther       = 7
)
View Source 
const ByteArrayMaxLen = 2 << 20
View Source 
const MaxLength = 8192

Variables

View Source 
var (
	CborBoolFalse = []byte{0xf4}
	CborBoolTrue  = []byte{0xf5}
	CborNull      = []byte{0xf6}
)

Functions

func CborEncodeMajorType 

func CborEncodeMajorType(t byte, l uint64) []byte

func CborReadHeader 

func CborReadHeader(br io.Reader) (byte, uint64, int, error)

func CborReadHeaderBuf 

func CborReadHeaderBuf(br io.Reader, scratch []byte) (byte, uint64, int, error)

same as the above, just tries to allocate less by using a passed in scratch buffer

func CborWriteHeader 

func CborWriteHeader(w io.Writer, t byte, l uint64) (n int, err error)

func EncodeBool 

func EncodeBool(b bool) []byte

func GenMapEncodersForType 

func GenMapEncodersForType(gti *GenTypeInfo, flattenEmbeddedStruct bool,
	embeddedByPointerStructs *[]string, w io.Writer) error

Generates 'tuple representation' cbor encoders for the given type

func GenTupleEncodersForType 

func GenTupleEncodersForType(gti *GenTypeInfo, flattenEmbeddedStruct bool,
	embeddedByPointerStructs *[]string, w io.Writer) error

Generates 'tuple representation' cbor encoders for the given type

func MapKeySort_RFC7049 

func MapKeySort_RFC7049(keys []string)

func PrintHeaderAndUtilityMethods 

func PrintHeaderAndUtilityMethods(w io.Writer, pkg string, typeInfos []*GenTypeInfo) error

func ReadByteArray 

func ReadByteArray(br io.Reader, maxlen uint64) ([]byte, int, error)

func ReadCid 

func ReadCid(br io.Reader) (cid.Cid, int, error)

func ReadString 

func ReadString(r io.Reader) (string, int, error)

func ReadStringBuf 

func ReadStringBuf(r io.Reader, scratch []byte) (string, int, error)

func ReadTaggedByteArray 

func ReadTaggedByteArray(br io.Reader, exptag uint64, maxlen uint64) ([]byte, int, error)
func ScanForLinks(br io.Reader, cb func(cid.Cid)) (int, error)

func ValidateCBOR 

func ValidateCBOR(b []byte) error

ValidateCBOR validates that a byte array is a single valid CBOR object.

func WriteBool 

func WriteBool(w io.Writer, b bool) (n int, err error)

func WriteCid 

func WriteCid(w io.Writer, c cid.Cid) (n int, err error)

func WriteCidBuf 

func WriteCidBuf(buf []byte, w io.Writer, c cid.Cid) (n int, er error)

func WriteMajorTypeHeader 

func WriteMajorTypeHeader(w io.Writer, t byte, l uint64) (n int, err error)

TODO: No matter what I do, this function *still* allocates. Its super frustrating. See issue: https://github.com/golang/go/issues/33160

func WriteMajorTypeHeaderBuf 

func WriteMajorTypeHeaderBuf(buf []byte, w io.Writer, t byte, l uint64) (n int, err error)

Same as the above, but uses a passed in buffer to avoid allocations

func WriteMapEncodersToFile 

func WriteMapEncodersToFile(fname, pkg string, flattenEmbeddedStruct bool,
	types ...interface{}) error

WriteMapFileEncodersToFile generates map backed MarshalCBOR and UnmarshalCBOR implementations for the given types in the specified file, with the specified package name.

The MarshalCBOR and UnmarshalCBOR implementations will marshal/unmarshal each type's fields as a map of field names to field values.

func WriteTupleEncodersToFile 

func WriteTupleEncodersToFile(fname, pkg string, flattenEmbeddedStruct bool,
	fieldOrder []string, types ...interface{}) error

WriteTupleFileEncodersToFile generates array backed MarshalCBOR and UnmarshalCBOR implementations for the given types in the specified file, with the specified package name.

The MarshalCBOR and UnmarshalCBOR implementations will marshal/unmarshal each type's fields as a fixed-length CBOR array of field values.

Types

type BytePeeker 

type BytePeeker interface {
	io.Reader
	io.ByteScanner
}

BytePeeker combines the Reader and ByteScanner interfaces.

func GetPeeker 

func GetPeeker(r io.Reader) BytePeeker

type CBORMarshaler 

type CBORMarshaler interface {
	// MarshalCBOR marshals the Go type into CBOR bytes and writes them into io.Writer,
	// it returns the length of the marshaled bytes written and the error if occurred.
	MarshalCBOR(io.Writer) (int, error)
}

type CBORUnmarshaler 

type CBORUnmarshaler interface {
	// UnmarshalCBOR reads CBOR bytes from io.Reader and unmarshals them into the Go type,
	// it returns the length of the unmarshaled bytes and the error if occurred.
	UnmarshalCBOR(io.Reader) (int, error)
}

type CborBool 

type CborBool bool

func (CborBool) MarshalCBOR 

func (cb CborBool) MarshalCBOR(w io.Writer) (n int, err error)

func (*CborBool) UnmarshalCBOR 

func (cb *CborBool) UnmarshalCBOR(r io.Reader) (int, error)

type CborCid 

type CborCid cid.Cid

func (CborCid) MarshalCBOR 

func (c CborCid) MarshalCBOR(w io.Writer) (n int, err error)

func (*CborCid) UnmarshalCBOR 

func (c *CborCid) UnmarshalCBOR(r io.Reader) (n int, err error)

type CborInt 

type CborInt int64

func (CborInt) MarshalCBOR 

func (ci CborInt) MarshalCBOR(w io.Writer) (n int, err error)

func (*CborInt) UnmarshalCBOR 

func (ci *CborInt) UnmarshalCBOR(r io.Reader) (int, error)

type CborTime 

type CborTime time.Time

func (CborTime) MarshalCBOR 

func (ct CborTime) MarshalCBOR(w io.Writer) (n int, err error)

func (CborTime) MarshalJSON 

func (ct CborTime) MarshalJSON() ([]byte, error)

func (CborTime) Time 

func (ct CborTime) Time() time.Time

func (*CborTime) UnmarshalCBOR 

func (ct *CborTime) UnmarshalCBOR(r io.Reader) (int, error)

func (*CborTime) UnmarshalJSON 

func (ct *CborTime) UnmarshalJSON(b []byte) error

type Deferred 

type Deferred struct {
	Raw []byte
}

func (*Deferred) MarshalCBOR 

func (d *Deferred) MarshalCBOR(w io.Writer) (int, error)

func (*Deferred) UnmarshalCBOR 

func (d *Deferred) UnmarshalCBOR(br io.Reader) (int, error)

type Field 

type Field struct {
	Name    string
	MapKey  string
	Pointer bool
	Type    reflect.Type
	Pkg     string

	IterLabel string
}

func (Field) ElemName 

func (f Field) ElemName() string

func (Field) IsArray 

func (f Field) IsArray() bool

func (Field) Len 

func (f Field) Len() int

func (Field) TypeName 

func (f Field) TypeName() string

type GenTypeInfo 

type GenTypeInfo struct {
	Name   string
	Fields []Field
}

func ParseTypeInfo 

func ParseTypeInfo(i interface{}, flattenEmbeddedStruct bool) (
	gti *GenTypeInfo, embeddedByPointerStructs *[]string, err error)

func (*GenTypeInfo) Imports 

func (gti *GenTypeInfo) Imports() []Import

func (GenTypeInfo) MapHeader 

func (gti GenTypeInfo) MapHeader() []byte

func (GenTypeInfo) MapHeaderAsByteString 

func (gti GenTypeInfo) MapHeaderAsByteString() string

func (*GenTypeInfo) NeedsScratch 

func (gti *GenTypeInfo) NeedsScratch() bool

func (GenTypeInfo) TupleHeader 

func (gti GenTypeInfo) TupleHeader() []byte

func (GenTypeInfo) TupleHeaderAsByteString 

func (gti GenTypeInfo) TupleHeaderAsByteString() string

type Import 

type Import struct {
	Name, PkgPath string
}

func ImportsForType 

func ImportsForType(currPkg string, t reflect.Type) []Import

type Initter 

type Initter interface {
	// InitNilEmbeddedStruct prepare the struct for marshalling & unmarshalling by
	// recursively initialize the structs embedded by pointer to their zero values.
	InitNilEmbeddedStruct()
}

Source Files

View all Source files

Directories

Path Synopsis
flatten_map
flatten_tuple
noflatten_map
noflatten_tuple

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.