schema

package
v1.21.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 3, 2024 License: MIT Imports: 12 Imported by: 11

README

x/schema - Golang recursive schema

Library allows to write code that work with any type of schemas. Regardless if those are JSON, XML, YAML, or golang structs.

Most benefits

  • Union types can be deserialized into interface field

How to convert between json <-> go

data := `{"name": "John", "cars": [{"name":"Ford"}]}`
schema := schema.FromJSON(data)
nativego, err := schema.ToGo(schema)

expected := map[string]any{
    "name": "John",
    "cars": []any{
        map[string]any{
            "name": "Ford",
        },
    },
}
assert.Equal(t, expected, nativego)

How to convert schema into named golang struct?

This example shows how to convert only part of schema to golang struct. List of cars will have type Car when parent Person object will be map[string]any.

type Car struct {
    Name string
}
nativego := schema.MustToGo(schema, WithOnlyTheseRules(
	WhenPath([]string{"cars", "[*]"}, UseStruct(Car{}))))

expected := map[string]any{
    "name": "John",
    "cars": []any{
        Car{
            Name: "Ford",
        },
    },
}
assert.Equal(t, expected, nativego)

How to define custom serialization and deserilization?

Currently, ser-deser operations are available on maps. This is current design decision, it might change in the future.

type Car struct {
    Name string
}

// make sure to Car implements schema.Marshaler and schema.Unmarshaler
var (
	_ schema.Marshaler = (*Car)(nil)
    _ schema.Unmarshaler = (*Car)(nil)
)

func (c *Car) MarshalSchema() (schema.*Map, error) {
    return schema.MkMap(map[string]schema.Schema{
        "name": schema.MkString(c.Name),
    }), nil
}

func (c *Car) UnmarshalSchema(x schema.*Map) error {
    for _, field := range x.Field {
        switch key {
        case "name":
            c.Name = s.MustToString()
        default:
            return fmt.Errorf("unknown key %s", key)
        }
    }
	
    return nil
}
How to convert well defined types from external packages?
type Car struct {
    Name string
    LastDriven time.Time
}

// Register conversion from time.Time to schema.String
schema.RegisterWellDefinedTypesConversion[time.Time](
  func(x time.Time) Schema {
      return MkString(x.Format(time.RFC3339Nano))
  },
  func(x Schema) time.Time {
      if v, ok := x.(*String); ok {
          t, _ := time.Parse(time.RFC3339Nano, string(*v))
          return t
      }

      panic("invalid type")
  },
)

// Then you can translate schema between back and forth without worrying about time.Time
schema := FromGo(data)
result, err := ToGoG[ExternalPackageStruct](schema)
assert.NoError(t, err)
assert.Equal(t, data, result)

Roadmap

V0.1.0
  • Json <-> Schema <-> Go (with structs mapping)
  • Write test with wrong type conversions
  • Value are split into Number(Int, Float), String, Bool, and Null
  • Default schema registry + mkunion make union serialization/deserialization work transperently
  • Support pointers *string, etc.
  • Support DynamoDB (FromDynamoDB, ToDynamoDB)
  • Support for pointer to types like *string, *int, etc.
  • Support for relative paths like WhenPath([]string{"*", "ListOfCars", "Car"}, UseStruct(Car{})). Absolute paths are without * at the beginning.
V0.2.x
  • Support options for ToGo & FromGo like WithOnlyTheseRules, WithExtraRules, WithDefaultMapDef, etc. Gives better control on how schema is converted to golang. It's especially important from security reasons, whey you want to allow rules only whitelisted rules, for user generated json input.
  • Schema support interface for custom type-setters, that don't require reflection, and mkunion can leverage them. Use UseTypeDef eg. WhenPath([]string{}, UseTypeDef(&someTypeDef{})),
  • Support for how union names should be expressed in schema WithUnionFormatter(func(t reflect.Type) string)
V0.3.x
  • schema.Compare method to compare two schemas
V0.4.x
  • Support for Binary type
  • Add missing function for MkBinary, MkFloat, MkNone
V0.5.x
  • schema.UnwrapDynamoDB takes DynamoDB specific nesting and removes it.
  • Eliminate data races in *UnionVariants[A] -> MapDefFor & UseUnionFormatter data race
  • Introduce ToGoG[T any] function, that makes ToGo with type assertion and tries to convert to T
  • Rename schema.As to schema.AsDefault and make schema.As as variant that returns false, if type is not supported
V0.6.x
  • Support serialization of schema.Marchaler to schema.Unmarshaller, that can dramatically improve performance in some cases. Limitation of current implementation is that it works only on *Map, and don't allow custom ser-deser on other types. It's not hard decision. It's just that I don't have use case for other types yet.
V0.7.x
  • schema.Schema is now serializable and deserilizable
V0.8.x
  • schema use x/shape and native types to represent variants like Map and List and Bytes
  • schema.Schema is refactored to leverage simpler types thanks to x/shape library
  • schema.ToJSON and schema.FromJSON are removed and replaced by mkunion defaults
V0.11.x
  • schema becomes data
  • data.FromGo and data.ToGo works only on primitive values
  • data.FromStruct and data.ToStruct works only on structs and reflection
  • data.FromDynamoDB and data.ToDynamoDB refactored

Documentation

Overview

Code generated by mkunion. DO NOT EDIT.

Code generated by mkunion. DO NOT EDIT.

Code generated by mkunion. DO NOT EDIT.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func As added in v1.9.0 

func As[A int | int8 | int16 | int32 | int64 |
	uint | uint8 | uint16 | uint32 | uint64 |
	float32 | float64 |
	bool | string | []byte](x Schema) (A, bool)

func AsDefault added in v1.12.0 

func AsDefault[A int | int8 | int16 | int32 | int64 |
	uint | uint8 | uint16 | uint32 | uint64 |
	float32 | float64 |
	bool | string | []byte](x Schema, def A) A

func BinaryShape added in v1.21.0 

func BinaryShape() shape.Shape

func BinaryToJSON added in v1.20.0 

func BinaryToJSON(x *Binary) ([]byte, error)

func BoolShape added in v1.21.0 

func BoolShape() shape.Shape

func BoolToJSON added in v1.20.0 

func BoolToJSON(x *Bool) ([]byte, error)

func Compare added in v1.10.0 

func Compare(a, b Schema) int

func FieldShape added in v1.21.0 

func FieldShape() shape.Shape

func IsNone added in v1.21.0 

func IsNone(x Schema) bool

func IsPrimitive added in v1.21.0 

func IsPrimitive(x any) bool

func ListShape added in v1.21.0 

func ListShape() shape.Shape

func ListToJSON added in v1.20.0 

func ListToJSON(x *List) ([]byte, error)

func LocationAnythingToJSON added in v1.20.0 

func LocationAnythingToJSON(x *LocationAnything) ([]byte, error)

func LocationFieldToJSON added in v1.20.0 

func LocationFieldToJSON(x *LocationField) ([]byte, error)

func LocationIndexToJSON added in v1.20.0 

func LocationIndexToJSON(x *LocationIndex) ([]byte, error)

func LocationToJSON added in v1.20.0 

func LocationToJSON(x Location) ([]byte, error)

func LocationToStr added in v1.19.0 

func LocationToStr(location []Location) string

func MapShape added in v1.21.0 

func MapShape() shape.Shape

func MapToJSON added in v1.20.0 

func MapToJSON(x *Map) ([]byte, error)

func MatchLocation added in v1.19.0 

func MatchLocation[TOut any](
	x Location,
	f1 func(x *LocationField) TOut,
	f2 func(x *LocationIndex) TOut,
	f3 func(x *LocationAnything) TOut,
	df func(x Location) TOut,
) TOut

func MatchLocationR2 added in v1.19.0 

func MatchLocationR2[TOut1, TOut2 any](
	x Location,
	f1 func(x *LocationField) (TOut1, TOut2),
	f2 func(x *LocationIndex) (TOut1, TOut2),
	f3 func(x *LocationAnything) (TOut1, TOut2),
	df func(x Location) (TOut1, TOut2),
) (TOut1, TOut2)

func MatchSchema added in v1.10.0 

func MatchSchema[TOut any](
	x Schema,
	f1 func(x *None) TOut,
	f2 func(x *Bool) TOut,
	f3 func(x *Number) TOut,
	f4 func(x *String) TOut,
	f5 func(x *Binary) TOut,
	f6 func(x *List) TOut,
	f7 func(x *Map) TOut,
	df func(x Schema) TOut,
) TOut

func MatchSchemaR2 added in v1.10.0 

func MatchSchemaR2[TOut1, TOut2 any](
	x Schema,
	f1 func(x *None) (TOut1, TOut2),
	f2 func(x *Bool) (TOut1, TOut2),
	f3 func(x *Number) (TOut1, TOut2),
	f4 func(x *String) (TOut1, TOut2),
	f5 func(x *Binary) (TOut1, TOut2),
	f6 func(x *List) (TOut1, TOut2),
	f7 func(x *Map) (TOut1, TOut2),
	df func(x Schema) (TOut1, TOut2),
) (TOut1, TOut2)

func MustMatchLocation added in v1.19.0 

func MustMatchLocation[TOut any](
	x Location,
	f1 func(x *LocationField) TOut,
	f2 func(x *LocationIndex) TOut,
	f3 func(x *LocationAnything) TOut,
) TOut

func MustMatchLocationR0 added in v1.19.0 

func MustMatchLocationR0(
	x Location,
	f1 func(x *LocationField),
	f2 func(x *LocationIndex),
	f3 func(x *LocationAnything),
)

func MustMatchLocationR2 added in v1.19.0 

func MustMatchLocationR2[TOut1, TOut2 any](
	x Location,
	f1 func(x *LocationField) (TOut1, TOut2),
	f2 func(x *LocationIndex) (TOut1, TOut2),
	f3 func(x *LocationAnything) (TOut1, TOut2),
) (TOut1, TOut2)

func MustMatchSchema added in v1.7.3 

func MustMatchSchema[TOut any](
	x Schema,
	f1 func(x *None) TOut,
	f2 func(x *Bool) TOut,
	f3 func(x *Number) TOut,
	f4 func(x *String) TOut,
	f5 func(x *Binary) TOut,
	f6 func(x *List) TOut,
	f7 func(x *Map) TOut,
) TOut

func MustMatchSchemaR0 added in v1.14.1 

func MustMatchSchemaR0(
	x Schema,
	f1 func(x *None),
	f2 func(x *Bool),
	f3 func(x *Number),
	f4 func(x *String),
	f5 func(x *Binary),
	f6 func(x *List),
	f7 func(x *Map),
)

func MustMatchSchemaR2 added in v1.7.3 

func MustMatchSchemaR2[TOut1, TOut2 any](
	x Schema,
	f1 func(x *None) (TOut1, TOut2),
	f2 func(x *Bool) (TOut1, TOut2),
	f3 func(x *Number) (TOut1, TOut2),
	f4 func(x *String) (TOut1, TOut2),
	f5 func(x *Binary) (TOut1, TOut2),
	f6 func(x *List) (TOut1, TOut2),
	f7 func(x *Map) (TOut1, TOut2),
) (TOut1, TOut2)

func NoneShape added in v1.21.0 

func NoneShape() shape.Shape

func NoneToJSON added in v1.20.0 

func NoneToJSON(x *None) ([]byte, error)

func NumberShape added in v1.21.0 

func NumberShape() shape.Shape

func NumberToJSON added in v1.20.0 

func NumberToJSON(x *Number) ([]byte, error)

func Reduce added in v1.7.7 

func Reduce[A any](data Schema, init A, fn func(Schema, A) A) A

func SchemaShape added in v1.21.0 

func SchemaShape() shape.Shape

func SchemaToJSON added in v1.20.0 

func SchemaToJSON(x Schema) ([]byte, error)

func StringShape added in v1.21.0 

func StringShape() shape.Shape

func StringToJSON added in v1.20.0 

func StringToJSON(x *String) ([]byte, error)

func ToDynamoDB added in v1.7.7 

func ToDynamoDB(x Schema) types.AttributeValue

func ToGo added in v1.7.2 

func ToGo[A any](x Schema) A

func ToGoG added in v1.12.0 

func ToGoG[A any](x Schema) (res A, err error)

func ToGoPrimitive added in v1.21.0 

func ToGoPrimitive(x Schema) (any, error)

func ToGoReflect added in v1.21.0 

func ToGoReflect(xshape shape.Shape, ydata Schema, zreflect reflect.Type) (reflect.Value, error)

Types

type Acc added in v1.19.0 

type Acc struct {
	Name  *string `@(String|Char|RawString)`
	Index *int    `| @Int`
	Any   bool    `| @("*") `
}

func (Acc) ToAccessor added in v1.19.0 

func (a Acc) ToAccessor() Location

type Binary added in v1.11.0 

type Binary []byte

func BinaryFromJSON added in v1.20.0 

func BinaryFromJSON(x []byte) (*Binary, error)

func MkBinary added in v1.11.0 

func MkBinary(b []byte) *Binary

func (*Binary) AcceptSchema added in v1.15.0 

func (r *Binary) AcceptSchema(v SchemaVisitor) any

func (*Binary) MarshalJSON added in v1.20.0 

func (r *Binary) MarshalJSON() ([]byte, error)

func (*Binary) UnmarshalJSON added in v1.20.0 

func (r *Binary) UnmarshalJSON(data []byte) error

type Bool 

type Bool bool

func BoolFromJSON added in v1.20.0 

func BoolFromJSON(x []byte) (*Bool, error)

func MkBool added in v1.10.0 

func MkBool(b bool) *Bool

func (*Bool) AcceptSchema added in v1.15.0 

func (r *Bool) AcceptSchema(v SchemaVisitor) any

func (*Bool) MarshalJSON added in v1.20.0 

func (r *Bool) MarshalJSON() ([]byte, error)

func (*Bool) UnmarshalJSON added in v1.20.0 

func (r *Bool) UnmarshalJSON(data []byte) error

type Field 

type Field struct {
	Name  string
	Value Schema
}

func MkField added in v1.10.0 

func MkField(name string, value Schema) Field

func (*Field) MarshalJSON added in v1.20.1 

func (r *Field) MarshalJSON() ([]byte, error)

func (*Field) UnmarshalJSON added in v1.20.1 

func (r *Field) UnmarshalJSON(data []byte) error

type List 

type List []Schema

func ListFromJSON added in v1.20.0 

func ListFromJSON(x []byte) (*List, error)

func MkList added in v1.10.0 

func MkList(items ...Schema) *List

func (*List) AcceptSchema added in v1.15.0 

func (r *List) AcceptSchema(v SchemaVisitor) any

func (*List) MarshalJSON added in v1.20.0 

func (r *List) MarshalJSON() ([]byte, error)

func (*List) UnmarshalJSON added in v1.20.0 

func (r *List) UnmarshalJSON(data []byte) error

type Location added in v1.19.0 

type Location interface {
	AcceptLocation(g LocationVisitor) any
}

func LocationFromJSON added in v1.20.0 

func LocationFromJSON(x []byte) (Location, error)

func MustParseLocation added in v1.19.0 

func MustParseLocation(input string) []Location

func ParseLocation added in v1.19.0 

func ParseLocation(input string) ([]Location, error)

type LocationAnything added in v1.19.0 

type LocationAnything struct{}

func LocationAnythingFromJSON added in v1.20.0 

func LocationAnythingFromJSON(x []byte) (*LocationAnything, error)

func (*LocationAnything) AcceptLocation added in v1.19.0 

func (r *LocationAnything) AcceptLocation(v LocationVisitor) any

func (*LocationAnything) MarshalJSON added in v1.20.0 

func (r *LocationAnything) MarshalJSON() ([]byte, error)

func (*LocationAnything) UnmarshalJSON added in v1.20.0 

func (r *LocationAnything) UnmarshalJSON(data []byte) error

type LocationField added in v1.19.0 

type LocationField struct {
	Name string
}

func LocationFieldFromJSON added in v1.20.0 

func LocationFieldFromJSON(x []byte) (*LocationField, error)

func (*LocationField) AcceptLocation added in v1.19.0 

func (r *LocationField) AcceptLocation(v LocationVisitor) any

func (*LocationField) MarshalJSON added in v1.20.0 

func (r *LocationField) MarshalJSON() ([]byte, error)

func (*LocationField) UnmarshalJSON added in v1.20.0 

func (r *LocationField) UnmarshalJSON(data []byte) error

type LocationIndex added in v1.19.0 

type LocationIndex struct {
	Index int
}

func LocationIndexFromJSON added in v1.20.0 

func LocationIndexFromJSON(x []byte) (*LocationIndex, error)

func (*LocationIndex) AcceptLocation added in v1.19.0 

func (r *LocationIndex) AcceptLocation(v LocationVisitor) any

func (*LocationIndex) MarshalJSON added in v1.20.0 

func (r *LocationIndex) MarshalJSON() ([]byte, error)

func (*LocationIndex) UnmarshalJSON added in v1.20.0 

func (r *LocationIndex) UnmarshalJSON(data []byte) error

type LocationUnionJSON added in v1.20.0 

type LocationUnionJSON struct {
	Type             string          `json:"$type,omitempty"`
	LocationField    json.RawMessage `json:"schema.LocationField,omitempty"`
	LocationIndex    json.RawMessage `json:"schema.LocationIndex,omitempty"`
	LocationAnything json.RawMessage `json:"schema.LocationAnything,omitempty"`
}

type LocationVisitor added in v1.19.0 

type LocationVisitor interface {
	VisitLocationField(v *LocationField) any
	VisitLocationIndex(v *LocationIndex) any
	VisitLocationAnything(v *LocationAnything) any
}

type Map 

type Map map[string]Schema

func MapFromJSON added in v1.20.0 

func MapFromJSON(x []byte) (*Map, error)

func MkMap added in v1.10.0 

func MkMap(fields ...Field) *Map

func (*Map) AcceptSchema added in v1.15.0 

func (r *Map) AcceptSchema(v SchemaVisitor) any

func (*Map) MarshalJSON added in v1.20.0 

func (r *Map) MarshalJSON() ([]byte, error)

func (*Map) UnmarshalJSON added in v1.20.0 

func (r *Map) UnmarshalJSON(data []byte) error

type None 

type None struct{}

func MkNone added in v1.11.0 

func MkNone() *None

func NoneFromJSON added in v1.20.0 

func NoneFromJSON(x []byte) (*None, error)

func (*None) AcceptSchema added in v1.15.0 

func (r *None) AcceptSchema(v SchemaVisitor) any

func (*None) MarshalJSON added in v1.20.0 

func (r *None) MarshalJSON() ([]byte, error)

func (*None) UnmarshalJSON added in v1.20.0 

func (r *None) UnmarshalJSON(data []byte) error

type Number 

type Number float64

func MkFloat added in v1.11.0 

func MkFloat(x float64) *Number

func MkInt 

func MkInt(x int) *Number

func NumberFromJSON added in v1.20.0 

func NumberFromJSON(x []byte) (*Number, error)

func (*Number) AcceptSchema added in v1.15.0 

func (r *Number) AcceptSchema(v SchemaVisitor) any

func (*Number) MarshalJSON added in v1.20.0 

func (r *Number) MarshalJSON() ([]byte, error)

func (*Number) UnmarshalJSON added in v1.20.0 

func (r *Number) UnmarshalJSON(data []byte) error

type Part added in v1.19.0 

type Part struct {
	Location string `@Ident`
	Acc      []Acc  `("[" @@ "]")*`
}

func (Part) ToLocation added in v1.19.0 

func (p Part) ToLocation() []Location

type PathAst added in v1.19.0 

type PathAst struct {
	Parts []Part `@@ ( "." @@ )*`
}

func (PathAst) ToLocation added in v1.19.0 

func (ast PathAst) ToLocation() ([]Location, error)

type Schema added in v1.7.3 

type Schema interface {
	AcceptSchema(g SchemaVisitor) any
}

func FromDynamoDB added in v1.7.7 

func FromDynamoDB(x types.AttributeValue) (Schema, error)

func FromGo added in v1.7.3 

func FromGo[A any](x A) Schema

func FromGoReflect added in v1.21.0 

func FromGoReflect(xschema shape.Shape, yreflect reflect.Value) Schema

func FromPrimitiveGo added in v1.21.0 

func FromPrimitiveGo(x any) Schema

func Get added in v1.9.0 

func Get[A any](data A, location string) (Schema, shape.Shape)

func GetSchema added in v1.21.0 

func GetSchema(data Schema, location string) Schema

func GetSchemaLocation added in v1.21.0 

func GetSchemaLocation(data Schema, locations []Location) Schema

func GetShapeLocation added in v1.21.0 

func GetShapeLocation(s shape.Shape, data Schema, location string) (Schema, shape.Shape)

func GetShapeSchemaLocation added in v1.21.0 

func GetShapeSchemaLocation(s shape.Shape, data Schema, locations []Location) (Schema, shape.Shape)

func SchemaFromJSON added in v1.20.0 

func SchemaFromJSON(x []byte) (Schema, error)

func UnwrapDynamoDB added in v1.12.0 

func UnwrapDynamoDB(data Schema) (Schema, error)

type SchemaUnionJSON added in v1.20.0 

type SchemaUnionJSON struct {
	Type   string          `json:"$type,omitempty"`
	None   json.RawMessage `json:"schema.None,omitempty"`
	Bool   json.RawMessage `json:"schema.Bool,omitempty"`
	Number json.RawMessage `json:"schema.Number,omitempty"`
	String json.RawMessage `json:"schema.String,omitempty"`
	Binary json.RawMessage `json:"schema.Binary,omitempty"`
	List   json.RawMessage `json:"schema.List,omitempty"`
	Map    json.RawMessage `json:"schema.Map,omitempty"`
}

type SchemaVisitor added in v1.7.3 

type SchemaVisitor interface {
	VisitNone(v *None) any
	VisitBool(v *Bool) any
	VisitNumber(v *Number) any
	VisitString(v *String) any
	VisitBinary(v *Binary) any
	VisitList(v *List) any
	VisitMap(v *Map) any
}

type String 

type String string

func MkString 

func MkString(s string) *String

func StringFromJSON added in v1.20.0 

func StringFromJSON(x []byte) (*String, error)

func (*String) AcceptSchema added in v1.15.0 

func (r *String) AcceptSchema(v SchemaVisitor) any

func (*String) MarshalJSON added in v1.20.0 

func (r *String) MarshalJSON() ([]byte, error)

func (*String) UnmarshalJSON added in v1.20.0 

func (r *String) UnmarshalJSON(data []byte) error

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.