jsonstruct

README

go-jsonstruct

Generate Go structs from multiple JSON objects.

• What does go-jsonstruct do and why should I use it?
• How do I use go-jsonstruct?
• What are go-jsonstruct's key features?
• How does go-jsonstruct work?
• License

What does go-jsonstruct do and why should I use it?

go-jsonstruct generates Go structs from multiple JSON objects. Existing Go struct generators such as json-to-go and json2struct take only a single JSON object as input. go-jsonstruct takes multiple JSON objects as input and generates the most specific Go struct possible into which all the input objects can be unmarshalled.

This is useful if you have a collection of JSON objects, where no single object has all properties present, and you want to unmarshal those JSON objects into a Go program. Example collections include:

• JSON responses received from a REST API with no defined schema.
• Multiple values from a JSON column in an SQL database.
• All the JSON documents in a document database.

How do I use go-jsonstruct?

Install go-jsonstruct:

go install github.com/liuyuqiang/go-jsonstruct-mapstructure/cmd/gojsonstruct@latest

Feed it some JSON objects. For example you can feed it with

{
  "age": 37,
  "user_height_m": 2
}

{
  "age": 38,
  "user_height_m": 1.7,
  "favoriteFoods": [
    "cake"
  ]
}

by running

echo '{"age":37,"user_height_m":2}' \
    '{"age":38,"user_height_m":1.7,"favoriteFoods":["cake"]}' \
    | gojsonstruct

This will output:

package main

type T struct {
    Age           int      `json:"age"`
    FavoriteFoods []string `json:"favoriteFoods,omitempty"`
    UserHeightM   float64  `json:"user_height_m"`
}

This example demonstrates:

• age is always observed as an integer, and so is given type int. The lower-case JSON property age is converted into an exported Go field name Age for compatibility with encoding/json.
• favoriteFoods is observed as a JSON array, but is not always present, but when it is present it only contains JSON strings, and so is given type []string. The camelCase name favoriteFoods is converted into the exported Go field name FavoriteFoods and is tagged with omitempty.
• user_height_m is observed as JSON numbers 2 and 1.7, for which the most specific Go type is float64. The snake_case name user_height_m is converted to the exported Go field name UserHeightM.
• Properties are sorted alphabetically.

go-jsonstruct recursively handles nested array elements and objects. For example, given the following three JSON objects input:

{
  "nested": {
    "bar": true,
    "foo": "baz"
  }
}

{
  "nested": {
    "bar": false,
    "foo": null
  }
}

{
  "nested": {
    "bar": true,
    "foo": ""
  }
}

which you can try by running

echo '{"nested":{"bar":true,"foo":"baz"}}' \
    '{"nested":{"bar":false,"foo":null}}' \
    '{"nested":{"bar":true,"foo":""}}' \
    | gojsonstruct --package-name mypackage --typename MyType

generates the output

package mypackage

type MyType struct {
    Nested struct {
        Bar bool    `json:"bar"`
        Foo *string `json:"foo"`
    } `json:"nested"`
}

This demonstrates:

• The package name and type name can be set on the command line.
• The arbitrarily-complex property nested is recursively converted to a nested struct that all values can be unmarshalled to. go-jsonstruct handles array elements in an identical fashion, resolving array elements to the most-specific type.
• nested.bar is always observed as a JSON bool, and is converted to a field of type bool.
• nested.foo is observed as a JSON string, a JSON null, and an empty JSON string and is converted to a field of type *string without omitempty. With omitempty, Go's encoding/json omits the field in the two cases of nil and a pointer to an empty string, so there is no way to distinguish between the observed values of null and "". go-jsonstruct falls back to the option of *string without omitempty which means that a value is always present, even if empty.

You can feed it your own data via the standard input, for example if you have a file with one JSON object per line in objects.json you can run:

gojsonstruct < objects.json

To learn about more about the available options, run:

gojsonstruct --help

What are go-jsonstruct's key features?

• Finds the most specific Go type that can represent all input values.
• Generates Go struct field names from camelCase, kebab-case, and snake_case JSON object property names.
• Capitalizes common abbreviations (e.g. HTTP, ID, and URL) when generating Go struct field names to follow Go conventions, with the option to add your own abbreviations.
• Gives you control over the output, including the generated package name, type name, and godoc-compatible comments.
• Generates deterministic output based only on the determined structure of the input, making it suitable for incorporation into build pipelines or detecting schema changes.
• Generates ,omitempty tags.
• Generates ,string tags.
• Uses the standard library's time.Time when possible.
• Gracefully handles properties with spaces that cannot be unmarshalled by encoding/json.

How does go-jsonstruct work?

go-jsonstruct consists of two phases: observation and code generation.

Firstly, in the observation phase, go-jsonstruct explores all the input objects and records statistics on what types are observed in each part. It recurses into objects and iterates over arrays.

Secondly, in the code generation phase, go-jsonstruct inspects the gathered statistics and determines the strictest possible Go type that can represent all the observed values. For example, the values 0 and 1 can be represented as an int, the values 0, 1, and 2.2 require a float64, and true, 3.3, and "name" require an any.

License

BSD

Documentation

Index

• func DefaultExportNameFunc(name string, abbreviations map[string]bool) string
• func SplitComponents(name string) []string
• type ExportNameFunc
• type Generator
  • func NewGenerator(options ...GeneratorOption) *Generator
  • func (g *Generator) Generate() ([]byte, error)
  • func (g *Generator) ObserveJSONFile(filename string) error
  • func (g *Generator) ObserveJSONReader(r io.Reader) error
  • func (g *Generator) ObserveValue(value any)
  • func (g *Generator) ObserveYAMLFile(filename string) error
  • func (g *Generator) ObserveYAMLReader(r io.Reader) error
• type GeneratorOption
  • func WithAbbreviations(abbreviations ...string) GeneratorOption
  • func WithAddStructTagName(structTagName string) GeneratorOption
  • func WithExportNameFunc(exportNameFunc ExportNameFunc) GeneratorOption
  • func WithExtraAbbreviations(abbreviations ...string) GeneratorOption
  • func WithGoFormat(goFormat bool) GeneratorOption
  • func WithImports(imports ...string) GeneratorOption
  • func WithIntType(intType string) GeneratorOption
  • func WithOmitEmptyTags(omitEmptyTags OmitEmptyTagsType) GeneratorOption
  • func WithPackageComment(packageComment string) GeneratorOption
  • func WithPackageName(packageName string) GeneratorOption
  • func WithRenames(renames map[string]string) GeneratorOption
  • func WithSkipUnparsableProperties(skipUnparsableProperties bool) GeneratorOption
  • func WithStringTags(stringTags bool) GeneratorOption
  • func WithStructTagName(structTagName string) GeneratorOption
  • func WithStructTagNames(structTagNames []string) GeneratorOption
  • func WithTypeComment(typeComment string) GeneratorOption
  • func WithTypeName(typeName string) GeneratorOption
  • func WithUseJSONNumber(useJSONNumber bool) GeneratorOption
• type OmitEmptyTagsType

Examples

• Generator.ObserveJSONFile
• Generator.ObserveYAMLFile

Functions

func DefaultExportNameFunc

func DefaultExportNameFunc(name string, abbreviations map[string]bool) string

DefaultExportNameFunc returns the exported name for name.

func SplitComponents

func SplitComponents(name string) []string

SplitComponents splits name into components. name may be kebab case, snake case, or camel case.

Types

type ExportNameFunc

type ExportNameFunc func(string) string

An ExportNameFunc returns the exported name for a property.

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

A Generator generates Go types from observed values.

func NewGenerator

func NewGenerator(options ...GeneratorOption) *Generator

NewGenerator returns a new Generator with options.

func (*Generator) Generate

func (g *Generator) Generate() ([]byte, error)

Generate returns the Go source code for the observed values.

func (*Generator) ObserveJSONFile

func (g *Generator) ObserveJSONFile(filename string) error

ObserveJSONFile observes JSON values from filename.

Example

generator := NewGenerator()
if err := generator.ObserveJSONFile("testdata/example.json"); err != nil {
	panic(err)
}
data, err := generator.Generate()
if err != nil {
	panic(err)
}
os.Stdout.Write(data)

Output:

package main

type T struct {
	Age           int      `json:"age"`
	FavoriteFoods []string `json:"favoriteFoods,omitempty"`
	UserHeightM   float64  `json:"user_height_m"`
}

func (*Generator) ObserveJSONReader

func (g *Generator) ObserveJSONReader(r io.Reader) error

ObserveJSONReader observes JSON values from r.

func (*Generator) ObserveValue

func (g *Generator) ObserveValue(value any)

ObserveValue observes value.

func (*Generator) ObserveYAMLFile

func (g *Generator) ObserveYAMLFile(filename string) error

ObserveYAMLFile observes YAML values from filename.

Example

generator := NewGenerator(
	WithPackageName("mypackage"),
	WithTypeName("MyType"),
)
if err := generator.ObserveYAMLFile("testdata/example.yaml"); err != nil {
	panic(err)
}
data, err := generator.Generate()
if err != nil {
	panic(err)
}
os.Stdout.Write(data)

Output:

package mypackage

type MyType struct {
	Nested struct {
		Bar bool    `json:"bar"`
		Foo *string `json:"foo"`
	} `json:"nested"`
}

func (*Generator) ObserveYAMLReader

func (g *Generator) ObserveYAMLReader(r io.Reader) error

ObserveYAMLReader observes YAML values from r.

type GeneratorOption

type GeneratorOption func(*Generator)

A GeneratorOption sets an option on a Generator.

func WithAbbreviations

func WithAbbreviations(abbreviations ...string) GeneratorOption

WithAbbreviations sets the abbreviations.

func WithAddStructTagName

func WithAddStructTagName(structTagName string) GeneratorOption

WithAddStructTagName adds a struct tag name.

func WithExportNameFunc

func WithExportNameFunc(exportNameFunc ExportNameFunc) GeneratorOption

WithExportNameFunc sets the export name function.

func WithExtraAbbreviations

func WithExtraAbbreviations(abbreviations ...string) GeneratorOption

WithExtraAbbreviations adds abbreviations.

func WithGoFormat

func WithGoFormat(goFormat bool) GeneratorOption

WithGoFormat sets whether the output is should be formatted with go fmt.

func WithImports

func WithImports(imports ...string) GeneratorOption

WithImports adds custom package imports.

func WithIntType

func WithIntType(intType string) GeneratorOption

WithIntType sets the integer type.

func WithOmitEmptyTags

func WithOmitEmptyTags(omitEmptyTags OmitEmptyTagsType) GeneratorOption

WithOmitEmptyTags sets whether ",omitempty" tags should be used.

func WithPackageComment

func WithPackageComment(packageComment string) GeneratorOption

WithPackageComment sets the package comment.

func WithPackageName

func WithPackageName(packageName string) GeneratorOption

WithPackageName sets the package name.

func WithRenames

func WithRenames(renames map[string]string) GeneratorOption

WithRenames sets the renames.

func WithSkipUnparsableProperties

func WithSkipUnparsableProperties(skipUnparsableProperties bool) GeneratorOption

WithSkipUnparsableProperties sets whether unparsable properties should be skipped.

func WithStringTags

func WithStringTags(stringTags bool) GeneratorOption

WithStringTags sets whether ",string" tags should be used.

func WithStructTagName

func WithStructTagName(structTagName string) GeneratorOption

WithStructTagName sets the struct tag name.

func WithStructTagNames

func WithStructTagNames(structTagNames []string) GeneratorOption

WithStructTagNames sets the struct tag names.

func WithTypeComment

func WithTypeComment(typeComment string) GeneratorOption

WithTypeComment sets the type comment.

func WithTypeName

func WithTypeName(typeName string) GeneratorOption

WithTypeName sets the type name.

func WithUseJSONNumber

func WithUseJSONNumber(useJSONNumber bool) GeneratorOption

WithUseJSONNumber sets whether to use json.Number when both int and float64s are observed for the same property.

type OmitEmptyTagsType

type OmitEmptyTagsType int

An OmitEmptyTagsType sets how to handle ,omitempty tags.

const (
	OmitEmptyTagsNever OmitEmptyTagsType = iota
	OmitEmptyTagsAlways
	OmitEmptyTagsAuto
)

OmitEmptyTags values.