extjson

README

extjson

extjson extends the JSON syntax with following features:

1. trailing comma of Object and Array
2. traditional, end of line, and pragma comments
3. simple identifier as object key without quotes
4. Python boolean constants
5. Python None as null
6. Python style single quote string
7. read environment variables
8. include other JSON files (with max depth limited)
9. reference to other values in same file, using gjson path syntax
10. evaluate expressions at runtime, with frequently used builtin functions

It helps in many scenes, such as:

1. working with data generated by python programs
2. configuration with comments
3. managing many json files with duplicate contents
4. data driven testing by JSON files

See example and godoc for more details.

Example

{
    // A comment! You normally can't put these in JSON
    "obj1": {
        "foo": "bar", // <-- A trailing comma! No worries.
    },
    /*
    This style of comments will also be safely removed.
    */
    "array": [1, 2, 3, ], // Trailing comma in array.
    "include": @incl("testdata.json"), // Include another json file.
    identifier_simple1: 1234,
    $identifierSimple2: "abc",
    "obj2": {
        "foo": "bar", /* Another style inline comment. */
    }, // <-- Another trailing comma!
    'py_true': True, // Single quote string and True as true value.
    py_false: False, // Simple identifier and Python False as false value.
    py_none: None,   /* Simple identifier and Python None as null value. */
    "test_env": @env("SOME_ENV"),  // Read environment variable.
    "test_ref1": @ref("obj1.foo"), // Reference to other values, wil be "bar".
    "test_ref2": @ref("array.2"),  // Another reference, will be 3.
    "test_ref3": @ref("array.#"),  // Get length of "array", will be 3.
    "friends": [
        {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]},
        {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]},
        {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]}
    ],
    "test_ref4": @ref("friends.#.first"), // Will be ["Dale","Roger","Jane"].
    "test_fn1": @fn("nowUnix"),   // Call builtin function nowUnix.
    test_fn2: @fn("nowFormat('2006-01-02')"), // Call builtin function nowFormat("2006-01-02").
    'test_fn3': @fn("uuid"),      // Call builtin function uuid.
    test_fn4: @fn('randN(10)'),   // Call builtin function randN(10).
    test_fn5: @fn('randStr(16)'), // Call builtin function randStr(16).
}

With environment variable "SOME_ENV" set to "some-env-value", the above JSON will be resolved to following:

{
  "obj1": { "foo": "bar" },
  "array": [ 1, 2, 3 ],
  "include": { "foo": "bar" },
  "identifier_simple1": 1234,
  "$identifierSimple2": "abc",
  "obj2": { "foo": "bar" },
  "py_true": true,
  "py_false": false,
  "py_none": null,
  "test_env": "some-env-value",
  "test_ref1": "bar",
  "test_ref2": 3,
  "test_ref3": 3,
  "friends": [
    { "age": 44, "first": "Dale", "last": "Murphy", "nets": [ "ig", "fb", "tw" ] },
    { "age": 68, "first": "Roger", "last": "Craig", "nets": [ "fb", "tw" ] },
    { "age": 47, "first": "Jane", "last": "Murphy", "nets": [ "ig", "tw" ] }
  ],
  "test_ref4": [ "Dale", "Roger", "Jane" ],
  "test_fn1": 1643162035,
  "test_fn2": "2022-01-26",
  "test_fn3": "4de97237-8ff1-4cc6-80db-d5485ad2b82e",
  "test_fn4": 5,
  "test_fn5": "YewEXAuRrsI3pUCC"
}

Documentation

Overview

Example

data := `{
		// A comment! You normally can't put these in JSON
		"obj1": {
			"foo": "bar", // <-- A trailing comma! No worries.
		},
		/*
		This style of comments will also be safely removed.
		*/
		"array": [1, 2, 3, ], // Trailing comma in array.
		"include": @incl("testdata.json"), // Include another json file.
		identifier_simple1: 1234,
		$identifierSimple2: "abc",
		"obj2": {
			"foo": "bar", /* Another style inline comment. */
		}, // <-- Another trailing comma!
		'py_true': True, // Single quote string and True as true value.
		py_false: False, // Simple identifier and Python False as false value.
		py_none: None,   /* Simple identifier and Python None as null value. */
		"test_env": @env("SOME_ENV"),  // Read environment variable.
		"test_ref1": @ref("obj1.foo"), // Reference to other values, wil be "bar".
		"test_ref2": @ref("array.2"),  // Another reference, will be 3.
		"test_ref3": @ref("array.#"),  // Get length of "array", will be 3.
		"friends": [
			{"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]},
			{"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]},
			{"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]}
		],
		"test_ref4": @ref("friends.#.first"), // Will be ["Dale","Roger","Jane"].
	}`

os.Setenv("SOME_ENV", "some-env-value")

var clean json.RawMessage
err := Unmarshal([]byte(data), &clean, EnableEnv())
if err != nil {
	panic(err)
}
out, _ := json.MarshalIndent(clean, "", "  ")
fmt.Println(string(out))

Output:

{
  "obj1": {
    "foo": "bar"
  },
  "array": [
    1,
    2,
    3
  ],
  "include": {
    "foo": "bar"
  },
  "identifier_simple1": 1234,
  "$identifierSimple2": "abc",
  "obj2": {
    "foo": "bar"
  },
  "py_true": true,
  "py_false": false,
  "py_none": null,
  "test_env": "some-env-value",
  "test_ref1": "bar",
  "test_ref2": 3,
  "test_ref3": 3,
  "friends": [
    {
      "first": "Dale",
      "last": "Murphy",
      "age": 44,
      "nets": [
        "ig",
        "fb",
        "tw"
      ]
    },
    {
      "first": "Roger",
      "last": "Craig",
      "age": 68,
      "nets": [
        "fb",
        "tw"
      ]
    },
    {
      "first": "Jane",
      "last": "Murphy",
      "age": 47,
      "nets": [
        "ig",
        "tw"
      ]
    }
  ],
  "test_ref4": [
    "Dale",
    "Roger",
    "Jane"
  ]
}

Index

• func Clean(data []byte, options ...ExtOption) ([]byte, error)
• func Dump(path string, v interface{}, prefix, indent string) error
• func Load(path string, v interface{}, options ...ExtOption) error
• func Unmarshal(data []byte, v interface{}, options ...ExtOption) error
• type ExtOption
  • func EnableEnv() ExtOption
  • func IncludeRoot(dir string) ExtOption
  • func WithFuncMap(funcMap FuncMap) ExtOption
• type FuncMap

Examples

• Package

Functions

func Clean

func Clean(data []byte, options ...ExtOption) ([]byte, error)

Clean parses data with extended feature and returns it as normal spec-compliant JSON data.

func Dump

func Dump(path string, v interface{}, prefix, indent string) error

Dump writes v to the named file at path using JSON encoding. It disables HTMLEscape. Optionally indent can be applied to the output, empty prefix and indent disables indentation. The output is friendly to read by humans.

func Load

func Load(path string, v interface{}, options ...ExtOption) error

Load reads JSON-encoded data from the named file at path and stores the result in the value pointed to by v.

In additional to features of encoding/json, it enables extended features such as "trailing comma", "comments", "file including", "refer" etc. The extended features are documented in the README file.

func Unmarshal

func Unmarshal(data []byte, v interface{}, options ...ExtOption) error

Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v.

In addition to features of encoding/json, it enables extended features such as "trailing comma", "comments", "file including", "refer", etc. The extended features are documented in the README file.

Types

type ExtOption

type ExtOption struct {
	// contains filtered or unexported fields
}

ExtOption represents an option to customize the extended features.

func EnableEnv

func EnableEnv() ExtOption

EnableEnv enables reading environment variables. By default, it is disabled for security consideration.

func IncludeRoot

func IncludeRoot(dir string) ExtOption

IncludeRoot specifies the root directory to use with the extended file including feature.

func WithFuncMap

func WithFuncMap(funcMap FuncMap) ExtOption

WithFuncMap specifies additional functions to use with the "@fn" directive.

type FuncMap

type FuncMap map[string]interface{}

FuncMap is the type of the map defining the mapping from names to functions. Each function must have either a single return value, or two return values of which the second has type error. In that case, if the second (error) return value evaluates to non-nil during execution, execution terminates and the error will be returned.