gjson

README

GJSON

This is a wrapper lib of gjson, which is accelerated by sonic's algorithm.

Performance

It's usually faster than original one, especially for large JSON. (see codes here)

goversion: 1.22.0
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz
                         │  origin.out   │             target.out              │
                         │    sec/op     │   sec/op     vs base                │
GetComplexPath/small-32     3.000µ ±  6%   2.591µ ± 6%  -13.65% (p=0.000 n=10)
GetComplexPath/medium-32   24.855µ ±  5%   6.464µ ± 8%  -74.00% (p=0.000 n=10)
GetComplexPath/large-32    1309.1µ ±  3%   270.0µ ± 8%  -79.37% (p=0.000 n=10)
GetSimplePath/small-32      702.1n ± 11%   634.3n ± 5%   -9.66% (p=0.000 n=10)
GetSimplePath/medium-32     9.558µ ±  3%   1.744µ ± 4%  -81.76% (p=0.000 n=10)
GetSimplePath/Large-32     342.03µ ± 14%   37.83µ ± 5%  -88.94% (p=0.000 n=10)
geomean                     24.64µ         7.576µ       -69.26%

                         │  origin.out  │             target.out              │
                         │     B/op     │    B/op     vs base                 │
GetComplexPath/small-32    104.0 ± 0%     104.0 ± 0%       ~ (p=1.000 n=10) ¹
GetComplexPath/medium-32   16.00 ± 0%     16.00 ± 0%       ~ (p=1.000 n=10) ¹
GetComplexPath/large-32    16.00 ± 0%     16.00 ± 0%       ~ (p=1.000 n=10) ¹
GetSimplePath/small-32     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
GetSimplePath/medium-32    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
GetSimplePath/Large-32     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                               ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                         │  origin.out  │             target.out              │
                         │  allocs/op   │ allocs/op   vs base                 │
GetComplexPath/small-32    6.000 ± 0%     6.000 ± 0%       ~ (p=1.000 n=10) ¹
GetComplexPath/medium-32   2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=10) ¹
GetComplexPath/large-32    2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=10) ¹
GetSimplePath/small-32     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
GetSimplePath/medium-32    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
GetSimplePath/Large-32     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                               ²               +0.00%       

Options

FastPath

This option is used to cache parsed simple paths and search JSON path all-in-once in C funtions, to reduce the overhead of c-go interaction. By default, this option is disabled, because it will and consumes a little more memory than default. You can enable it by set environment variable GJSON_FAST_PATH=1.

FastString

By default, Gjson doesn't use the SIMD algorithm when decoding a string and does not validate UTF8 either, thus its string-value APIs' behaviors are slow and different from encoding/json. You can change the behaviors by setting the environment variable below:

• GJSON_FAST_STRING=1: SIMD-implemented string parsing. The string-value APIs' behaviors will be the same as sonic/decoder's default behavior, with 2~3X times the speed of default string-parsing.
• GJSON_FAST_STRING=2, SIMD-implemented string parsing, and UTF-8 validating. String-value APIs' behaviors will be totally same with encoding/json.Decode and also faster than the default.

Benchmark

This is a benchmark on the above options (see codes):

goversion: 1.22.0
goos: linux
goarch: amd64
pkg: github.com/tidwall/gjson/testdata
cpu: Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz
BenchmarkFastPath/normal/small-32                  741.1 ns/op        0 B/op        0 allocs/op
BenchmarkFastPath/normal/medium-32                  1846 ns/op        0 B/op        0 allocs/op
BenchmarkFastPath/normal/Large-32                  38675 ns/op        0 B/op        0 allocs/op
BenchmarkFastPath/fast-path/small-32               364.4 ns/op        0 B/op        0 allocs/op
BenchmarkFastPath/fast-path/medium-32               1662 ns/op        0 B/op        0 allocs/op
BenchmarkFastPath/fast-path/Large-32               38976 ns/op        0 B/op        0 allocs/op
BenchmarkParseString/normal/small-32               401.8 ns/op      192 B/op        2 allocs/op
BenchmarkParseString/normal/medium-32               4600 ns/op     2560 B/op        2 allocs/op
BenchmarkParseString/normal/large-32               63202 ns/op    27904 B/op        2 allocs/op
BenchmarkParseString/fast-string/small-32          219.7 ns/op       96 B/op        1 allocs/op
BenchmarkParseString/fast-string/medium-32          2076 ns/op     1408 B/op        1 allocs/op
BenchmarkParseString/fast-string/large-32          15814 ns/op    14336 B/op        1 allocs/op
BenchmarkParseString/validate-string/small-32      231.6 ns/op       96 B/op        1 allocs/op
BenchmarkParseString/validate-string/medium-32      2116 ns/op     1408 B/op        1 allocs/op
BenchmarkParseString/validate-string/large-32      16779 ns/op    14336 B/op        1 allocs/op

Documentation

Overview

Package gjson provides searching for json strings.

Index

• Variables
• func AddModifier(name string, fn func(json, arg string) string)
• func AppendJSONString(dst []byte, s string) []byte
• func Escape(comp string) string
• func ForEachLine(json string, iterator func(line Result) bool)
• func ModifierExists(name string, fn func(json, arg string) string) bool
• func Valid(json string) bool
• func ValidBytes(json []byte) bool
• type Result
  • func Get(json, path string) Result
  • func GetBytes(json []byte, path string) Result
  • func GetMany(json string, path ...string) []Result
  • func GetManyBytes(json []byte, path ...string) []Result
  • func Parse(json string) Result
  • func ParseBytes(json []byte) Result
  • func (t Result) Array() []Result
  • func (t Result) Bool() bool
  • func (t Result) Exists() bool
  • func (t Result) Float() float64
  • func (t Result) ForEach(iterator func(key, value Result) bool)
  • func (t Result) Get(path string) Result
  • func (t Result) Int() int64
  • func (t Result) IsArray() bool
  • func (t Result) IsBool() bool
  • func (t Result) IsObject() bool
  • func (t Result) Less(token Result, caseSensitive bool) bool
  • func (t Result) Map() map[string]Result
  • func (t Result) Path(json string) string
  • func (t Result) Paths(json string) []string
  • func (t Result) String() string
  • func (t Result) Time() time.Time
  • func (t Result) Uint() uint64
  • func (t Result) Value() interface{}
• type Type
  • func (t Type) String() string

Variables

var DisableEscapeHTML = false

DisableEscapeHTML will disable the automatic escaping of certain "problamatic" HTML characters when encoding to JSON. These character include '>', '<' and '&', which get escaped to \u003e, \u0026, and \u003c respectively.

This is a global flag and will affect all further gjson operations. Ideally, if used, it should be set one time before other gjson functions are called.

var DisableModifiers = false

DisableModifiers will disable the modifier syntax

Functions

func AddModifier

func AddModifier(name string, fn func(json, arg string) string)

AddModifier binds a custom modifier command to the GJSON syntax. This operation is not thread safe and should be executed prior to using all other gjson function.

func AppendJSONString

func AppendJSONString(dst []byte, s string) []byte

AppendJSONString is a convenience function that converts the provided string to a valid JSON string and appends it to dst.

func Escape

func Escape(comp string) string

Escape returns an escaped path component.

json := `{
  "user":{
     "first.name": "Janet",
     "last.name": "Prichard"
   }
}`
user := gjson.Get(json, "user")
println(user.Get(gjson.Escape("first.name"))
println(user.Get(gjson.Escape("last.name"))
// Output:
// Janet
// Prichard

func ForEachLine

func ForEachLine(json string, iterator func(line Result) bool)

ForEachLine iterates through lines of JSON as specified by the JSON Lines format (http://jsonlines.org/). Each line is returned as a GJSON Result.

func ModifierExists

func ModifierExists(name string, fn func(json, arg string) string) bool

ModifierExists returns true when the specified modifier exists.

func Valid

func Valid(json string) bool

Valid returns true if the input is valid json.

if !gjson.Valid(json) {
	return errors.New("invalid json")
}
value := gjson.Get(json, "name.last")

func ValidBytes

func ValidBytes(json []byte) bool

ValidBytes returns true if the input is valid json.

if !gjson.Valid(json) {
	return errors.New("invalid json")
}
value := gjson.Get(json, "name.last")

If working with bytes, this method preferred over ValidBytes(string(data))

Types

type Result

type Result struct {
	// Type is the json type
	Type Type
	// Raw is the raw json
	Raw string
	// Str is the json string
	Str string
	// Num is the json number
	Num float64
	// Index of raw value in original json, zero means index unknown
	Index int
	// Indexes of all the elements that match on a path containing the '#'
	// query character.
	Indexes []int
}

Result represents a json value that is returned from Get().

func Get

func Get(json, path string) Result

Get searches json for the specified path. A path is in dot syntax, such as "name.last" or "age". When the value is found it's returned immediately.

A path is a series of keys separated by a dot. A key may contain special wildcard characters '*' and '?'. To access an array value use the index as the key. To get the number of elements in an array or to access a child path, use the '#' character. The dot and wildcard character can be escaped with '\'.

{
  "name": {"first": "Tom", "last": "Anderson"},
  "age":37,
  "children": ["Sara","Alex","Jack"],
  "friends": [
    {"first": "James", "last": "Murphy"},
    {"first": "Roger", "last": "Craig"}
  ]
}
"name.last"          >> "Anderson"
"age"                >> 37
"children"           >> ["Sara","Alex","Jack"]
"children.#"         >> 3
"children.1"         >> "Alex"
"child*.2"           >> "Jack"
"c?ildren.0"         >> "Sara"
"friends.#.first"    >> ["James","Roger"]

This function expects that the json is well-formed, and does not validate. Invalid json will not panic, but it may return back unexpected results. If you are consuming JSON from an unpredictable source then you may want to use the Valid function first.

func GetBytes

func GetBytes(json []byte, path string) Result

GetBytes searches json for the specified path. If working with bytes, this method preferred over Get(string(data), path)

func GetMany

func GetMany(json string, path ...string) []Result

GetMany searches json for the multiple paths. The return value is a Result array where the number of items will be equal to the number of input paths.

func GetManyBytes

func GetManyBytes(json []byte, path ...string) []Result

GetManyBytes searches json for the multiple paths. The return value is a Result array where the number of items will be equal to the number of input paths.

func Parse

func Parse(json string) Result

Parse parses the json and returns a result.

This function expects that the json is well-formed, and does not validate. Invalid json will not panic, but it may return back unexpected results. If you are consuming JSON from an unpredictable source then you may want to use the Valid function first.

func ParseBytes

func ParseBytes(json []byte) Result

ParseBytes parses the json and returns a result. If working with bytes, this method preferred over Parse(string(data))

func (Result) Array

func (t Result) Array() []Result

Array returns back an array of values. If the result represents a null value or is non-existent, then an empty array will be returned. If the result is not a JSON array, the return value will be an array containing one result.

func (Result) Bool

func (t Result) Bool() bool

Bool returns an boolean representation.

func (Result) Exists

func (t Result) Exists() bool

Exists returns true if value exists.

 if gjson.Get(json, "name.last").Exists(){
		println("value exists")
 }

func (Result) Float

func (t Result) Float() float64

Float returns an float64 representation.

func (Result) ForEach

func (t Result) ForEach(iterator func(key, value Result) bool)

ForEach iterates through values. If the result represents a non-existent value, then no values will be iterated. If the result is an Object, the iterator will pass the key and value of each item. If the result is an Array, the iterator will only pass the value of each item. If the result is not a JSON array or object, the iterator will pass back one value equal to the result.

func (Result) Get

func (t Result) Get(path string) Result

Get searches result for the specified path. The result should be a JSON array or object.

func (Result) Int

func (t Result) Int() int64

Int returns an integer representation.

func (Result) IsArray

func (t Result) IsArray() bool

IsArray returns true if the result value is a JSON array.

func (Result) IsBool

func (t Result) IsBool() bool

IsBool returns true if the result value is a JSON boolean.

func (Result) IsObject

func (t Result) IsObject() bool

IsObject returns true if the result value is a JSON object.

func (Result) Less

func (t Result) Less(token Result, caseSensitive bool) bool

Less return true if a token is less than another token. The caseSensitive parameter is used when the tokens are Strings. The order when comparing two different type is:

Null < False < Number < String < True < JSON

func (Result) Map

func (t Result) Map() map[string]Result

Map returns back a map of values. The result should be a JSON object. If the result is not a JSON object, the return value will be an empty map.

func (Result) Path

func (t Result) Path(json string) string

Path returns the original GJSON path for a Result where the Result came from a simple path that returns a single value, like:

gjson.Get(json, "friends.#(last=Murphy)")

The returned value will be in the form of a JSON string:

"friends.0"

The param 'json' must be the original JSON used when calling Get.

Returns an empty string if the paths cannot be determined, which can happen when the Result came from a path that contained a multipath, modifier, or a nested query.

func (Result) Paths

func (t Result) Paths(json string) []string

Paths returns the original GJSON paths for a Result where the Result came from a simple query path that returns an array, like:

gjson.Get(json, "friends.#.first")

The returned value will be in the form of a JSON array:

["friends.0.first","friends.1.first","friends.2.first"]

The param 'json' must be the original JSON used when calling Get.

Returns an empty string if the paths cannot be determined, which can happen when the Result came from a path that contained a multipath, modifier, or a nested query.

func (Result) String

func (t Result) String() string

String returns a string representation of the value.

func (Result) Time

func (t Result) Time() time.Time

Time returns a time.Time representation.

func (Result) Uint

func (t Result) Uint() uint64

Uint returns an unsigned integer representation.

func (Result) Value

func (t Result) Value() interface{}

Value returns one of these types:

bool, for JSON booleans
float64, for JSON numbers
Number, for JSON numbers
string, for JSON string literals
nil, for JSON null
map[string]interface{}, for JSON objects
[]interface{}, for JSON arrays

type Type

type Type int

Type is Result type

const (
	// Null is a null json value
	Null Type = iota
	// False is a json false boolean
	False
	// Number is json number
	Number
	// String is a json string
	String
	// True is a json true boolean
	True
	// JSON is a raw block of JSON
	JSON
)

func (Type) String

func (t Type) String() string

String returns a string representation of the type.