README
¶
go-pgquery
go-pgquery is a drop-in replacement for pg_query_go which uses cgo to invoke the libpg_query library. By default, the c code is compiled to a WebAssembly module and accessed with the pure Go runtime, wazero. This means that it is compatible with any Go application, regardless of availability of cgo, and will not have toolchain issues such as those that libpg_query commonly sees with MacOS upgrades.
Unlike other wasilibs libraries, this library cannot be used with TinyGo because it relies on WebAssembly threads support, which is not expected in TinyGo for quite some time.
Compatibility
The library is expected to be fully compatible with pg_query_go. The same tests from pg_query_go are run, and in addition test cases from libpg_query are also imported to increase coverage. There are no known behavior differences with upstream libraries.
Stability
While all tests from both upstream sources currently execute, this is despite issues that were found that only affected the WebAssembly toolchain. While coverage is expected to be high, it is not perfect and it is possible future issues that only affect WebAssembly compilation will be found. If encountering any issues, always feel free to file an issue.
Usage
go-pgquery is a standard Go library package and can be added to a go.mod file. It will work fine in any Go project.
go get github.com/wasilibs/go-pgquery
Because the library is a drop-in replacement for pg_query_go, it is sufficient to change import statements.
import "github.com/pganalyze/pg_query_go/v5/parser"
can be changed to
import "github.com/wasilibs/go-pgquery"
Note that to allow users to write transformation logic targeting both libraries, this library
returns parse trees using the types from pg_query_go. This means you will have both libraries
in your requirements and will need to be careful calling the entry point functions like Parse
from go-pgquery, not pg_query_go. This may change in the future.
cgo
This library also supports opting into using cgo to wrap libpg_query instead of using WebAssembly.
The build tag pgquery_cgo can be used to enable cgo support. Note, this is exactly the same
code as pg_query_go - if you only need cgo support, it is recommended to use the official
library instead of this one.
Performance
Benchmarks are run against every commit in the bench workflow. GitHub action runners are highly virtualized and do not have stable performance across runs, but the relative numbers within a run should still be somewhat, though not precisely, informative.
One run looks like this
│ build/bench_default.txt │ build/bench.txt │ build/bench_cgo.txt │
│ sec/op │ sec/op vs base │ sec/op vs base │
ParseSelect1-4 24.574µ ± ∞ ¹ 24.859µ ± ∞ ¹ ~ (p=0.421 n=5) 6.558µ ± ∞ ¹ -73.31% (p=0.008 n=5)
ParseSelect2-4 82.56µ ± ∞ ¹ 81.40µ ± ∞ ¹ -1.41% (p=0.032 n=5) 21.68µ ± ∞ ¹ -73.74% (p=0.008 n=5)
ParseCreateTable-4 186.70µ ± ∞ ¹ 186.20µ ± ∞ ¹ ~ (p=1.000 n=5) 51.30µ ± ∞ ¹ -72.52% (p=0.008 n=5)
ParseSelect1Parallel-4 12.027µ ± ∞ ¹ 11.793µ ± ∞ ¹ ~ (p=0.548 n=5) 3.294µ ± ∞ ¹ -72.61% (p=0.008 n=5)
ParseSelect2Parallel-4 39.83µ ± ∞ ¹ 39.23µ ± ∞ ¹ -1.50% (p=0.032 n=5) 11.41µ ± ∞ ¹ -71.36% (p=0.008 n=5)
ParseCreateTableParallel-4 91.27µ ± ∞ ¹ 90.28µ ± ∞ ¹ ~ (p=0.222 n=5) 27.76µ ± ∞ ¹ -69.58% (p=0.008 n=5)
RawParseSelect1-4 22.439µ ± ∞ ¹ 22.049µ ± ∞ ¹ ~ (p=0.548 n=5) 4.613µ ± ∞ ¹ -79.44% (p=0.008 n=5)
RawParseSelect2-4 73.07µ ± ∞ ¹ 74.15µ ± ∞ ¹ ~ (p=0.690 n=5) 15.50µ ± ∞ ¹ -78.80% (p=0.008 n=5)
RawParseCreateTable-4 167.01µ ± ∞ ¹ 167.32µ ± ∞ ¹ ~ (p=1.000 n=5) 36.12µ ± ∞ ¹ -78.38% (p=0.008 n=5)
RawParseSelect1Parallel-4 10.667µ ± ∞ ¹ 10.552µ ± ∞ ¹ ~ (p=0.095 n=5) 2.280µ ± ∞ ¹ -78.63% (p=0.008 n=5)
RawParseSelect2Parallel-4 35.777µ ± ∞ ¹ 35.309µ ± ∞ ¹ ~ (p=0.135 n=5) 7.798µ ± ∞ ¹ -78.20% (p=0.008 n=5)
RawParseCreateTableParallel-4 82.15µ ± ∞ ¹ 81.99µ ± ∞ ¹ ~ (p=0.079 n=5) 18.68µ ± ∞ ¹ -77.26% (p=0.008 n=5)
FingerprintSelect1-4 15.984µ ± ∞ ¹ 15.631µ ± ∞ ¹ ~ (p=0.310 n=5) 2.465µ ± ∞ ¹ -84.58% (p=0.008 n=5)
FingerprintSelect2-4 36.803µ ± ∞ ¹ 36.140µ ± ∞ ¹ ~ (p=0.095 n=5) 5.006µ ± ∞ ¹ -86.40% (p=0.008 n=5)
FingerprintCreateTable-4 60.191µ ± ∞ ¹ 60.951µ ± ∞ ¹ ~ (p=0.151 n=5) 9.172µ ± ∞ ¹ -84.76% (p=0.008 n=5)
NormalizeSelect1-4 9.864µ ± ∞ ¹ 10.151µ ± ∞ ¹ ~ (p=0.095 n=5) 1.691µ ± ∞ ¹ -82.86% (p=0.008 n=5)
NormalizeSelect2-4 21.154µ ± ∞ ¹ 22.804µ ± ∞ ¹ +7.80% (p=0.008 n=5) 2.987µ ± ∞ ¹ -85.88% (p=0.008 n=5)
NormalizeCreateTable-4 21.750µ ± ∞ ¹ 22.841µ ± ∞ ¹ +5.02% (p=0.008 n=5) 4.312µ ± ∞ ¹ -80.17% (p=0.008 n=5)
We see that the WebAssembly version performs about 4-5x slower than cgo. This is a big difference and the largest we've found in wasilibs. It may be because of inefficiency in the implementation of exception handling. If your application requires online processing of high volume of queries, it may be better to continue with cgo. However, if it is an offline tool that only parses a small number of queries, it may still be reasonable performance in exchange for a simplified build toolchain - operations still complete within dozens of microseconds.
Documentation
¶
Index ¶
- func Deparse(tree *pganalyze.ParseResult) (output string, err error)
- func Fingerprint(input string) (result string, err error)
- func FingerprintToUInt64(input string) (result uint64, err error)
- func HashXXH3_64(input []byte, seed uint64) (result uint64)
- func Normalize(input string) (result string, err error)
- func Parse(input string) (tree *pganalyze.ParseResult, err error)
- func ParsePlPgSqlToJSON(input string) (result string, err error)
- func ParseToJSON(input string) (result string, err error)
- func Scan(input string) (result *pganalyze.ScanResult, err error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Deparse ¶
func Deparse(tree *pganalyze.ParseResult) (output string, err error)
Deparses a given Go parse tree into a SQL statement
func Fingerprint ¶
Fingerprint - Fingerprint the passed SQL statement to a hex string
func FingerprintToUInt64 ¶
FingerprintToUInt64 - Fingerprint the passed SQL statement to a uint64
func HashXXH3_64 ¶
HashXXH3_64 - Helper method to run XXH3 hash function (64-bit variant) on the given bytes, with the specified seed
func Parse ¶
func Parse(input string) (tree *pganalyze.ParseResult, err error)
Parse the given SQL statement into a parse tree (Go struct format)
func ParsePlPgSqlToJSON ¶
ParsePlPgSqlToJSON - Parses the given PL/pgSQL function statement into a parse tree (JSON format)
func ParseToJSON ¶
ParseToJSON - Parses the given SQL statement into a parse tree (JSON format)
Types ¶
This section is empty.
Source Files
Directories