pgxtypefaster

README

Faster pgx types

This repository contains types for the pgx Go Postgres driver that are faster, but have incompatible APIs. It currently contains two variants of Hstore. This Hstore implementation uses a single string for all key/value pairs, instead of separate strings. This makes it about ~40% faster when parsing values from Postgres. However, it can have a larger memory footprint, if an application keeps pointers to a subset of the keys/values.

• Hstore: This is a map[string]pgtype.Text instead of map[string]*string as used by pgtype.Hstore. Since this removes pointers, it requires one fewer allocation per Hstore, and is about ~5% faster than HstoreCompat when parsing. However, it appears to allocate a bit more total memory, I think because the map itself is larger. It is not API compatible with pgtype.Hstore.
• HstoreCompat: This is API compatible with pgx/pgtype.Hstore because it uses a map[string]*string, but is about ~5% slower.

This code has the same LICENSE as the upstream repository since it basically copied the code then edited it. See the original upstream pull request discussion for details where it was decided not to make this change upstream.

The only tests are two fuzz test, since porting the tests was going to be challenging. The fuzz tests included here should provide excellent coverage, without too much code.

Using this type in your program

To get the best performance, you want to use the binary protocol. To do that, you must register this type:

TODO document

Benchmark results

Results from this repository's benchmark, run with go test . -bench=. -benchtime=2s:

ARM M1 Max (Macbook Pro 2021)

BenchmarkHstoreScan/pgxtypefaster/databasesql.Scan-10         	  237426	     10002 ns/op	   13976 B/op	      34 allocs/op
BenchmarkHstoreScan/fastercompat/databasesql.Scan-10          	  234495	     10189 ns/op	   11960 B/op	      44 allocs/op
BenchmarkHstoreScan/pgtype/databasesql.Scan-10                	  168888	     14179 ns/op	   20592 B/op	     340 allocs/op

BenchmarkHstoreScan/pgxfastertype/text-10                     	  218518	     11130 ns/op	   23087 B/op	      34 allocs/op
BenchmarkHstoreScan/fastercompat/text-10                      	  212636	     11264 ns/op	   21072 B/op	      44 allocs/op
BenchmarkHstoreScan/pgtype/text-10                            	  158030	     15043 ns/op	   29696 B/op	     339 allocs/op

BenchmarkHstoreScan/pgxfastertype/binary-10                   	  341541	      7127 ns/op	   23240 B/op	      31 allocs/op
BenchmarkHstoreScan/fastercompat/binary-10                    	  315151	      7467 ns/op	   21224 B/op	      41 allocs/op
BenchmarkHstoreScan/pgtype/binary-10                          	  229638	     10381 ns/op	   20368 B/op	     316 allocs/op

Documentation

Overview

Package pgxtypefaster provides types for use with the pgx Postgres driver that are faster, but not completely API compatible.

Package pgxtypefaster provides types for use with the pgx Postgres driver that are faster, but not completely API compatible.

Index

• Variables
• func NewText(s string) pgtype.Text
• func RegisterHstore(ctx context.Context, conn *pgx.Conn) error
• func RegisterHstoreCompat(ctx context.Context, conn *pgx.Conn) error
• type Hstore
  • func PGXToFasterHstore(m map[string]*string) Hstore
  • func (h Hstore) HstoreValue() (Hstore, error)
  • func (h *Hstore) Scan(src any) error
  • func (h *Hstore) ScanHstore(v Hstore) error
  • func (h Hstore) Value() (driver.Value, error)
• type HstoreCodec
  • func (c HstoreCodec) DecodeDatabaseSQLValue(m *pgtype.Map, oid uint32, format int16, src []byte) (driver.Value, error)
  • func (c HstoreCodec) DecodeValue(m *pgtype.Map, oid uint32, format int16, src []byte) (any, error)
  • func (HstoreCodec) FormatSupported(format int16) bool
  • func (HstoreCodec) PlanEncode(m *pgtype.Map, oid uint32, format int16, value any) pgtype.EncodePlan
  • func (HstoreCodec) PlanScan(m *pgtype.Map, oid uint32, format int16, target any) pgtype.ScanPlan
  • func (HstoreCodec) PreferredFormat() int16
• type HstoreCompat
  • func (h HstoreCompat) HstoreCompatValue() (HstoreCompat, error)
  • func (h *HstoreCompat) Scan(src any) error
  • func (h *HstoreCompat) ScanHstoreCompat(v HstoreCompat) error
  • func (h HstoreCompat) Value() (driver.Value, error)
• type HstoreCompatCodec
  • func (c HstoreCompatCodec) DecodeDatabaseSQLValue(m *pgtype.Map, oid uint32, format int16, src []byte) (driver.Value, error)
  • func (c HstoreCompatCodec) DecodeValue(m *pgtype.Map, oid uint32, format int16, src []byte) (any, error)
  • func (HstoreCompatCodec) FormatSupported(format int16) bool
  • func (HstoreCompatCodec) PlanEncode(m *pgtype.Map, oid uint32, format int16, value any) pgtype.EncodePlan
  • func (HstoreCompatCodec) PlanScan(m *pgtype.Map, oid uint32, format int16, target any) pgtype.ScanPlan
  • func (HstoreCompatCodec) PreferredFormat() int16
• type HstoreCompatScanner
• type HstoreCompatValuer
• type HstoreScanner
• type HstoreValuer

Variables

var ErrHstoreDoesNotExist = errors.New("postgres type hstore does not exist (the extension may not be loaded)")

Functions

func NewText

func NewText(s string) pgtype.Text

NewText converts a string to a non-NULL pgtype.Text.

func RegisterHstore

func RegisterHstore(ctx context.Context, conn *pgx.Conn) error

RegisterHstore registers the Hstore type with conn's default type map. It queries the database for the Hstore OID to be able to register it.

func RegisterHstoreCompat

func RegisterHstoreCompat(ctx context.Context, conn *pgx.Conn) error

RegisterHstoreCompat registers the HstoreCompat type with conn's default type map. It queries the database for the Hstore OID to be able to register it.

Types

type Hstore

type Hstore map[string]pgtype.Text

Hstore represents an hstore column that can be null or have null values associated with its keys.

func PGXToFasterHstore

func PGXToFasterHstore(m map[string]*string) Hstore

PGXToFasterHstore copies a pgtype.Hstore into a pgxtypefaster.Hstore.

func (Hstore) HstoreValue

func (h Hstore) HstoreValue() (Hstore, error)

func (*Hstore) Scan

func (h *Hstore) Scan(src any) error

Scan implements the database/sql Scanner interface.

func (*Hstore) ScanHstore

func (h *Hstore) ScanHstore(v Hstore) error

func (Hstore) Value

func (h Hstore) Value() (driver.Value, error)

Value implements the database/sql/driver Valuer interface.

type HstoreCodec

type HstoreCodec struct{}

func (HstoreCodec) DecodeDatabaseSQLValue

func (c HstoreCodec) DecodeDatabaseSQLValue(m *pgtype.Map, oid uint32, format int16, src []byte) (driver.Value, error)

func (HstoreCodec) DecodeValue

func (c HstoreCodec) DecodeValue(m *pgtype.Map, oid uint32, format int16, src []byte) (any, error)

func (HstoreCodec) FormatSupported

func (HstoreCodec) FormatSupported(format int16) bool

func (HstoreCodec) PlanEncode

func (HstoreCodec) PlanEncode(m *pgtype.Map, oid uint32, format int16, value any) pgtype.EncodePlan

func (HstoreCodec) PlanScan

func (HstoreCodec) PlanScan(m *pgtype.Map, oid uint32, format int16, target any) pgtype.ScanPlan

func (HstoreCodec) PreferredFormat

func (HstoreCodec) PreferredFormat() int16

type HstoreCompat

type HstoreCompat map[string]*string

HstoreCompat represents an hstore column that can be null or have null values associated with its keys.

func (HstoreCompat) HstoreCompatValue

func (h HstoreCompat) HstoreCompatValue() (HstoreCompat, error)

func (*HstoreCompat) Scan

func (h *HstoreCompat) Scan(src any) error

Scan implements the database/sql Scanner interface.

func (*HstoreCompat) ScanHstoreCompat

func (h *HstoreCompat) ScanHstoreCompat(v HstoreCompat) error

func (HstoreCompat) Value

func (h HstoreCompat) Value() (driver.Value, error)

Value implements the database/sql/driver Valuer interface.

type HstoreCompatCodec

type HstoreCompatCodec struct{}

func (HstoreCompatCodec) DecodeDatabaseSQLValue

func (c HstoreCompatCodec) DecodeDatabaseSQLValue(m *pgtype.Map, oid uint32, format int16, src []byte) (driver.Value, error)

func (HstoreCompatCodec) DecodeValue

func (c HstoreCompatCodec) DecodeValue(m *pgtype.Map, oid uint32, format int16, src []byte) (any, error)

func (HstoreCompatCodec) FormatSupported

func (HstoreCompatCodec) FormatSupported(format int16) bool

func (HstoreCompatCodec) PlanEncode

func (HstoreCompatCodec) PlanEncode(m *pgtype.Map, oid uint32, format int16, value any) pgtype.EncodePlan

func (HstoreCompatCodec) PlanScan

func (HstoreCompatCodec) PlanScan(m *pgtype.Map, oid uint32, format int16, target any) pgtype.ScanPlan

func (HstoreCompatCodec) PreferredFormat

func (HstoreCompatCodec) PreferredFormat() int16

type HstoreCompatScanner

type HstoreCompatScanner interface {
	ScanHstoreCompat(v HstoreCompat) error
}

type HstoreCompatValuer

type HstoreCompatValuer interface {
	HstoreCompatValue() (HstoreCompat, error)
}

type HstoreScanner

type HstoreScanner interface {
	ScanHstore(v Hstore) error
}

type HstoreValuer

type HstoreValuer interface {
	HstoreValue() (Hstore, error)
}