scan

package module
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 18, 2024 License: MIT Imports: 10 Imported by: 12

README

Scan

Test Status GitHub go.mod Go version Go Reference Go Report Card GitHub tag (latest SemVer) Coverage Status

Scan provides the ability to use database/sql/rows to scan datasets directly to any defined structure.

Reference

  • Standard library scan package. For use with database/sql. Link
  • PGX library scan package. For use with github.com/jackc/pgx/v5. Link
  • Base scan package. For use with any implementation of scan.Queryer. Link

Using with database/sql

package main

import (
    "context"
    "database/sql"

    "github.com/stephenafamo/scan"
    "github.com/stephenafamo/scan/stdscan"
)

type User struct {
    ID    int
    Name  string
    Email string
    Age   int
}

func main() {
    ctx := context.Background()
    db, _ := sql.Open("postgres", "example-connection-url")

    // count: 5
    count, _ := stdscan.One(ctx, db, scan.SingleColumnMapper[int], "SELECT COUNT(*) FROM users")
    // []int{1, 2, 3, 4, 5}
    userIDs, _ := stdscan.All(ctx, db, scan.SingleColumnMapper[int], "SELECT id FROM users")
    // []User{...}
    users, _ := stdscan.All(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)
}

func collectIDandEmail(_ context.Context, c cols) any {
    return func(v *Values) (int, string, error) {
        return Value[int](v, "id"), Value[string](v, "email"), nil
    }
}

And many more!!

Using with pgx

ctx := context.Background()
db, _ := pgxpool.New(ctx, "example-connection-url")

// []User{...}
users, _ := pgxscan.All(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)

Using with other DB packages

Instead of github.com/stephenafamo/scan/stdscan, use the base package github.com/stephenafam/scan which only needs an executor that implements the right interface.
Both stdscan and pgxscan are based on this.

How it works

Scanning Functions
One()

Use One() to scan and return a single row.

// User{...}
user, _ := stdscan.One(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)
All()

Use All() to scan and return all rows.

// []User{...}
users, _ := stdscan.All(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)
Each()

Use Each() to iterate over the rows of a query using range.

It works with the range-over-func syntax.

// []User{...}
users, _ := stdscan.All(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)
for user, err := range scan.Each(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`) {
    if err != nil {
        return err
    }
    // do something with user
}
Cursor()

Use Cursor() to scan each row on demand. This is useful when retrieving large results.

c, _ := stdscan.Cursor(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)
defer c.Close()

for c.Next() {
    // User{...}
    user := c.Get()
}
Mappers

Each of these functions takes a Mapper to indicate how each row should be scanned.
The Mapper has the signature:

type Mapper[T any] func(context.Context, cols) (before BeforeFunc, after func(any) (T, error))

type BeforeFunc = func(*Row) (link any, err error)

A mapper returns 2 functions

  • before: This is called before scanning the row. The mapper should schedule scans using the ScheduleScan or ScheduleScanx methods of the Row. The return value of the before function is passed to the after function after scanning values from the database.
  • after: This is called after the scan operation. The mapper should then covert the link value back to the desired concrete type.

There are some builtin mappers for common cases:

ColumnMapper[T any](name string)

Maps the value of a single column to the given type. The name of the column must be specified

// []string{"user1@example.com", "user2@example.com", "user3@example.com", ...}
emails, _ := stdscan.All(ctx, db, scan.ColumnMapper[string]("email"), `SELECT id, name, email FROM users`)
SingleColumnMapper[T any]

For queries that return only one column. Since only one column is returned, there is no need to specify the column name.
This is why it throws an error if the query returns more than one column.

// []string{"user1@example.com", "user2@example.com", "user3@example.com", ...}
emails, _ := stdscan.All(ctx, db, scan.SingleColumnMapper[string], `SELECT email FROM users`)
SliceMapper[T any]

Maps a row into a slice of values []T. Unless all the columns are of the same type, it will likely be used to map the row to []any.

// [][]any{
//    []any{1, "John Doe", "john@example.com"},
//    []any{2, "Jane Doe", "jane@example.com"},
//    ...
// }
users, _ := stdscan.All(ctx, db, scan.SliceMapper[any], `SELECT id, name, email FROM users`)
MapMapper[T any]

Maps a row into a map of values map[string]T. The key of the map is the column names. Unless all columns are of the same type, it will likely be used to map to map[string]any.

// []map[string]any{
//    map[string]any{"id": 1, "name": John Doe", "email": "john@example.com"},
//    map[string]any{"id": 2, "name": Jane Doe", "email": "jane@example.com"},
//    ...
// }
users, _ := stdscan.All(ctx, db, scan.MapMapper[any], `SELECT id, name, email FROM users`)
StructMapper[T any](...MappingOption)

This is the most advanced mapper. Scans column values into the fields of the struct.

type User struct {
    ID    int    `db:"id"`
    Name  string `db:"name"`
    Email string `db:"email"`
    Age   int    `db:"age"`
}

// []User{...}
users, _ := stdscan.All(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)

The default behaviour of StructMapper is often good enough. For more advanced use cases, some options can be passed to the StructMapper.

  • WithStructTagPrefix: Use this when every column from the database has a prefix.

    users, _ := stdscan.All(ctx, db, scan.StructMapper[User](scan.WithStructTagPrefix("user-")),
    `SELECT id AS "user-id", name AS "user-name" FROM users`,
)

  • WithRowValidator: If the StructMapper has a row validator, the values will be sent to it before scanning. If the row is invalid (i.e. it returns false), then scanning is skipped and the zero value of the row-type is returned.

  • WithTypeConverter: If the StructMapper has a type converter, all fields of the struct are converted to a new type using the ConverType method. After scanning, the values are restored into the struct using the OriginalValue method.

CustomStructMapper[T any](MapperSource, ...MappingSourceOption)

Uses a custom struct maping source which should have been created with NewStructMapperSource.

This works the same way as StructMapper, but instead of using the default mapping source, it uses a custom one.

In the example below, we want to use a scan as the struct tag key instead of db

type User struct {
    ID    int    `scan:"id"`
    Name  string `scan:"name"`
    Email string `scan:"email"`
    Age   int    `scan:"age"`
}

src, _ := NewStructMapperSource(scan.WithStructTagKey("scan"))
// []User{...}
users, _ := stdscan.All(ctx, db, scan.StructMapper[User](), `SELECT id, name, email, age FROM users`)

These are the options that can be passed to NewStructMapperSource:

  • WithStructTagKey: Change the struct tag used to map columns to struct fields. Default: db
  • WithColumnSeparator: Change the separator for column names of nested struct fields. Default: .
  • WithFieldNameMapper: Change how Struct field names are mapped to column names when there are no struct tags. Default: snake_case (i.e. CreatedAt is mapped to created_at).
  • WithScannableTypes: Pass a list of interfaces that if implemented, can be scanned by the executor. This means that a field with this type is treated as a single value and will not check the nested fields. Default: *sql.Scanner.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var CtxKeyAllowUnknownColumns contextKey = "allow unknown columns"

CtxKeyAllowUnknownColumns makes it possible to allow unknown columns using the context

Functions

func All 

func All[T any](ctx context.Context, exec Queryer, m Mapper[T], query string, args ...any) ([]T, error)

All scans all rows from the query and returns a slice []T of all rows using a Queryer

func AllFromRows added in v0.2.0 

func AllFromRows[T any](ctx context.Context, m Mapper[T], rows Rows) ([]T, error)

AllFromRows scans all rows from the given Rows and returns a slice []T of all rows using a Queryer

func ColumnMapper 

func ColumnMapper[T any](name string) func(ctx context.Context, c cols) (before func(*Row) (any, error), after func(any) (T, error))

Map a column by name.

func Each added in v0.6.0 

func Each[T any](ctx context.Context, exec Queryer, m Mapper[T], query string, args ...any) func(func(T, error) bool)

Each returns a function that can be used to iterate over the rows of a query this function works with range-over-func so it is possible to do 

for val, err := range scan.Each(ctx, exec, m, query, args...) {
    if err != nil {
        return err
    }
    // do something with val
}

func ErrorMapper added in v0.3.0 

func ErrorMapper[T any](err error, meta ...string) (func(*Row) (any, error), func(any) (T, error))

The generator function does not return an error itself to make it less cumbersome so we return a function that only returns an error instead This function makes it easy to return this error

func MapMapper 

func MapMapper[T any](ctx context.Context, c cols) (before func(*Row) (any, error), after func(any) (map[string]T, error))

Maps all rows into map[string]T Most likely used with interface{} to get a map[string]interface{}

func One 

func One[T any](ctx context.Context, exec Queryer, m Mapper[T], query string, args ...any) (T, error)

One scans a single row from the query and maps it to T using a Queryer

func OneFromRows added in v0.2.0 

func OneFromRows[T any](ctx context.Context, m Mapper[T], rows Rows) (T, error)

OneFromRows scans a single row from the given Rows result and maps it to T using a Queryer

func SingleColumnMapper 

func SingleColumnMapper[T any](ctx context.Context, c cols) (before func(*Row) (any, error), after func(any) (T, error))

For queries that return only one column throws an error if there is more than one column

func SliceMapper 

func SliceMapper[T any](ctx context.Context, c cols) (before func(*Row) (any, error), after func(any) ([]T, error))

Maps each row into []any in the order

Types

type AfterMod added in v0.3.0 

type AfterMod = func(link any, retrieved any) error

AfterMod receives both the link of the MapperMod and the retrieved value from the original mapper

type BeforeFunc added in v0.3.0 

type BeforeFunc = func(*Row) (link any, err error)

BeforeFunc is returned by a mapper and is called before a row is scanned Scans should be scheduled with either the *Row.ScheduleScan or *Row.ScheduleScanx methods

type ICursor 

type ICursor[T any] interface {
	// Close the underlying rows
	Close() error
	// Prepare the next row
	Next() bool
	// Get the values of the current row
	Get() (T, error)
	// Return any error with the underlying rows
	Err() error
}

func Cursor 

func Cursor[T any](ctx context.Context, exec Queryer, m Mapper[T], query string, args ...any) (ICursor[T], error)

Cursor runs a query and returns a cursor that works similar to *sql.Rows

func CursorFromRows added in v0.2.0 

func CursorFromRows[T any](ctx context.Context, m Mapper[T], rows Rows) (ICursor[T], error)

CursorFromRows returns a cursor from Rows that works similar to *sql.Rows

type Mapper 

type Mapper[T any] func(context.Context, cols) (before BeforeFunc, after func(any) (T, error))

Mapper is a function that return the mapping functions. Any expensive operation, like reflection should be done outside the returned function. It is called with the columns from the query to get the mapping functions which is then used to map every row.

The Mapper does not return an error itself to make it less cumbersome It is recommended to instead return a function that returns an error the ErrorMapper is provider for this

func CustomStructMapper 

func CustomStructMapper[T any](src StructMapperSource, optMod ...MappingOption) Mapper[T]

Uses reflection to create a mapping function for a struct type using with custom options

func Mod 

func Mod[T any](m Mapper[T], mods ...MapperMod) Mapper[T]

Mod converts an existing mapper into a new mapper with [MapperMod]s

func StructMapper 

func StructMapper[T any](opts ...MappingOption) Mapper[T]

Uses reflection to create a mapping function for a struct type using the default options

type MapperMod 

type MapperMod = func(context.Context, cols) (BeforeFunc, AfterMod)

MapperMod is a function that can be used to convert an existing mapper into a new mapper using Mod

type MappingError 

type MappingError struct {
	// contains filtered or unexported fields
}

MappingError wraps another error and holds some additional metadata

func (*MappingError) Error 

func (m *MappingError) Error() string

Error implements the error interface

func (*MappingError) Unwrap 

func (m *MappingError) Unwrap() error

Unwrap returns the wrapped error

type MappingOption 

type MappingOption func(*mappingOptions)

MappingeOption is a function type that changes how the mapper is generated

func WithMapperMods added in v0.4.0 

func WithMapperMods(mods ...MapperMod) MappingOption

WithMapperMods accepts mods used to modify the mapper

func WithRowValidator 

func WithRowValidator(rv RowValidator) MappingOption

WithRowValidator sets the RowValidator for the struct mapper after scanning all values in a row, they are passed to the RowValidator if it returns false, the zero value for that row is returned

func WithStructTagPrefix 

func WithStructTagPrefix(prefix string) MappingOption

WithStructTagPrefix should be used when every column from the database has a prefix.

func WithTypeConverter 

func WithTypeConverter(tc TypeConverter) MappingOption

TypeConverter sets the TypeConverter for the struct mapper it is called to modify the type of a column and get the original value back

type MappingSourceOption 

type MappingSourceOption func(src *mapperSourceImpl) error

MappingSourceOption are options to modify how a struct's mappings are interpreted

func WithColumnSeparator 

func WithColumnSeparator(separator string) MappingSourceOption

WithColumnSeparator allows to use a custom separator character for column name when combining nested structs. The default separator is "." character.

func WithFieldNameMapper 

func WithFieldNameMapper(mapperFn func(string) string) MappingSourceOption

WithFieldNameMapper allows to use a custom function to map field name to column names. The default function maps fields names to "snake_case"

func WithScannableTypes 

func WithScannableTypes(scannableTypes ...any) MappingSourceOption

WithScannableTypes specifies a list of interfaces that underlying database library can scan into. In case the destination type passed to scan implements one of those interfaces, scan will handle it as primitive type case i.e. simply pass the destination to the database library. Instead of attempting to map database columns to destination struct fields or map keys. In order for reflection to capture the interface type, you must pass it by pointer.

For example your database library defines a scanner interface like this: 

type Scanner interface {
    Scan(...) error
}

You can pass it to scan this way: scan.WithScannableTypes((*Scanner)(nil)).

func WithStructTagKey 

func WithStructTagKey(tagKey string) MappingSourceOption

WithStructTagKey allows to use a custom struct tag key. The default tag key is `db`.

type Queryer 

type Queryer interface {
	QueryContext(ctx context.Context, query string, args ...any) (Rows, error)
}

Queryer is the main interface used in this package it is expected to run the query and args and return a set of Rows

func Debug 

func Debug(q Queryer, w io.Writer) Queryer

type Row 

type Row struct {
	// contains filtered or unexported fields
}

Row represents a single row from the query and is passed to the BeforeFunc when sent to a mapper's before function, scans should be scheduled with either the [ScheduleScan] or [ScheduleScanx] methods

func (*Row) ScheduleScan added in v0.3.0 

func (r *Row) ScheduleScan(colName string, val any)

ScheduleScan schedules a scan for the column name into the given value val should be a pointer

func (*Row) ScheduleScanx added in v0.3.0 

func (r *Row) ScheduleScanx(colName string, val reflect.Value)

ScheduleScanx schedules a scan for the column name into the given reflect.Value val.Kind() should be reflect.Pointer

type RowValidator 

type RowValidator = func(cols []string, vals []reflect.Value) bool

RowValidator is called with pointer to all the values from a row to determine if the row is valid if it is not, the zero type for that row is returned

type Rows 

type Rows interface {
	Scan(...any) error
	Columns() ([]string, error)
	Next() bool
	Close() error
	Err() error
}

Rows is an interface that is expected to be returned as the result of a query

type StructMapperSource 

type StructMapperSource interface {
	// contains filtered or unexported methods
}

func NewStructMapperSource 

func NewStructMapperSource(opts ...MappingSourceOption) (StructMapperSource, error)

NewStructMapperSource creates a new Mapping object with provided list of options.

type TypeConverter 

type TypeConverter interface {
	// TypeToDestination is called with the expected type of the column
	// it is expected to return a pointer to the desired value to scan into
	// the returned destination is directly scanned into
	TypeToDestination(reflect.Type) reflect.Value

	// ValueFromDestination retrieves the original value from the destination
	// the returned value is set back to the appropriate struct field
	ValueFromDestination(reflect.Value) reflect.Value
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.