meddler

package module
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 23, 2021 License: BSD-2-Clause Imports: 13 Imported by: 0

README

Meddler Build Status GoDoc Go Report Card

Meddler is a small toolkit to take some of the tedium out of moving data back and forth between SQL queries and structs.

It is not a complete ORM. Meddler is intended to be a lightweight way to add some of the convenience of an ORM while leaving more control in the hands of the programmer.

Package docs are available at:

The package is housed on GitHub, and the README there has more info:

Meddler is currently configured for SQLite, MySQL, and PostgreSQL, but it can be configured for use with other databases. If you use it successfully with a different database, please contact me and I will add it to the list of pre-configured databases.

DANGER

Meddler is still a work in progress, and additional backward-incompatible changes to the API are likely. The most recent change added support for multiple database types and made it easier to switch between them. This is most likely to affect the way you initialize the library to work with your database (see the install section below).

Another recent update is the change to int64 for primary keys. This matches the convention used in database/sql, and is more portable, but it may require some minor changes to existing code.

Install

The usual go get command will put it in your $GOPATH:

go get github.com/russross/meddler

If you are only using one type of database, you should set Default to match your database type, e.g.:

meddler.Default = meddler.PostgreSQL

The default database is MySQL, so you should change it for anything else. To use multiple databases within a single project, or to use a database other than MySQL, PostgreSQL, or SQLite, see below.

Note: If you are using MySQL with the github.com/go-sql-driver/mysql driver, you must set "parseTime=true" in the sql.Open call or the time conversion meddlers will not work.

Why?

These are the features that set meddler apart from similar libraries:

  • It uses standard database/sql types, and does not require special fields in your structs. This lets you use meddler selectively, without having to alter other database code already in your project. After creating meddler, I incorporated it into an existing project, and I was able to convert the code one struct and one query at a time.
  • It leaves query writing to you. It has convenience functions for simple INSERT/UPDATE/SELECT queries by integer primary key, but beyond that it stays out of query writing.
  • It supports on-the-fly data transformations. If you have a map or a slice in your struct, you can instruct meddler to encode/decode using JSON or Gob automatically. If you have time fields, you can have meddler automatically write them into the database as UTC, and convert them to the local time zone on reads. These processors are called “meddlers”, because they meddle with the data instead of passing it through directly.
  • NULL fields in the database can be read as zero values in the struct, and zero values in the struct can be written as NULL values. This is not always the right thing to do, but it is often good enough and is much simpler than most alternatives.
  • It exposes low-level hooks for more complex situations. If you are writing a query that does not map well to the main helper functions, you can still get some help by using the lower-level functions to build your own helpers.

High-level functions

Meddler does not create or alter tables. It just provides a little glue to make it easier to read and write structs as SQL rows. Start by annotating a struct:

type Person struct {
    ID      int       `meddler:"id,pk"`
    Name    string    `meddler:"name"`
    Age     int
    salary  int
    Created time.Time `meddler:"created,localtime"`
    Closed  time.Time `meddler:",localtimez"`
}

Notes about this example:

  • If the optional tag is provided, the first field is the database column name. Note that "Closed" does not provide a column name, so it will default to "Closed". Likewise, if there is no tag, the field name will be used.
  • ID is marked as the primary key. Currently only integer primary keys are supported. This is only relevant to Load, Save, Insert, and Update, a few of the higher-level functions that need to understand primary keys. Meddler assumes that pk fields have an autoincrement mechanism set in the database.
  • Age has a column name of "Age". A tag is only necessary when the column name is not the same as the field name, or when you need to select other options.
  • salary is not an exported field, so meddler does not see it. It will be ignored.
  • Created is marked with "localtime". This means that it will be converted to UTC when being saved, and back to the local time zone when being loaded.
  • Closed has a column name of "Closed", since the tag did not specify anything different. Closed is marked as "localtimez". This has the same properties as "localtime", except that the zero time will be saved in the database as a null column (and null values will be loaded as the zero time value).
  • You can set a default column name mapping by setting meddler.Mapper to a func(s string) string function. For example, meddler.Mapper = meddler.SnakeCase will convert field names to snake_case unless an explict column name is specified.

Meddler provides a few high-level functions (note: DB is an interface that works with a *sql.DB or a *sql.Tx):

  • Load(db DB, table string, dst interface{}, pk int64) error

    This loads a single record by its primary key. For example:

    elt := new(Person)
err := meddler.Load(db, "person", elt, 15)

    db can be a *sql.DB or a *sql.Tx. The table is the name of the table, pk is the primary key value, and dst is a pointer to the struct where it should be stored.

    Note: this call requires that the struct have an integer primary key field marked.

  • Insert(db DB, table string, src interface{}) error

    This inserts a new row into the database. If the struct value has a primary key field, it must be zero (and will be omitted from the insert statement, prompting a default autoincrement value).

    elt := &Person{
    Name: "Alice",
    Age: 22,
    // ...
}
err := meddler.Insert(db, "person", elt)
// elt.ID is updated to the value assigned by the database

  • Update(db DB, table string, src interface{}) error

    This updates an existing row. It must have a primary key, which must be non-zero.

    Note: this call requires that the struct have an integer primary key field marked.

  • Save(db DB, table string, src interface{}) error

    Pick Insert or Update automatically. If there is a non-zero primary key present, it uses Update, otherwise it uses Insert.

    Note: this call requires that the struct have an integer primary key field marked.

  • QueryRow(db DB, dst interface{}, query string, args ...interface) error

    Perform the given query, and scan the single-row result into dst, which must be a pointer to a struct.

    For example:

    elt := new(Person)
err := meddler.QueryRow(db, elt, "select * from person where name = ?", "bob")

  • QueryAll(db DB, dst interface{}, query string, args ...interface) error

    Perform the given query, and scan the results into dst, which must be a pointer to a slice of struct pointers.

    For example:

    var people []*Person
err := meddler.QueryAll(db, &people, "select * from person")

  • Scan(rows *sql.Rows, dst interface{}) error

    Scans a single row of data into a struct, complete with meddling. Can be called repeatedly to walk through all of the rows in a result set. Returns sql.ErrNoRows when there is no more data.

  • ScanRow(rows *sql.Rows, dst interface{}) error

    Similar to Scan, but guarantees that the rows object is closed when it returns. Also returns sql.ErrNoRows if there was no row.

  • ScanAll(rows *sql.Rows, dst interface{}) error

    Expects a pointer to a slice of structs/pointers to structs, and appends as many elements as it finds in the row set. Closes the row set when it is finished. Does not return sql.ErrNoRows on an empty set; instead it just does not add anything to the slice.

Note: all of these functions can also be used as methods on Database objects. When used as package functions, they use the Default Database object, which is MySQL unless you change it.

Meddlers

A meddler is a handler that gets to meddle with a field before it is saved, or when it is loaded. "localtime" and "localtimez" are examples of built-in meddlers. The full list of built-in meddlers includes:

  • identity: the default meddler, which does not do anything

  • localtime: for time.Time and *time.Time fields. Converts the value to UTC on save, and back to the local time zone on loads. To set your local time zone, make sure the TZ environment variable is set when your program is launched, or use something like:

    os.Setenv("TZ", "America/Denver")

    in your initial setup, before you start using time functions.

  • localtimez: same, but only for time.Time, and treats the zero time as a null field (converts both ways)

  • utctime: similar to localtime, but keeps the value in UTC on loads. This ensures that the time is always coverted to UTC on save, which is the sane way to save time values in a database.

  • utctimez: same, but with zero time means null.

  • zeroisnull: for other types where a zero value should be inserted as null, and null values should be read as zero values. Works for integer, unsigned integer, float, complex number, and string types. Note: not for pointer types.

  • json: marshals the field value into JSON when saving, and unmarshals on load.

  • jsongzip: same, but compresses using gzip on save, and uncompresses on load

  • gob: encodes the field value using Gob when saving, and decodes on load.

  • gobgzip: same, but compresses using gzip on save, and uncompresses on load

You can implement custom meddlers as well by implementing the Meddler interface. See the existing implementations in medder.go for examples.

Working with different database types

Meddler can work with multiple database types simultaneously. Database-specific parameters are stored in a Database struct, and structs are pre-defined for MySQL, PostgreSQL, and SQLite.

Instead of relying on the package-level functions, use the method form on the appropriate database type, e.g.:

err = meddler.PostgreSQL.Load(...)

instead of

err = meddler.Load(...)

Or to save typing, define your own abbreviated name for each database:

ms := meddler.MySQL
pg := meddler.PostgreSQL
err = ms.Load(...)
err = pg.QueryAll(...)

If you need a different database, create your own Database instance with the appropriate parameters set. If everything works okay, please contact me with the parameters you used so I can add the new database to the pre-defined list.

Lower-level functions

If you are using more complex queries and just want to reduce the tedium of reading and writing values, there are some lower-level helper functions as well. See the package docs for details, and see the implementations of the higher-level functions to see how they are used.

License

Meddler is distributed under the BSD 2-Clause License. If this license prevents you from using Meddler in your project, please contact me and I will consider adding an additional license that is better suited to your needs.

Copyright © 2013 Russ Ross. All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var Debug = true

Debug enables debug mode, where unused columns and struct fields will be logged

View Source 
var Default = MySQL

Default contains the default database options (which defaults to MySQL)

View Source 
var MySQL = &Database{
	Quote:               "`",
	Placeholder:         "?",
	UseReturningToGetID: false,
}

MySQL contains database specific options for executing queries in a MySQL database

View Source 
var PostgreSQL = &Database{
	Quote:               `"`,
	Placeholder:         "$1",
	UseReturningToGetID: true,
}

PostgreSQL contains database specific options for executing queries in a PostgreSQL database

View Source 
var SQLite = &Database{
	Quote:               `"`,
	Placeholder:         "?",
	UseReturningToGetID: false,
}

SQLite contains database specific options for executing queries in a SQLite database

Functions

func Columns 

func Columns(src interface{}, includePk bool) ([]string, error)

Columns using the Default Database type

func ColumnsQuoted 

func ColumnsQuoted(src interface{}, includePk bool) (string, error)

ColumnsQuoted using the Default Database type

func DriverErr 

func DriverErr(err error) (error, bool)

DriverErr returns the original error as returned by the database driver if the error comes from the driver, with the second value set to true. Otherwise, it returns err itself with false as second value.

func Insert 

func Insert(db DB, table string, src interface{}) error

Insert using the Default Database type

func Load 

func Load(db DB, table string, dst interface{}, pk int64) error

Load using the Default Database type

func LowerCase 

func LowerCase(in string) string

LowerCase returns a lowercased version of the input string

func Placeholders 

func Placeholders(src interface{}, includePk bool) ([]string, error)

Placeholders using the Default Database type

func PlaceholdersString 

func PlaceholdersString(src interface{}, includePk bool) (string, error)

PlaceholdersString using the Default Database type

func PrimaryKey 

func PrimaryKey(src interface{}) (name string, pk int64, err error)

PrimaryKey using the Default Database type

func QueryAll 

func QueryAll(db DB, dst interface{}, query string, args ...interface{}) error

QueryAll using the Default Database type

func QueryRow 

func QueryRow(db DB, dst interface{}, query string, args ...interface{}) error

QueryRow using the Default Database type

func Register 

func Register(name string, m Meddler)

Register sets up a meddler type. Meddlers get a chance to meddle with the data being loaded or saved when a field is annotated with the name of the meddler. The registry is global.

func Save 

func Save(db DB, table string, src interface{}) error

Save using the Default Database type

func Scan 

func Scan(rows *sql.Rows, dst interface{}) error

Scan using the Default Database type

func ScanAll 

func ScanAll(rows *sql.Rows, dst interface{}) error

ScanAll using the Default Database type

func ScanRow 

func ScanRow(rows *sql.Rows, dst interface{}) error

ScanRow using the Default Database type

func SetPrimaryKey 

func SetPrimaryKey(src interface{}, pk int64) error

SetPrimaryKey using the Default Database type

func SnakeCase 

func SnakeCase(in string) string

SnakeCase returns a snake_cased version of the input string

func SomeValues 

func SomeValues(src interface{}, columns []string) ([]interface{}, error)

SomeValues using the Default Database type

func Targets 

func Targets(dst interface{}, columns []string) ([]interface{}, error)

Targets using the Default Database type

func Update 

func Update(db DB, table string, src interface{}) error

Update using the Default Database type

func Values 

func Values(src interface{}, includePk bool) ([]interface{}, error)

Values using the Default Database type

func WriteTargets 

func WriteTargets(dst interface{}, columns []string, targets []interface{}) error

WriteTargets using the Default Database type

Types

type DB 

type DB interface {
	Exec(query string, args ...interface{}) (sql.Result, error)
	Query(query string, args ...interface{}) (*sql.Rows, error)
	QueryRow(query string, args ...interface{}) *sql.Row
}

DB is a generic database interface, matching both *sql.Db and *sql.Tx

type Database 

type Database struct {
	Quote               string // the quote character for table and column names
	Placeholder         string // the placeholder style to use in generated queries
	UseReturningToGetID bool   // use PostgreSQL-style RETURNING "ID" instead of calling sql.Result.LastInsertID
}

Database contains database-specific options. MySQL, PostgreSQL, and SQLite are provided for convenience. Setting Default to any of these lets you use the package-level convenience functions.

func (*Database) Columns 

func (d *Database) Columns(src interface{}, includePk bool) ([]string, error)

Columns returns a list of column names for its input struct.

func (*Database) ColumnsQuoted 

func (d *Database) ColumnsQuoted(src interface{}, includePk bool) (string, error)

ColumnsQuoted is similar to Columns, but it return the list of columns in the form: 

`column1`,`column2`,...

using Quote as the quote character.

func (*Database) Insert 

func (d *Database) Insert(db DB, table string, src interface{}) error

Insert performs an INSERT query for the given record. If the record has a primary key flagged, it must be zero, and it will be set to the newly-allocated primary key value from the database as returned by LastInsertId.

func (*Database) Load 

func (d *Database) Load(db DB, table string, dst interface{}, pk int64) error

Load loads a record using a query for the primary key field. Returns sql.ErrNoRows if not found.

func (*Database) Placeholders 

func (d *Database) Placeholders(src interface{}, includePk bool) ([]string, error)

Placeholders returns a list of placeholders suitable for an INSERT or UPDATE query. If includePk is false, the primary key field is omitted.

func (*Database) PlaceholdersString 

func (d *Database) PlaceholdersString(src interface{}, includePk bool) (string, error)

PlaceholdersString returns a list of placeholders suitable for an INSERT or UPDATE query in string form, e.g.: 

?,?,?,?

if includePk is false, the primary key field is omitted.

func (*Database) PrimaryKey 

func (d *Database) PrimaryKey(src interface{}) (name string, pk int64, err error)

PrimaryKey returns the name and value of the primary key field. The name is the empty string if there is not primary key field marked.

func (*Database) QueryAll 

func (d *Database) QueryAll(db DB, dst interface{}, query string, args ...interface{}) error

QueryAll performs the given query with the given arguments, scanning all results rows into dst.

func (*Database) QueryRow 

func (d *Database) QueryRow(db DB, dst interface{}, query string, args ...interface{}) error

QueryRow performs the given query with the given arguments, scanning a single row of results into dst. Returns sql.ErrNoRows if there was no result row.

func (*Database) Save 

func (d *Database) Save(db DB, table string, src interface{}) error

Save performs an INSERT or an UPDATE, depending on whether or not a primary keys exists and is non-zero.

func (*Database) Scan 

func (d *Database) Scan(rows *sql.Rows, dst interface{}) error

Scan scans a single sql result row into a struct. It leaves rows ready to be scanned again for the next row. Returns sql.ErrNoRows if there is no data to read.

func (*Database) ScanAll 

func (d *Database) ScanAll(rows *sql.Rows, dst interface{}) error

ScanAll scans all sql result rows into a slice of structs. It reads all rows and closes rows when finished. dst should be a pointer to a slice of the appropriate type. The new results will be appended to any existing data in dst.

func (*Database) ScanRow 

func (d *Database) ScanRow(rows *sql.Rows, dst interface{}) error

ScanRow scans a single sql result row into a struct. It reads exactly one result row and closes rows when finished. Returns sql.ErrNoRows if there is no result row.

func (*Database) SetPrimaryKey 

func (d *Database) SetPrimaryKey(src interface{}, pk int64) error

SetPrimaryKey sets the primary key field to the given int value.

func (*Database) SomeValues 

func (d *Database) SomeValues(src interface{}, columns []string) ([]interface{}, error)

SomeValues returns a list of PreWrite processed values suitable for use in an INSERT or UPDATE query. The columns used are the same ones (in the same order) as specified in the columns argument.

func (*Database) Targets 

func (d *Database) Targets(dst interface{}, columns []string) ([]interface{}, error)

Targets returns a list of values suitable for handing to a Scan function in the sql package, complete with meddling. After the Scan is performed, the same values should be handed to WriteTargets to finalize the values and record them in the struct.

func (*Database) Update 

func (d *Database) Update(db DB, table string, src interface{}) error

Update performs and UPDATE query for the given record. The record must have an integer primary key field that is non-zero, and it will be used to select the database row that gets updated.

func (*Database) Values 

func (d *Database) Values(src interface{}, includePk bool) ([]interface{}, error)

Values returns a list of PreWrite processed values suitable for use in an INSERT or UPDATE query. If includePk is false, the primary key field is omitted. The columns used are the same ones (in the same order) as returned by Columns.

func (*Database) WriteTargets 

func (d *Database) WriteTargets(dst interface{}, columns []string, targets []interface{}) error

WriteTargets post-processes values with meddlers after a Scan from the sql package has been performed. The list of targets is normally produced by Targets.

type GobMeddler 

type GobMeddler bool

GobMeddler encodes or decodes the field value to or from gob

func (GobMeddler) PostRead 

func (zip GobMeddler) PostRead(fieldAddr, scanTarget interface{}) error

PostRead is called after a Scan operation for fields that have the GobMeddler

func (GobMeddler) PreRead 

func (zip GobMeddler) PreRead(fieldAddr interface{}) (scanTarget interface{}, err error)

PreRead is called before a Scan operation for fields that have the GobMeddler

func (GobMeddler) PreWrite 

func (zip GobMeddler) PreWrite(field interface{}) (saveValue interface{}, err error)

PreWrite is called before an Insert or Update operation for fields that have the GobMeddler

type IdentityMeddler 

type IdentityMeddler bool

IdentityMeddler is the default meddler, and it passes the original value through with no changes.

func (IdentityMeddler) PostRead 

func (elt IdentityMeddler) PostRead(fieldAddr, scanTarget interface{}) error

PostRead is called after a Scan operation for fields that have the IdentityMeddler

func (IdentityMeddler) PreRead 

func (elt IdentityMeddler) PreRead(fieldAddr interface{}) (scanTarget interface{}, err error)

PreRead is called before a Scan operation for fields that have the IdentityMeddler

func (IdentityMeddler) PreWrite 

func (elt IdentityMeddler) PreWrite(field interface{}) (saveValue interface{}, err error)

PreWrite is called before an Insert or Update operation for fields that have the IdentityMeddler

type JSONMeddler 

type JSONMeddler bool

JSONMeddler encodes or decodes the field value to or from JSON

func (JSONMeddler) PostRead 

func (zip JSONMeddler) PostRead(fieldAddr, scanTarget interface{}) error

PostRead is called after a Scan operation for fields that have the JSONMeddler

func (JSONMeddler) PreRead 

func (zip JSONMeddler) PreRead(fieldAddr interface{}) (scanTarget interface{}, err error)

PreRead is called before a Scan operation for fields that have the JSONMeddler

func (JSONMeddler) PreWrite 

func (zip JSONMeddler) PreWrite(field interface{}) (saveValue interface{}, err error)

PreWrite is called before an Insert or Update operation for fields that have the JSONMeddler

type MapperFunc 

type MapperFunc func(in string) string

MapperFunc signature. Argument is field name, return value is database column. 

var Mapper MapperFunc = strings.TrimSpace

Mapper defines the function to transform struct field names into database columns. Default is strings.TrimSpace, basically a no-op

type Meddler 

type Meddler interface {
	// PreRead is called before a Scan operation. It is given a pointer to
	// the raw struct field, and returns the value that will be given to
	// the database driver.
	PreRead(fieldAddr interface{}) (scanTarget interface{}, err error)

	// PostRead is called after a Scan operation. It is given the value returned
	// by PreRead and a pointer to the raw struct field. It is expected to fill
	// in the struct field if the two are different.
	PostRead(fieldAddr interface{}, scanTarget interface{}) error

	// PreWrite is called before an Insert or Update operation. It is given
	// a pointer to the raw struct field, and returns the value that will be
	// given to the database driver.
	PreWrite(field interface{}) (saveValue interface{}, err error)
}

Meddler is the interface for a field meddler. Implementations can be registered to convert struct fields being loaded and saved in the database.

type TimeMeddler 

type TimeMeddler struct {
	ZeroIsNull bool
	Local      bool
}

TimeMeddler provides useful operations on time.Time fields. It can convert the zero time to and from a null column, and it can convert the time zone to UTC on save and to Local on load.

func (TimeMeddler) PostRead 

func (elt TimeMeddler) PostRead(fieldAddr, scanTarget interface{}) error

PostRead is called after a Scan operation for fields that have a TimeMeddler

func (TimeMeddler) PreRead 

func (elt TimeMeddler) PreRead(fieldAddr interface{}) (scanTarget interface{}, err error)

PreRead is called before a Scan operation for fields that have a TimeMeddler

func (TimeMeddler) PreWrite 

func (elt TimeMeddler) PreWrite(field interface{}) (saveValue interface{}, err error)

PreWrite is called before an Insert or Update operation for fields that have a TimeMeddler

type ZeroIsNullMeddler 

type ZeroIsNullMeddler bool

ZeroIsNullMeddler converts zero value fields (integers both signed and unsigned, floats, complex numbers, and strings) to and from null database columns.

func (ZeroIsNullMeddler) PostRead 

func (elt ZeroIsNullMeddler) PostRead(fieldAddr, scanTarget interface{}) error

PostRead is called after a Scan operation for fields that have the ZeroIsNullMeddler

func (ZeroIsNullMeddler) PreRead 

func (elt ZeroIsNullMeddler) PreRead(fieldAddr interface{}) (scanTarget interface{}, err error)

PreRead is called before a Scan operation for fields that have the ZeroIsNullMeddler

func (ZeroIsNullMeddler) PreWrite 

func (elt ZeroIsNullMeddler) PreWrite(field interface{}) (saveValue interface{}, err error)

PreWrite is called before an Insert or Update operation for fields that have the ZeroIsNullMeddler

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.