gorp

package module
v0.0.0-...-af8337d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 18, 2012 License: MIT Imports: 7 Imported by: 0

README

Go Relational Persistence

I hesitate to call gorp an ORM. Go doesn't really have objects, at least not in the classic Smalltalk/Java sense. There goes the "O". gorp doesn't know anything about the relationships between your structs (at least not yet). So the "R" is questionable too (but I use it in the name because, well, it seemed more clever).

The "M" is alive and well. Given some Go structs and a database, gorp should remove a fair amount of boilerplate busy-work from your code.

I hope that gorp saves you time, minimizes the drudgery of getting data in and out of your database, and helps your code focus on algorithms, not infrastructure.

Database Drivers

gorp uses the Go 1 database/sql package. A full list of compliant drivers is available here:

http://code.google.com/p/go-wiki/wiki/SQLDrivers

Sadly, SQL databases differ on various issues. gorp provides a Dialect interface that should be implemented per database vendor. Dialects are provided for:

  • MySQL
  • PostgreSQL
  • sqlite3

Each of these three databases pass the test suite. See gorp_test.go for example DSNs for these three databases.

Features

  • Bind struct fields to table columns via API or tag
  • Support for transactions
  • Forward engineer db schema from structs (great for unit tests)
  • Pre/post insert/update/delete hooks
  • Automatically generate insert/update/delete statements for a struct
  • Automatic binding of auto increment PKs back to struct after insert
  • Delete by primary key(s)
  • Select by primary key(s)
  • Optional trace sql logging
  • Bind arbitrary SQL queries to a struct
  • Optional optimistic locking using a version column (for update/deletes)

TODO

  • Test with more database/sql drivers, such as Postgresql

Installation

# install the library:
go get github.com/coopernurse/gorp

// use in your .go code:
import (
    "github.com/coopernurse/gorp"
)

Running the tests

The included tests are written against MySQL at the moment. I need to look into adding additional drivers to the test suite, but for now you can clone the repo and setup an environment variable before running "go test"

# Set env variable with dsn using mymysql format.  From the mymysql docs, 
# the format can be of 3 types:
#
#    DBNAME/USER/PASSWD
#    unix:SOCKPATH*DBNAME/USER/PASSWD
#    tcp:ADDR*DBNAME/USER/PASSWD
#
# for example, on my box I use:
export GORP_TEST_DSN=gomysql_test/gomysql_test/abc123

# run the tests
go test

# run the tests and benchmarks
go test -bench="Bench" -benchtime 10

Performance

gorp uses reflection to construct SQL queries and bind parameters. See the BenchmarkNativeCrud vs BenchmarkGorpCrud in gorp_test.go for a simple perf test. On my MacBook Pro gorp is about 2-3% slower than hand written SQL.

Examples

First define some types:

type Invoice struct {
    Id       int64
    Created  int64
    Updated  int64
    Memo     string
    PersonId int64
}

type Person struct {
    Id      int64    
    Created int64
    Updated int64
    FName   string
    LName   string
}

// Example of using tags to alias fields to column names
// The 'db' value is the column name
//
// This is equivalent to using the ColMap.Rename() function:
//
//   table := dbmap.AddTableWithName(Product{}, "product")
//   table.ColMap("Id").Rename("product_id")
//   table.ColMap("Price").Rename("unit_price")
//
type Product struct {
    Id      int64     `db:"product_id"`
    Price   int64     `db:"unit_price"`
}

Then create a mapper, typically you'd do this one time at app startup:

// connect to db using standard Go database/sql API
// use whatever database/sql driver you wish
db, err := sql.Open("mysql", "myuser:mypassword@localhost:3306/dbname")

// construct a gorp DbMap
dbmap := &gorp.DbMap{Db: db, Dialect: gorp.MySQLDialect{"InnoDB", "UTF8"}}

// register the structs you wish to use with gorp
// you can also use the shorter dbmap.AddTable() if you 
// don't want to override the table name
//
// SetKeys(true) means we have a auto increment primary key, which
// will get automatically bound to your struct post-insert
//
t1 := dbmap.AddTableWithName(Invoice{}, "invoice_test").SetKeys(true, "Id")
t2 := dbmap.AddTableWithName(Person{}, "person_test").SetKeys(true, "Id")

Automatically create / drop registered tables. Great for unit tests:

// create all registered tables
dbmap.CreateTables()

// drop
dbmap.DropTables()

Optionally you can pass in a log.Logger to trace all SQL statements:

// Will log all SQL statements + args as they are run
// The first arg is a string prefix to prepend to all log messages
dbmap.TraceOn("[gorp]", log.New(os.Stdout, "myapp:", log.Lmicroseconds)) 

// Turn off tracing
dbmap.TraceOff()

Then save some data:

// Must declare as pointers so optional callback hooks
// can operate on your data, not copies
inv1 := &Invoice{0, 100, 200, "first order", 0}
inv2 := &Invoice{0, 100, 200, "second order", 0}

// Insert your rows
err := dbmap.Insert(inv1, inv2)

// Because we called SetKeys(true) on Invoice, the Id field
// will be populated after the Insert() automatically
fmt.Printf("inv1.Id=%d  inv2.Id=%d\n", inv1.Id, inv2.Id)

You can execute raw SQL if you wish. Particularly good for batch operations.

res, err := dbmap.Exec("delete from invoice_test where PersonId=?", 10)

Want to do joins? Just write the SQL and the struct. gorp will bind them:

// Define a type for your join
// It *must* contain all the columns in your SELECT statement
//
// The names here should match the aliased column names you specify
// in your SQL - no additional binding work required.  simple.
//
type InvoicePersonView struct {
    InvoiceId   int64
    PersonId    int64
    Memo        string
    FName       string
}

// Create some rows
p1 := &Person{0, 0, 0, "bob", "smith"}
dbmap.Insert(p1)

// notice how we can wire up p1.Id to the invoice easily
inv1 := &Invoice{0, 0, 0, "xmas order", p1.Id}
dbmap.Insert(inv1)

// Run your query
query := "select i.Id InvoiceId, p.Id PersonId, i.Memo, p.FName " +
	"from invoice_test i, person_test p " +
	"where i.PersonId = p.Id"
list, err := dbmap.Select(InvoicePersonView{}, query)

// this should test true
expected := &InvoicePersonView{inv1.Id, p1.Id, inv1.Memo, p1.FName}
if reflect.DeepEqual(list[0], expected) {
    fmt.Println("Woot! My join worked!")
}

You can also batch operations into a transaction:

func InsertInv(dbmap *DbMap, inv *Invoice, per *Person) error {
    // Start a new transaction
    trans := dbmap.Begin()

    trans.Insert(per)
    inv.PersonId = per.Id
    trans.Insert(inv)

    // if the commit is successful, a nil error is returned
    return trans.Commit()
}

Use hooks to update data before/after saving to the db. Good for timestamps:

// implement the PreInsert and PreUpdate hooks
func (i *Invoice) PreInsert(s gorp.SqlExecutor) error {
    i.Created = time.Now().UnixNano()
    i.Updated = i.Created
    return nil
}

func (i *Invoice) PreUpdate(s gorp.SqlExecutor) error {
    i.Updated = time.Now().UnixNano()
    return nil
}

// You can use the SqlExecutor to cascade additional SQL
// Take care to avoid cycles. gorp won't prevent them.
//
// Here's an example of a cascading delete
//
func (p *Person) PreDelete(s gorp.SqlExecutor) error {
    query := "delete from invoice_test where PersonId=?"
    err := s.Exec(query, p.Id); if err != nil {
        return err
    }
    return nil
}

Full list of hooks that you can implement:

PostGet
PreInsert
PostInsert
PreUpdate
PostUpdate
PreDelete
PostDelete

All have the same signature.  for example:

func (p *MyStruct) PostUpdate(s gorp.SqlExecutor) error

Optimistic locking (similar to JPA)

// Version is an auto-incremented number, managed by gorp
// If this property is present on your struct, update
// operations will be constrained
//
// For example, say we defined Person as:

type Person struct {
    Id       int64
    Created  int64
    Updated  int64
    FName    string
    LName    string
    
    // automatically used as the Version col
    // use table.SetVersionCol("columnName") to map a different
    // struct field as the version field
    Version  int64
}

p1 := &Person{0, 0, 0, "Bob", "Smith", 0}
dbmap.Insert(p1)  // Version is now 1

obj, err := dbmap.Get(Person{}, p1.Id)
p2 := obj.(*Person)
p2.LName = "Edwards"
dbmap.Update(p2)  // Version is now 2

p1.LName = "Howard"

// Raises error because p1.Version == 1, which is out of date
count, err := dbmap.Update(p1)
_, ok := err.(gorp.OptimisticLockError)
if ok {
    // should reach this statement
    
    // in a real app you might reload the row and retry, or
    // you might propegate this to the user, depending on the desired
    // semantics
    fmt.Printf("Tried to update row with stale data: %v\n", err)
} else {
    // some other db error occurred - log or return up the stack
    fmt.Printf("Unknown db err: %v\n", err)
}

Contributors

  • matthias-margush - column aliasing via tags

Documentation

Overview

Package gorp provides a simple way to marshal Go structs to and from SQL databases. It uses the database/sql package, and should work with any compliant database/sql driver.

Source code and project home: https://github.com/coopernurse/gorp

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ColumnMap 

type ColumnMap struct {
	// Column name in db table
	ColumnName string

	// If true, this column is skipped in generated SQL statements
	Transient bool

	// If true, " unique" is added to create table statements.
	// Not used elsewhere
	Unique bool

	// Passed to Dialect.ToSqlType() to assist in informing the
	// correct column type to map to in CreateTables()
	// Not used elsewhere
	MaxSize int
	// contains filtered or unexported fields
}

ColumnMap represents a mapping between a Go struct field and a single column in a table. Unique and MaxSize only inform the CreateTables() function and are not used by Insert/Update/Delete/Get.

func (*ColumnMap) Rename 

func (c *ColumnMap) Rename(colname string) *ColumnMap

Rename allows you to specify the column name in the table

Example: table.ColMap("Updated").Rename("date_updated")

func (*ColumnMap) SetMaxSize 

func (c *ColumnMap) SetMaxSize(size int) *ColumnMap

SetMaxSize specifies the max length of values of this column. This is passed to the dialect.ToSqlType() function, which can use the value to alter the generated type for "create table" statements

func (*ColumnMap) SetTransient 

func (c *ColumnMap) SetTransient(b bool) *ColumnMap

SetTransient allows you to mark the column as transient. If true this column will be skipped when SQL statements are generated

func (*ColumnMap) SetUnique 

func (c *ColumnMap) SetUnique(b bool) *ColumnMap

If true " unique" will be added to create table statements for this column

type DbMap 

type DbMap struct {
	// Db handle to use with this map
	Db *sql.DB

	// Dialect implementation to use with this map
	Dialect Dialect
	// contains filtered or unexported fields
}

DbMap is the root gorp mapping object. Create one of these for each database schema you wish to map. Each DbMap contains a list of mapped tables.

Example: 

dialect := gorp.MySQLDialect{"InnoDB", "UTF8"}
dbmap := &gorp.DbMap{Db: db, Dialect: dialect}

func (*DbMap) AddTable 

func (m *DbMap) AddTable(i interface{}) *TableMap

AddTable registers the given interface type with gorp. The table name will be given the name of the TypeOf(i). You must call this function, or AddTableWithName, for any struct type you wish to persist with the given DbMap.

This operation is idempotent. If i's type is already mapped, the existing *TableMap is returned

func (*DbMap) AddTableWithName 

func (m *DbMap) AddTableWithName(i interface{}, name string) *TableMap

AddTableWithName has the same behavior as AddTable, but sets table.TableName to name.

func (*DbMap) Begin 

func (m *DbMap) Begin() (*Transaction, error)

Begin starts a gorp Transaction

func (*DbMap) CreateTables 

func (m *DbMap) CreateTables() error

CreateTables iterates through TableMaps registered to this DbMap and executes "create table" statements against the database for each.

This is particularly useful in unit tests where you want to create and destroy the schema automatically.

func (*DbMap) Delete 

func (m *DbMap) Delete(list ...interface{}) (int64, error)

Delete runs a SQL DELETE statement for each element in list. List items must be pointers.

Hook functions PreDelete() and/or PostDelete() will be executed before/after the DELETE statement if the interface defines them.

Returns number of rows deleted

Returns an error if SetKeys has not been called on the TableMap Panics if any interface in the list has not been registered with AddTable

func (*DbMap) DropTables 

func (m *DbMap) DropTables() error

DropTables iterates through TableMaps registered to this DbMap and executes "drop table" statements against the database for each.

func (*DbMap) Exec 

func (m *DbMap) Exec(query string, args ...interface{}) (sql.Result, error)

Exec runs an arbitrary SQL statement. args represent the bind parameters. This is equivalent to running: Prepare(), Exec() using database/sql

func (*DbMap) Get 

func (m *DbMap) Get(i interface{}, keys ...interface{}) (interface{}, error)

Get runs a SQL SELECT to fetch a single row from the table based on the primary key(s) 

i should be an empty value for the struct to load
keys should be the primary key value(s) for the row to load.  If
multiple keys exist on the table, the order should match the column
order specified in SetKeys() when the table mapping was defined.

Hook function PostGet() will be executed after the SELECT statement if the interface defines them.

Returns a pointer to a struct that matches or nil if no row is found

Returns an error if SetKeys has not been called on the TableMap Panics if any interface in the list has not been registered with AddTable

func (*DbMap) Insert 

func (m *DbMap) Insert(list ...interface{}) error

Insert runs a SQL INSERT statement for each element in list. List items must be pointers.

Any interface whose TableMap has an auto-increment primary key will have its last insert id bound to the PK field on the struct.

Hook functions PreInsert() and/or PostInsert() will be executed before/after the INSERT statement if the interface defines them.

Panics if any interface in the list has not been registered with AddTable

func (*DbMap) Select 

func (m *DbMap) Select(i interface{}, query string, args ...interface{}) ([]interface{}, error)

Select runs an arbitrary SQL query, binding the columns in the result to fields on the struct specified by i. args represent the bind parameters for the SQL statement.

Column names on the SELECT statement should be aliased to the field names on the struct i. Returns an error if one or more columns in the result do not match. It is OK if fields on i are not part of the SQL statement.

Hook function PostGet() will be executed after the SELECT statement if the interface defines them.

Returns a slice of pointers to matching rows of type i.

i does NOT need to be registered with AddTable()

func (*DbMap) TraceOff 

func (m *DbMap) TraceOff()

TraceOff turns off tracing. It is idempotent.

func (*DbMap) TraceOn 

func (m *DbMap) TraceOn(prefix string, logger *log.Logger)

TraceOn turns on SQL statement logging for this DbMap. After this is called, all SQL statements will be sent to the logger. If prefix is a non-empty string, it will be written to the front of all logged strings, which can aid in filtering log lines.

Use TraceOn if you want to spy on the SQL statements that gorp generates.

func (*DbMap) Update 

func (m *DbMap) Update(list ...interface{}) (int64, error)

Update runs a SQL UPDATE statement for each element in list. List items must be pointers.

Hook functions PreUpdate() and/or PostUpdate() will be executed before/after the UPDATE statement if the interface defines them.

Returns number of rows updated

Returns an error if SetKeys has not been called on the TableMap Panics if any interface in the list has not been registered with AddTable

type Dialect 

type Dialect interface {

	// ToSqlType returns the SQL column type to use when creating a
	// table of the given Go Type.  maxsize can be used to switch based on
	// size.  For example, in MySQL []byte could map to BLOB, MEDIUMBLOB,
	// or LONGBLOB depending on the maxsize
	ToSqlType(val reflect.Type, maxsize int, isAutoIncr bool) string

	// string to append to primary key column definitions
	AutoIncrStr() string

	// string to append to "create table" statement for vendor specific
	// table attributes
	CreateTableSuffix() string

	LastInsertId(res *sql.Result, table *TableMap, exec SqlExecutor) (int64, error)

	// bind variable string to use when forming SQL statements
	// in many dbs it is "?", but Postgres appears to use $1
	//
	// i is a zero based index of the bind variable in this statement
	//
	BindVar(i int) string
}

The Dialect interface encapsulates behaviors that differ across SQL databases. At present the Dialect is only used by CreateTables() but this could change in the future

type MySQLDialect 

type MySQLDialect struct {

	// Engine is the storage engine to use "InnoDB" vs "MyISAM" for example
	Engine string

	// Encoding is the character encoding to use for created tables
	Encoding string
}

Implementation of Dialect for MySQL databases.

func (MySQLDialect) AutoIncrStr 

func (m MySQLDialect) AutoIncrStr() string

Returns auto_increment

func (MySQLDialect) BindVar 

func (m MySQLDialect) BindVar(i int) string

Returns "?"

func (MySQLDialect) CreateTableSuffix 

func (m MySQLDialect) CreateTableSuffix() string

Returns engine=%s charset=%s based on values stored on struct

func (MySQLDialect) LastInsertId 

func (m MySQLDialect) LastInsertId(res *sql.Result, table *TableMap, exec SqlExecutor) (int64, error)

func (MySQLDialect) ToSqlType 

func (m MySQLDialect) ToSqlType(val reflect.Type, maxsize int, isAutoIncr bool) string

type OptimisticLockError 

type OptimisticLockError struct {
	// Table name where the lock error occurred
	TableName string

	// Primary key values of the row being updated/deleted
	Keys []interface{}

	// true if a row was found with those keys, indicating the
	// LocalVersion is stale.  false if no value was found with those
	// keys, suggesting the row has been deleted since loaded, or
	// was never inserted to begin with
	RowExists bool

	// Version value on the struct passed to Update/Delete. This value is
	// out of sync with the database.
	LocalVersion int64
}

OptimisticLockError is returned by Update() or Delete() if the struct being modified has a Version field and the value is not equal to the current value in the database

func (OptimisticLockError) Error 

func (e OptimisticLockError) Error() string

Error returns a description of the cause of the lock error

type PostgresDialect 

type PostgresDialect struct {
	// contains filtered or unexported fields
}

func (PostgresDialect) AutoIncrStr 

func (d PostgresDialect) AutoIncrStr() string

Returns empty string

func (PostgresDialect) BindVar 

func (d PostgresDialect) BindVar(i int) string

Returns "$(i+1)"

func (PostgresDialect) CreateTableSuffix 

func (d PostgresDialect) CreateTableSuffix() string

Returns suffix

func (PostgresDialect) LastInsertId 

func (d PostgresDialect) LastInsertId(res *sql.Result, table *TableMap, exec SqlExecutor) (int64, error)

func (PostgresDialect) ToSqlType 

func (d PostgresDialect) ToSqlType(val reflect.Type, maxsize int, isAutoIncr bool) string

type SqlExecutor 

type SqlExecutor interface {
	Get(i interface{}, keys ...interface{}) (interface{}, error)
	Insert(list ...interface{}) error
	Update(list ...interface{}) (int64, error)
	Delete(list ...interface{}) (int64, error)
	Exec(query string, args ...interface{}) (sql.Result, error)
	Select(i interface{}, query string,
		args ...interface{}) ([]interface{}, error)
	// contains filtered or unexported methods
}

SqlExecutor exposes gorp operations that can be run from Pre/Post hooks. This hides whether the current operation that triggered the hook is in a transaction.

See the DbMap function docs for each of the functions below for more information.

type SqliteDialect 

type SqliteDialect struct {
	// contains filtered or unexported fields
}

func (SqliteDialect) AutoIncrStr 

func (d SqliteDialect) AutoIncrStr() string

Returns autoincrement

func (SqliteDialect) BindVar 

func (d SqliteDialect) BindVar(i int) string

Returns "?"

func (SqliteDialect) CreateTableSuffix 

func (d SqliteDialect) CreateTableSuffix() string

Returns suffix

func (SqliteDialect) LastInsertId 

func (d SqliteDialect) LastInsertId(res *sql.Result, table *TableMap, exec SqlExecutor) (int64, error)

func (SqliteDialect) ToSqlType 

func (d SqliteDialect) ToSqlType(val reflect.Type, maxsize int, isAutoIncr bool) string

type TableMap 

type TableMap struct {
	// Name of database table.
	TableName string
	// contains filtered or unexported fields
}

TableMap represents a mapping between a Go struct and a database table Use dbmap.AddTable() or dbmap.AddTableWithName() to create these

func (*TableMap) ColMap 

func (t *TableMap) ColMap(field string) *ColumnMap

ColMap returns the ColumnMap pointer matching the given struct field name. It panics if the struct does not contain a field matching this name.

func (*TableMap) ResetSql 

func (t *TableMap) ResetSql()

ResetSql removes cached insert/update/select/delete SQL strings associated with this TableMap. Call this if you've modified any column names or the table name itself.

func (*TableMap) SetKeys 

func (t *TableMap) SetKeys(isAutoIncr bool, fieldNames ...string) *TableMap

SetKeys lets you specify the fields on a struct that map to primary key columns on the table. If isAutoIncr is set, result.LastInsertId() will be used after INSERT to bind the generated id to the Go struct.

Automatically calls ResetSql() to ensure SQL statements are regenerated.

func (*TableMap) SetVersionCol 

func (t *TableMap) SetVersionCol(field string) *ColumnMap

SetVersionCol sets the column to use as the Version field. By default the "Version" field is used. Returns the column found, or panics if the struct does not contain a field matching this name.

Automatically calls ResetSql() to ensure SQL statements are regenerated.

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction represents a database transaction. Insert/Update/Delete/Get/Exec operations will be run in the context of that transaction. Transactions should be terminated with a call to Commit() or Rollback()

func (*Transaction) Commit 

func (t *Transaction) Commit() error

Commits the underlying database transaction

func (*Transaction) Delete 

func (t *Transaction) Delete(list ...interface{}) (int64, error)

Same behavior as DbMap.Delete(), but runs in a transaction

func (*Transaction) Exec 

func (t *Transaction) Exec(query string, args ...interface{}) (sql.Result, error)

Same behavior as DbMap.Exec(), but runs in a transaction

func (*Transaction) Get 

func (t *Transaction) Get(i interface{}, keys ...interface{}) (interface{}, error)

Same behavior as DbMap.Get(), but runs in a transaction

func (*Transaction) Insert 

func (t *Transaction) Insert(list ...interface{}) error

Same behavior as DbMap.Insert(), but runs in a transaction

func (*Transaction) Rollback 

func (t *Transaction) Rollback() error

Rolls back the underlying database transaction

func (*Transaction) Select 

func (t *Transaction) Select(i interface{}, query string, args ...interface{}) ([]interface{}, error)

Same behavior as DbMap.Select(), but runs in a transaction

func (*Transaction) Update 

func (t *Transaction) Update(list ...interface{}) (int64, error)

Same behavior as DbMap.Update(), but runs in a transaction

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.