db

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2021 License: MPL-2.0 Imports: 31 Imported by: 0

README

db package

Usage

Just some high-level usage highlights to get you started. Read the godocs for a complete list of capabilities and their documentation.

    conn, _ := gorm.Open("postgres", url)
    
    // Db implements both the Reader and Writer interfaces
    rw := Db{Tx: conn}
    
    // There are writer methods like: Create, Update and Delete
    // that will write Gorm struct to the db.  These writer methods
    // all support options for writing Oplog entries for the 
    // caller: WithOplog(yourWrapper, yourMetadata)
    // the caller is responsible for the transaction life cycle of the writer
    // and if an error is returned the caller must decide what to do with 
    // the transaction, which is almost always a rollback for the caller.
    err = rw.Create(context.Background(), user)
   
    // There are reader methods like: LookupByPublicId,  
    // LookupByName, SearchBy, LookupBy, etc
    // which will lookup resources for you and scan them into your Gorm struct
    err = rw.LookupByPublicId(context.Background(), foundUser)

    // There's reader ScanRows that facilitates scanning rows from 
    // a query into your Gorm struct
    where := "select * from test_users where name in ($1, $2)"
    rows, err := rw.Query(context.Background(), where, []interface{}{"alice", "bob"})
	defer rows.Close()
	for rows.Next() {
        user := db_test.NewTestUser()
        // scan the row into your Gorm struct
		if err := rw.ScanRows(rows, &user); err != nil {
            return err
        }
        // Do something with the Gorm user struct
    }

    // DoTx is a writer function that wraps a TxHandler 
    // in a retryable transaction.  You simply implement a
    // TxHandler that does your writes and hand the handler
    // to DoTx, which will wrap the writes in a retryable 
    // transaction with the retry attempts and backoff
    // strategy you specify via options.
    _, err = w.DoTx(
        context.Background(),
        10,           // ten retries
        ExpBackoff{}, // exponential backoff
        func(w Writer) error {
            // the TxHandler updates the user's friendly name
            return w.Update(context.Background(), user, []string{"FriendlyName"},
                // write oplogs for this update
                WithOplog(
                    InitTestWrapper(t),
                    oplog.Metadata{
                        "key-only":   nil,
                        "deployment": []string{"amex"},
                        "project":    []string{"central-info-systems", "local-info-systems"},
                    },
                ),
            )
        })

Documentation

Index

Constants

View Source 
const (
	NoRowsAffected = 0

	// DefaultLimit is the default for results for boundary
	DefaultLimit = 10000
)
View Source 
const (
	// UnknownOrderBy would designate an unknown ordering of the column, which
	// is the standard ordering for any select without an order by clause.
	UnknownOrderBy = iota

	// AscendingOrderBy would designate ordering the column in ascending order.
	AscendingOrderBy

	// DescendingOrderBy would designate ordering the column in decending order.
	DescendingOrderBy
)
View Source 
const (
	StdRetryCnt = 20
)

Variables

View Source 
var (
	NegativeInfinityTS = time.Date(math.MinInt32, time.January, 1, 0, 0, 0, 0, time.UTC)
	PositiveInfinityTS = time.Date(math.MaxInt32, time.December, 31, 23, 59, 59, 1e9-1, time.UTC)
)

Functions

func AssertPublicId 

func AssertPublicId(t *testing.T, prefix, actual string)

AssertPublicId is a test helper that asserts that the provided id is in the format of a public id.

func Clear 

func Clear(i interface{}, fields []string, depth int) error

Clear sets fields in the value pointed to by i to their zero value. Clear descends i to depth clearing fields at each level. i must be a pointer to a struct. Cycles in i are not detected.

A depth of 2 will change i and i's children. A depth of 1 will change i but no children of i. A depth of 0 will return with no changes to i.

func GetGormLogFormatter 

func GetGormLogFormatter(log hclog.Logger) func(values ...interface{}) (messages []interface{})

func GetGormLogger 

func GetGormLogger(log hclog.Logger) gormLogger

func NewPrivateId 

func NewPrivateId(prefix string, opt ...Option) (string, error)

func NewPublicId 

func NewPublicId(prefix string, opt ...Option) (string, error)

NewPublicId creates a new public id with the prefix

func Open 

func Open(dbType DbType, connectionUrl string) (*gorm.DB, error)

Open a database connection which is long-lived. You need to call Close() on the returned gorm.DB

func TestSetup 

func TestSetup(t *testing.T, dialect string, opt ...TestOption) (*gorm.DB, string)

setup the tests (initialize the database one-time and intialized testDatabaseURL). Do not close the returned db.

func TestVerifyOplog 

func TestVerifyOplog(t *testing.T, r Reader, resourceId string, opt ...TestOption) error

TestVerifyOplog will verify that there is an oplog entry that matches the provided resourceId. By default it matches the provided resourceId against the `resource-public-id` tag. If the WithResourcePrivateId option is provided, the lookup will use the `resource-private-id` tag. An error is returned if the entry or it's metadata is not found. Returning an error allows clients to test if an entry was not written, which is a valid use case.

func TestWrapper 

func TestWrapper(t *testing.T) wrapping.Wrapper

TestWrapper initializes an AEAD wrapping.Wrapper for testing the oplog

Types

type Backoff 

type Backoff interface {
	Duration(attemptNumber uint) time.Duration
}

type ConstBackoff 

type ConstBackoff struct {
	DurationMs time.Duration
}

func (ConstBackoff) Duration 

func (b ConstBackoff) Duration(attempt uint) time.Duration

type Db 

type Db struct {
	// contains filtered or unexported fields
}

Db uses a gorm DB connection for read/write

func New 

func New(underlying *gorm.DB) *Db

func (*Db) Create 

func (rw *Db) Create(ctx context.Context, i interface{}, opt ...Option) error

Create an object in the db with options: WithOplog, NewOplogMsg and WithLookup. WithOplog will write an oplog entry for the create. NewOplogMsg will return in-memory oplog message. WithOplog and NewOplogMsg cannot be used together. WithLookup with to force a lookup after create.

func (*Db) CreateItems 

func (rw *Db) CreateItems(ctx context.Context, createItems []interface{}, opt ...Option) error

CreateItems will create multiple items of the same type. Supported options: WithOplog and WithOplogMsgs. WithOplog and WithOplogMsgs may not be used together. WithLookup is not a supported option.

func (*Db) Delete 

func (rw *Db) Delete(ctx context.Context, i interface{}, opt ...Option) (int, error)

Delete an object in the db with options: WithOplog, NewOplogMsg, WithWhere. WithOplog will write an oplog entry for the delete. NewOplogMsg will return in-memory oplog message. WithOplog and NewOplogMsg cannot be used together. WithWhere allows specifying a constraint. Delete returns the number of rows deleted and any errors.

func (*Db) DeleteItems 

func (rw *Db) DeleteItems(ctx context.Context, deleteItems []interface{}, opt ...Option) (int, error)

DeleteItems will delete multiple items of the same type. Supported options: WithOplog and WithOplogMsgs. WithOplog and WithOplogMsgs may not be used together.

func (*Db) DoTx 

func (w *Db) DoTx(ctx context.Context, retries uint, backOff Backoff, Handler TxHandler) (RetryInfo, error)

DoTx will wrap the Handler func passed within a transaction with retries you should ensure that any objects written to the db in your TxHandler are retryable, which means that the object may be sent to the db several times (retried), so things like the primary key must be reset before retry

func (*Db) Exec 

func (rw *Db) Exec(ctx context.Context, sql string, values []interface{}, _ ...Option) (int, error)

Exec will execute the sql with the values as parameters. The int returned is the number of rows affected by the sql. No options are currently supported.

func (*Db) GetTicket 

func (rw *Db) GetTicket(i interface{}) (*store.Ticket, error)

GetTicket returns an oplog ticket for the aggregate root of "i" which can be used to WriteOplogEntryWith for that aggregate root.

func (*Db) LookupById 

func (rw *Db) LookupById(ctx context.Context, resourceWithIder interface{}, _ ...Option) error

LookupByPublicId will lookup resource by its public_id or private_id, which must be unique. Options are ignored.

func (*Db) LookupByPublicId 

func (rw *Db) LookupByPublicId(ctx context.Context, resource ResourcePublicIder, opt ...Option) error

LookupByPublicId will lookup resource by its public_id, which must be unique. Options are ignored.

func (*Db) LookupWhere 

func (rw *Db) LookupWhere(ctx context.Context, resource interface{}, where string, args ...interface{}) error

LookupWhere will lookup the first resource using a where clause with parameters (it only returns the first one)

func (*Db) Query 

func (rw *Db) Query(ctx context.Context, sql string, values []interface{}, _ ...Option) (*sql.Rows, error)

Query will run the raw query and return the *sql.Rows results. Query will operate within the context of any ongoing transaction for the db.Reader. The caller must close the returned *sql.Rows. Query can/should be used in combination with ScanRows.

func (*Db) ScanRows 

func (rw *Db) ScanRows(rows *sql.Rows, result interface{}) error

Scan rows will scan the rows into the interface

func (*Db) SearchWhere 

func (rw *Db) SearchWhere(ctx context.Context, resources interface{}, where string, args []interface{}, opt ...Option) error

SearchWhere will search for all the resources it can find using a where clause with parameters. An error will be returned if args are provided without a where clause.

Supports the WithLimit option. If WithLimit < 0, then unlimited results are returned. If WithLimit == 0, then default limits are used for results. Supports the WithOrder option.

func (*Db) Update 

func (rw *Db) Update(ctx context.Context, i interface{}, fieldMaskPaths []string, setToNullPaths []string, opt ...Option) (int, error)

Update an object in the db, fieldMask is required and provides field_mask.proto paths for fields that should be updated. The i interface parameter is the type the caller wants to update in the db and its fields are set to the update values. setToNullPaths is optional and provides field_mask.proto paths for the fields that should be set to null. fieldMaskPaths and setToNullPaths must not intersect. The caller is responsible for the transaction life cycle of the writer and if an error is returned the caller must decide what to do with the transaction, which almost always should be to rollback. Update returns the number of rows updated.

Supported options: WithOplog, NewOplogMsg and WithVersion. WithOplog will write an oplog entry for the update. NewOplogMsg will return in-memory oplog message. WithOplog and NewOplogMsg cannot be used together. If WithVersion is used, then the update will include the version number in the update where clause, which basically makes the update use optimistic locking and the update will only succeed if the existing rows version matches the WithVersion option. Zero is not a valid value for the WithVersion option and will return an error.

func (*Db) WriteOplogEntryWith 

func (rw *Db) WriteOplogEntryWith(ctx context.Context, wrapper wrapping.Wrapper, ticket *store.Ticket, metadata oplog.Metadata, msgs []*oplog.Message, _ ...Option) error

WriteOplogEntryWith will write an oplog entry with the msgs provided for the ticket's aggregateName. No options are currently supported.

type DbType 

type DbType int
const (
	UnknownDB DbType = 0
	Postgres  DbType = 1
)

func (DbType) String 

func (db DbType) String() string

type ExpBackoff 

type ExpBackoff struct{}

func (ExpBackoff) Duration 

func (b ExpBackoff) Duration(attempt uint) time.Duration

type OpType 

type OpType int
const (
	UnknownOp OpType = 0
	CreateOp  OpType = 1
	UpdateOp  OpType = 2
	DeleteOp  OpType = 3
)

type Option 

type Option func(*Options)

Option - how Options are passed as arguments.

func NewOplogMsg 

func NewOplogMsg(msg *oplog.Message) Option

NewOplogMsg provides an option to ask for a new in-memory oplog message. The new msg will be returned in the provided *oplog.Message parameter. WithOplog and NewOplogMsg cannot be used together.

func NewOplogMsgs 

func NewOplogMsgs(msgs *[]*oplog.Message) Option

NewOplogMsgs provides an option to ask for multiple new in-memory oplog messages. The new msgs will be returned in the provided *[]oplog.Message parameter. NewOplogMsgs can only be used with write functions that operate on multiple items(CreateItems, DeleteItems). WithOplog and NewOplogMsgs cannot be used together.

func WithFieldMaskPaths 

func WithFieldMaskPaths(paths []string) Option

WithFieldMaskPaths provides an option to provide field mask paths.

func WithLimit 

func WithLimit(limit int) Option

WithLimit provides an option to provide a limit. Intentionally allowing negative integers. If WithLimit < 0, then unlimited results are returned. If WithLimit == 0, then default limits are used for results.

func WithLookup 

func WithLookup(enable bool) Option

WithLookup enables a lookup.

func WithNullPaths 

func WithNullPaths(paths []string) Option

WithNullPaths provides an option to provide null paths.

func WithOplog 

func WithOplog(wrapper wrapping.Wrapper, md oplog.Metadata) Option

WithOplog provides an option to write an oplog entry. WithOplog and NewOplogMsg cannot be used together.

func WithOrder 

func WithOrder(withOrder string) Option

WithOrder provides an option to provide an order when searching and looking up.

func WithPrngValues added in v0.2.0 

func WithPrngValues(withPrngValues []string) Option

WithPrngValues provides an option to provide values to seed an PRNG when generating IDs

func WithSkipVetForWrite 

func WithSkipVetForWrite(enable bool) Option

WithSkipVetForWrite provides an option to allow skipping vet checks to allow testing lower-level SQL triggers and constraints

func WithVersion 

func WithVersion(version *uint32) Option

WithVersion provides an option version number for update operations.

func WithWhere 

func WithWhere(whereClause string, args ...interface{}) Option

WithWhere provides an option to provide a where clause with arguments for an operation.

type Options 

type Options struct {

	// WithLimit must be accessible in other packages.
	WithLimit int
	// WithFieldMaskPaths must be accessible from other packages.
	WithFieldMaskPaths []string
	// WithNullPaths must be accessible from other packages.
	WithNullPaths []string

	// WithVersion must be accessible from other packages.
	WithVersion *uint32
	// contains filtered or unexported fields
}

Options - how Options are represented.

func GetOpts 

func GetOpts(opt ...Option) Options

GetOpts - iterate the inbound Options and return a struct.

type OrderBy added in v0.2.0 

type OrderBy int

OrderBy defines an enum type for declaring a column's order by criteria.

type Reader 

type Reader interface {
	// LookupById will lookup a resource by its primary key id, which must be
	// unique. The resourceWithIder must implement either ResourcePublicIder or
	// ResourcePrivateIder interface.
	LookupById(ctx context.Context, resourceWithIder interface{}, opt ...Option) error

	// LookupByPublicId will lookup resource by its public_id which must be unique.
	LookupByPublicId(ctx context.Context, resource ResourcePublicIder, opt ...Option) error

	// LookupWhere will lookup and return the first resource using a where clause with parameters
	LookupWhere(ctx context.Context, resource interface{}, where string, args ...interface{}) error

	// SearchWhere will search for all the resources it can find using a where
	// clause with parameters. Supports the WithLimit option.  If
	// WithLimit < 0, then unlimited results are returned.  If WithLimit == 0, then
	// default limits are used for results.
	SearchWhere(ctx context.Context, resources interface{}, where string, args []interface{}, opt ...Option) error

	// Query will run the raw query and return the *sql.Rows results. Query will
	// operate within the context of any ongoing transaction for the db.Reader.  The
	// caller must close the returned *sql.Rows. Query can/should be used in
	// combination with ScanRows.
	Query(ctx context.Context, sql string, values []interface{}, opt ...Option) (*sql.Rows, error)

	// ScanRows will scan sql rows into the interface provided
	ScanRows(rows *sql.Rows, result interface{}) error
}

Reader interface defines lookups/searching for resources

type ResourcePrivateIder 

type ResourcePrivateIder interface {
	GetPrivateId() string
}

ResourcePrivateIder defines an interface that LookupById() can use to get the resource's private id.

type ResourcePublicIder 

type ResourcePublicIder interface {
	GetPublicId() string
}

ResourcePublicIder defines an interface that LookupByPublicId() can use to get the resource's public id.

type RetryInfo 

type RetryInfo struct {
	Retries int
	Backoff time.Duration
}

RetryInfo provides information on the retries of a transaction

type TestOption 

type TestOption func(*testOptions)

TestOption - how Options are passed as arguments

func WithCreateNotBefore 

func WithCreateNotBefore(nbfDuration time.Duration) TestOption

WithCreateNotBefore provides an option to specify that the create time is not before (nbf) N seconds

func WithOperation 

func WithOperation(op oplog.OpType) TestOption

WithOperation provides an option to specify the operation type

func WithResourcePrivateId added in v0.2.2 

func WithResourcePrivateId(enable bool) TestOption

WithResourcePrivateId provides a way to specify that the resource lookup action uses `resource-private-id` tag instead of the default `resource-public-id` tag

func WithTemplate added in v0.6.1 

func WithTemplate(template string) TestOption

WithTemplate provides a way to specify the source database template for creating a database.

func WithTestDatabaseUrl added in v0.1.1 

func WithTestDatabaseUrl(url string) TestOption

WithTestDatabaseUrl provides a way to specify an existing database for tests

type TxHandler 

type TxHandler func(Reader, Writer) error

TxHandler defines a handler for a func that writes a transaction for use with DoTx

type VetForWriter 

type VetForWriter interface {
	VetForWrite(ctx context.Context, r Reader, opType OpType, opt ...Option) error
}

VetForWriter provides an interface that Create and Update can use to vet the resource before before writing it to the db. For optType == UpdateOp, options WithFieldMaskPath and WithNullPaths are supported. For optType == CreateOp, no options are supported

type Writer 

type Writer interface {
	// DoTx will wrap the TxHandler in a retryable transaction
	DoTx(ctx context.Context, retries uint, backOff Backoff, Handler TxHandler) (RetryInfo, error)

	// Update an object in the db, fieldMask is required and provides
	// field_mask.proto paths for fields that should be updated. The i interface
	// parameter is the type the caller wants to update in the db and its
	// fields are set to the update values. setToNullPaths is optional and
	// provides field_mask.proto paths for the fields that should be set to
	// null.  fieldMaskPaths and setToNullPaths must not intersect. The caller
	// is responsible for the transaction life cycle of the writer and if an
	// error is returned the caller must decide what to do with the transaction,
	// which almost always should be to rollback.  Update returns the number of
	// rows updated or an error. Supported options: WithOplog.
	Update(ctx context.Context, i interface{}, fieldMaskPaths []string, setToNullPaths []string, opt ...Option) (int, error)

	// Create an object in the db with options: WithOplog
	// the caller is responsible for the transaction life cycle of the writer
	// and if an error is returned the caller must decide what to do with
	// the transaction, which almost always should be to rollback.
	Create(ctx context.Context, i interface{}, opt ...Option) error

	// CreateItems will create multiple items of the same type.
	// Supported options: WithOplog and WithOplogMsgs.  WithOplog and
	// WithOplogMsgs may not be used together. WithLookup is not a
	// supported option. The caller is responsible for the transaction life
	// cycle of the writer and if an error is returned the caller must decide
	// what to do with the transaction, which almost always should be to
	// rollback.
	CreateItems(ctx context.Context, createItems []interface{}, opt ...Option) error

	// Delete an object in the db with options: WithOplog
	// the caller is responsible for the transaction life cycle of the writer
	// and if an error is returned the caller must decide what to do with
	// the transaction, which almost always should be to rollback. Delete
	// returns the number of rows deleted or an error.
	Delete(ctx context.Context, i interface{}, opt ...Option) (int, error)

	// DeleteItems will delete multiple items of the same type.
	// Supported options: WithOplog and WithOplogMsgs.  WithOplog and
	// WithOplogMsgs may not be used together. The caller is responsible for the
	// transaction life cycle of the writer and if an error is returned the
	// caller must decide what to do with the transaction, which almost always
	// should be to rollback. Delete returns the number of rows deleted or an error.
	DeleteItems(ctx context.Context, deleteItems []interface{}, opt ...Option) (int, error)

	// Exec will execute the sql with the values as parameters. The int returned
	// is the number of rows affected by the sql. No options are currently
	// supported.
	Exec(ctx context.Context, sql string, values []interface{}, opt ...Option) (int, error)

	// Query will run the raw query and return the *sql.Rows results. Query will
	// operate within the context of any ongoing transaction for the db.Writer.  The
	// caller must close the returned *sql.Rows. Query can/should be used in
	// combination with ScanRows.  Query is included in the Writer interface
	// so callers can execute updates and inserts with returning values.
	Query(ctx context.Context, sql string, values []interface{}, opt ...Option) (*sql.Rows, error)

	// GetTicket returns an oplog ticket for the aggregate root of "i" which can
	// be used to WriteOplogEntryWith for that aggregate root.
	GetTicket(i interface{}) (*store.Ticket, error)

	// WriteOplogEntryWith will write an oplog entry with the msgs provided for
	// the ticket's aggregateName. No options are currently supported.
	WriteOplogEntryWith(
		ctx context.Context,
		wrapper wrapping.Wrapper,
		ticket *store.Ticket,
		metadata oplog.Metadata,
		msgs []*oplog.Message,
		opt ...Option,
	) error

	// ScanRows will scan sql rows into the interface provided
	ScanRows(rows *sql.Rows, result interface{}) error
}

Writer interface defines create, update and retryable transaction handlers

Source Files

View all Source files

Directories

Path Synopsis
Package dbassert provides a set of assertions for testing the boundary database applications.
 Package dbassert provides a set of assertions for testing the boundary database applications.
common package contains functions from internal/db which need to be shared commonly with other packages that have a cyclic dependency on internal/db like internal/oplog.
 common package contains functions from internal/db which need to be shared commonly with other packages that have a cyclic dependency on internal/db like internal/oplog.
Package db_test provides some helper funcs for testing db integrations
 Package db_test provides some helper funcs for testing db integrations
Package sanitize contains a set of functions that sanitizes input received from external systems before being persisted in the database.
 Package sanitize contains a set of functions that sanitizes input received from external systems before being persisted in the database.
migrations/generate
postgres
Package sentinel allows for the use of Unicode non-characters to distinguish between Boundary defined sentinels and values provided by external systems.
 Package sentinel allows for the use of Unicode non-characters to distinguish between Boundary defined sentinels and values provided by external systems.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.