pivot

README

Pivot

Pivot is a library used to access, query, and aggregate data across a variety of database systems, written in Golang.

Where: Packages and Usage

Pivot is organized into multiple sub-packages that perform various functions:

Package	What it does...
pivot	Entry point for the package. Connect to a database from here.
pivot/dal	Data Abstraction Layer; provides a database-agnostic view of collections (i.e: tables), records (i.e.: rows), and the fields that make up those records (i.e.: columns).
pivot/filter	A database-agnostic representation of queries that return some subset of the data from a collection.
pivot/mapper	A more user-friendly data mapping layer that provides more traditional ODM/ORM semantics for working with collections.
pivot/backends	Where all the database and search index adapters live.
pivot/utils	Utilities largely used within the rest of this library.

Supported Backends

Below is a table describing which data systems are currently supported. For systems with Backend support, Pivot can create and delete collections, and create/retrieve/update/delete records by their primary key / ID. If a system has Indexer support, Pivot can perform arbitrary queries against collections using a standard filter syntax, and return the results as a set of standard Record objects.

Product	Backend	Indexer	Notes
MySQL / MariaDB	X	X
PostgreSQL	X	X
SQLite 3.x	X	X
Filesystem	X	X
MongoDB	X	X
Amazon DynamoDB	X	partial	Supports queries that involve Range Key and Sort Key only
Redis	X
Elasticsearch		X

How: Examples

Example 1: Basic CRUD operations using the mapper.Mapper interface

Here is a simple example for connecting to a SQLite database, creating a table; and inserting, retrieving, and deleting a record.

package main

import (
    "fmt"
    "time"

    "github.com/PerformLine/pivot/v3"
    "github.com/PerformLine/pivot/v3/dal"
    "github.com/PerformLine/pivot/v3/mapper"
)

var Widgets mapper.Mapper

var WidgetsSchema = &dal.Collection{
    Name:                   `widgets`,
    IdentityFieldType:      dal.StringType,
    IdentityFieldFormatter: dal.GenerateUUID,
    Fields: []dal.Field{
        {
            Name:        `type`,
            Description: `The type of widget.`,
            Type:        dal.StringType,
            Validator:   dal.ValidateIsOneOf(`foo`, `bar`, `baz`),
            Required:    true,
        }, {
            Name:        `usage`,
            Description: `Short description on how to use this widget.`,
            Type:        dal.StringType,
        }, {
            Name:        `created_at`,
            Description: `When the widget was created.`,
            Type:        dal.TimeType,
            Formatter:   dal.CurrentTimeIfUnset,
        }, {
            Name:        `updated_at`,
            Description: `Last time the widget was updated.`,
            Type:        dal.TimeType,
            Formatter:   dal.CurrentTime,
        },
    },
}

type Widget struct {
    ID        string    `pivot:"id,identity"`
    Type      string    `pivot:"type"`
    Usage     string    `pivot:"usage"`
    CreatedAt time.Time `pivot:"created_at"`
    UpdatedAt time.Time `pivot:"updated_at"`
}

func main() {
    // setup a new backend instance based on the supplied connection string
    if backend, err := pivot.NewDatabase(`sqlite:///./test.db`); err == nil {

        // initialize the backend (connect to/open it)
        if err := backend.Initialize(); err == nil {

            // register models to this database backend
            Widgets = mapper.NewModel(backend, WidgetsSchema)

            // create the model tables if they don't exist
            if err := Widgets.Migrate(); err != nil {
                fmt.Printf("failed to create widget table: %v\n", err)
                return
            }

            // make a new Widget instance, containing the data we want to see
            // the ID field will be populated after creation with the auto-
            // generated UUID.
            newWidget := Widget{
                Type:  `foo`,
                Usage: `A fooable widget.`,
            }

            // insert a widget (ID will be auto-generated because of dal.GenerateUUID)
            if err := Widgets.Create(&newWidget); err != nil {
                fmt.Printf("failed to insert widget: %v\n", err)
                return
            }

            // retrieve the widget using the ID we just got back
            var gotWidget Widget

            if err := Widgets.Get(newWidget.ID, &gotWidget); err != nil {
                fmt.Printf("failed to retrieve widget: %v\n", err)
                return
            }

            fmt.Printf("Got Widget: %#+v", gotWidget)

            // delete the widget
            if err := Widgets.Delete(newWidget.ID); err != nil {
                fmt.Printf("failed to delete widget: %v\n", err)
                return
            }
        } else {
            fmt.Printf("failed to initialize backend: %v\n", err)
            return
        }
    } else {
        fmt.Printf("failed to create backend: %v\n", err)
        return
    }

}

Why: Why Use This?

The ability to mix and match persistent structured data storage and retrieval mechanisms with various indexing strategies is a powerful one. The idea here is to provide a common interface for systems to integrate with in a way that doesn't tightly couple those systems to specific databases, query languages, and infrastructures. It's an attempt to deliver on the promises of traditional ORM/ODM libraries in a platform- and language-agnostic way.

Documentation

Index

• Constants
• Variables
• func ApplySchemata(fileOrDirPath string, db Backend) error
• func Dir(useLocal bool, name string) http.FileSystem
• func FS(useLocal bool) http.FileSystem
• func FSByte(useLocal bool, name string) ([]byte, error)
• func FSMustByte(useLocal bool, name string) []byte
• func FSMustString(useLocal bool, name string) string
• func FSString(useLocal bool, name string) (string, error)
• func LoadFixtures(fileOrDirPath string, db Backend) error
• func LoadFixturesFromFile(filename string, db Backend) error
• func LoadSchemata(fileOrDirPaths ...string) ([]*dal.Collection, error)
• func LoadSchemataFromFile(filename string) ([]*dal.Collection, error)
• func MustGetCollection(db Backend, name string) *dal.Collection
• type Backend
• type Collection
• type Configuration
  • func LoadConfigFile(path string) (Configuration, error)
  • func (self *Configuration) ForEnv(env string) Configuration
• type ConnectOptions
• type DB
  • func NewDatabase(connection string) (DB, error)
  • func NewDatabaseWithOptions(connection string, options ConnectOptions) (DB, error)
• type Filter
• type Model
• type Record
• type RecordSet

Constants

const ApplicationName = `pivot`

const ApplicationSummary = `an extensible database abstraction service`

const ApplicationVersion = `3.3.18`

Variables

var MonitorCheckInterval = time.Duration(10) * time.Second

var NetrcFile = ``

Functions

func ApplySchemata

func ApplySchemata(fileOrDirPath string, db Backend) error

Creates all non-existent schemata in the given directory.

func Dir

func Dir(useLocal bool, name string) http.FileSystem

Dir returns a http.Filesystem for the embedded assets on a given prefix dir. If useLocal is true, the filesystem's contents are instead used.

func FS

func FS(useLocal bool) http.FileSystem

FS returns a http.Filesystem for the embedded assets. If useLocal is true, the filesystem's contents are instead used.

func FSByte

func FSByte(useLocal bool, name string) ([]byte, error)

FSByte returns the named file from the embedded assets. If useLocal is true, the filesystem's contents are instead used.

func FSMustByte

func FSMustByte(useLocal bool, name string) []byte

FSMustByte is the same as FSByte, but panics if name is not present.

func FSMustString

func FSMustString(useLocal bool, name string) string

FSMustString is the string version of FSMustByte.

func FSString

func FSString(useLocal bool, name string) (string, error)

FSString is the string version of FSByte.

func LoadFixtures

func LoadFixtures(fileOrDirPath string, db Backend) error

Calls LoadFixturesFromFile from all *.json files in the given directory.

func LoadFixturesFromFile

func LoadFixturesFromFile(filename string, db Backend) error

Loads a JSON-encoded array of dal.Record objects from a file into the given DB backend instance.

func LoadSchemata

func LoadSchemata(fileOrDirPaths ...string) ([]*dal.Collection, error)

Calls LoadSchemataFromFile from all *.json files in the given directory.

func LoadSchemataFromFile

func LoadSchemataFromFile(filename string) ([]*dal.Collection, error)

Loads and registers a JSON-encoded array of dal.Collection objects into the given DB backend instance.

func MustGetCollection

func MustGetCollection(db Backend, name string) *dal.Collection

A panicky version of backends.Backend.GetCollection

Types

type Backend

type Backend = backends.Backend

create handy type aliases to avoid importing from all over the place

type Collection

type Collection = dal.Collection

type Configuration

type Configuration struct {
	Backend               string                   `json:"backend"`
	Indexer               string                   `json:"indexer"`
	Autoexpand            bool                     `json:"autoexpand"`
	AutocreateCollections bool                     `json:"autocreate"`
	Environments          map[string]Configuration `json:"environments"`
}

func LoadConfigFile

func LoadConfigFile(path string) (Configuration, error)

func (*Configuration) ForEnv

func (self *Configuration) ForEnv(env string) Configuration

type ConnectOptions

type ConnectOptions = backends.ConnectOptions

type DB

type DB interface {
	backends.Backend
	AttachCollection(*Collection) Model
	C(string) *Collection
	Migrate(context.Context) error
	Models() []Model
	ApplySchemata(fileOrDirPath string) error
	LoadFixtures(fileOrDirPath string) error
	GetBackend() Backend
	SetBackend(Backend)
}

func NewDatabase

func NewDatabase(connection string) (DB, error)

Create a new database connection with the default options.

func NewDatabaseWithOptions

func NewDatabaseWithOptions(connection string, options ConnectOptions) (DB, error)

Create a new database connection with the given options.

type Filter

type Filter = filter.Filter

type Model

type Model = backends.Mapper

type Record

type Record = dal.Record

type RecordSet

type RecordSet = dal.RecordSet