dbsample

package module
v0.1.2-0...-cf14a80 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 24, 2017 License: MIT Imports: 19 Imported by: 1

README

DBSample

DBSample creates database dumps which make sense.

ALPHA RELEASE! USE WITH CAUTION!

Dealing with large development/staging databases is unacceptable in an age of cloud services and containers, where personal dev platforms can be spun up in seconds and then discarded. Containers have made it simple for each member of the dev team to work on their own copy of an application, but often the team is sharing a single dev/staging database at a remote location. Those databases often contain old data, and altering them is impossible without effecting everyone else on the team.

Random data generators are a common solution to the problem of creating small testable databases, but they generate data that is usually a poor representation of the real application data, and the generator itself is difficult to create and becomes another piece of software to be maintained. DBSample solves the problem by creating a snapshot of your real database with a small sample of the real data.

Currently supports MySQL 5+. Other drivers and versions may be supported in the future.

Install

Linux and Windows builds are available on the releases page.

Usage

usage: dbsample [<flags>] <database>

Flags:
      --help                 Show context-sensitive help (also try --help-long and --help-man).
      --version              Show application version.
  -h, --host="127.0.0.1"     The database host.
  -P, --port="3306"          The database port.
      --protocol="tcp"       The protocol to use for the connection (tcp, socket, pip, memory).
  -u, --user=USER            User for login if not current user.
  -p, --password=PASSWORD    Password to use when connecting to server. If password is not given it's asked from stderr.
      --routines             Dump procedures and functions.
      --triggers             Dump triggers.
  -l, --limit=100            Max number of rows from each table to dump.
  -n, --no-create-database   Disable adding CREATE DATABASE statement.
      --skip-lock-tables     Disable locking tables on read.
      --skip-add-drop-table  Disable adding DROP TABLE statements.
      --extended-insert      Use multiple-row INSERT syntax that include several VALUES lists.
      --rename-database=DUMP-NAME  
                             Use this database name in the dump.
  -c, --constraint=CONSTRAINT ...  
                             Assigns one or more foreign key constraints.
  -f, --filter=FILTER ...    Apply a filter to the output.

Args:
  <database>  Name of the database to dump.

Filters:
Filters alter column values in the dump. For example they can remove passwords or
other sensitive information. Each --filter flag should be passed the name of the
filter, e.g. "empty", the name of a table.column, e.g. "users.passwords", and one
or more arguments.

  --filter="empty table.column"
  --filter="repeat table.column <string>"

Examples:
dbsample --limit=100 blog > dump.sql
dbsample --limit=100 -h db1 -u admin -p blog > dump.sql
dbsample --limit=100 --rename-database=blog_dev blog > dump.sql
dbsample --limit=100 --filter="empty users.password" --filter="repeat users.email X" blog > dump.sql
dbsample --limit=100 -c "posts.user_id users.id" -c "posts.cat_id categories.id" blog > dump.sql

Documentation

Index

Constants

View Source 
const (
	Name    = "DBSample"
	Version = "0.1"
)
View Source 
const (
	DriverMySQL = "mysql"
)
View Source 
const MySQL5DumperTemplatesPath = "./templates/mysql"

Variables

View Source 
var Filters = filters.NewFilterController()
View Source 
var IsDebugBuild = false
View Source 
var IsDebugging = false

Functions

func Dump 

func Dump() error

Dump...

func MySQL5Backtick 

func MySQL5Backtick(col string) string

MySQL5Backtick...

func MySQL5BacktickUser 

func MySQL5BacktickUser(user string) string

MySQL5BacktickUser...

func MySQL5Escape 

func MySQL5Escape(val string) string

MySQL5Escape... @see https://dev.mysql.com/doc/refman/5.7/en/string-literals.html

func MySQL5JoinColumns 

func MySQL5JoinColumns(cols []string) string

MySQL5JoinColumns...

func MySQL5JoinValues 

func MySQL5JoinValues(vals []string) string

MySQL5JoinValues...

func MySQL5Quote 

func MySQL5Quote(val string) string

MySQL5Quote...

func ParseFlags 

func ParseFlags() (*ConnectionArgs, *DumpArgs, error)

ParseFlags parses the command line flags.

Types

type Column 

type Column struct {
	Name                   string
	OrdinalPosition        int
	Type                   string
	CharacterMaximumLength int64
	DataType               string
}

Column...

type ColumnMap 

type ColumnMap map[string]*Column

type ConnectionArgs 

type ConnectionArgs struct {
	Driver   string
	Name     string
	Host     string
	Port     string
	User     string
	Pass     string
	Protocol string
}

ConnectionArgs...

type Constraint 

type Constraint struct {
	TableName            string
	ColumnName           string
	ReferencedColumnName string
}

Constraint...

type Database 

type Database interface {
	Name() string
	SetName(string)
	CharSet() string
	Collation() string
	CreateSQL() (string, error)
	SetCreateSQL(string)
	Tables() (TableGraph, error)
	Views() (ViewGraph, error)
	Routines() (RoutineGraph, error)
	Server() *Server
}

Database queries a database.

type DumpArgs 

type DumpArgs struct {
	Limit            int
	Routines         bool
	Triggers         bool
	RenameDatabase   string
	NoCreateDatabase bool
	SkipLockTables   bool
	SkipAddDropTable bool
	ExtendedInsert   bool
	Filters          []string
	Constraints      map[string][]*Constraint
}

DumpArgs...

type Dumper 

type Dumper interface {
	Dump(w io.Writer, db Database) error
}

Dumper...

func NewDumper 

func NewDumper(s *Server) (Dumper, error)

NewDumper returns a Dumper instance.

type Field 

type Field struct {
	Column string
	Value  string
}

Field...

type MySQL5Database 

type MySQL5Database struct {
	// contains filtered or unexported fields
}

MySQL5Database implements Database for MySQL5.

func NewMySQL5Database 

func NewMySQL5Database(server *Server, name, charSet, collation string) *MySQL5Database

NewMySQL5Database returns a new *MySQL5Database instance.

func (*MySQL5Database) CharSet 

func (db *MySQL5Database) CharSet() string

CharSet...

func (*MySQL5Database) Collation 

func (db *MySQL5Database) Collation() string

Collation...

func (*MySQL5Database) CreateSQL 

func (db *MySQL5Database) CreateSQL() (string, error)

CreateSQL...

func (*MySQL5Database) Name 

func (db *MySQL5Database) Name() string

Name...

func (*MySQL5Database) Routines 

func (db *MySQL5Database) Routines() (routines RoutineGraph, err error)

Routines...

func (*MySQL5Database) Server 

func (db *MySQL5Database) Server() *Server

Server...

func (*MySQL5Database) SetCreateSQL 

func (db *MySQL5Database) SetCreateSQL(sql string)

SetCreateSQL...

func (*MySQL5Database) SetName 

func (db *MySQL5Database) SetName(name string)

SetName...

func (*MySQL5Database) Tables 

func (db *MySQL5Database) Tables() (tables TableGraph, err error)

Tables...

func (*MySQL5Database) Views 

func (db *MySQL5Database) Views() (views ViewGraph, err error)

Views...

type MySQL5Dumper 

type MySQL5Dumper struct {
	// contains filtered or unexported fields
}

MySQL5Dumper...

func NewMySQL5Dumper 

func NewMySQL5Dumper(args *DumpArgs) *MySQL5Dumper

NewMySQL5Dumper returns a new *MySQL5Dumper instance.

func (*MySQL5Dumper) Dump 

func (g *MySQL5Dumper) Dump(w io.Writer, db Database) error

Dump...

type MySQL5DumperTemplateValues 

type MySQL5DumperTemplateValues struct {
	ShouldDumpDatabase   bool
	ShouldDumpTables     bool
	ShouldDumpViews      bool
	ShouldDumpRoutines   bool
	ShouldDumpTriggers   bool
	Debug                bool
	AppName              string
	AppVersion           string
	DumpDate             string
	DumpDuration         string
	CharSet              string
	Collation            string
	OriginalDatabaseName string
	Database             Database
	Args                 *DumpArgs
	Connection           *ConnectionArgs
	Server               *Server
	Tables               TableGraph
	Views                ViewGraph
	Routines             RoutineGraph
}

MySQL5DumperTemplateValues...

type MySQL5PreparedStatements 

type MySQL5PreparedStatements struct {
	// contains filtered or unexported fields
}

MySQL5PreparedStatements...

func NewMySQL5PreparedStatements 

func NewMySQL5PreparedStatements(db *gosql.DB) *MySQL5PreparedStatements

NewMySQL5PreparedStatements returns a new *MySQL5PreparedStatements instance.

func (*MySQL5PreparedStatements) Err 

func (p *MySQL5PreparedStatements) Err() error

Err returns the last error.

func (*MySQL5PreparedStatements) Prepare 

func (p *MySQL5PreparedStatements) Prepare(name, sql string) error

Prepare...

func (*MySQL5PreparedStatements) Query 

func (p *MySQL5PreparedStatements) Query(name string, args ...interface{}) (*gosql.Rows, error)

Query...

type Routine 

type Routine struct {
	Name            string
	Type            string
	CreateSQL       string
	SecurityType    string
	Definer         string
	ParamList       string
	Returns         string
	IsDeterministic string
	SQLMode         string
	CharSet         string
	Collation       string
}

Routine...

type RoutineGraph 

type RoutineGraph []*Routine

type Row 

type Row []Field

type Rows 

type Rows []Row

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server...

func NewServer 

func NewServer(conn *ConnectionArgs, args *DumpArgs) *Server

NewServer returns a new *Server instance.

func (*Server) Close 

func (s *Server) Close() error

Close...

func (*Server) Database 

func (s *Server) Database(name string) (Database, error)

Database returns a new Database instance.

func (*Server) Open 

func (s *Server) Open() error

Open...

func (*Server) Variable 

func (s *Server) Variable(v string) (string, error)

Variable...

func (*Server) Version 

func (s *Server) Version() string

Version...

func (*Server) VersionNumber 

func (s *Server) VersionNumber() string

VersionNumber...

type Table 

type Table struct {
	Name        string
	CreateSQL   string
	CharSet     string
	Collation   string
	DebugMsgs   []string
	Columns     ColumnMap
	Constraints []*Constraint
	Triggers    TriggerGraph
	Rows        Rows
}

Table stores the details of a single database table.

func NewTable 

func NewTable() *Table

NewTable returns a new *Table instance.

func (*Table) AppendDebugMsg 

func (t *Table) AppendDebugMsg(s string, v ...interface{})

AppendDebugMsg appends a message to the debug messages.

type TableGraph 

type TableGraph []*Table

type Trigger 

type Trigger struct {
	Name              string
	CreateSQL         string
	ActionTiming      string
	EventManipulation string
	EventObjectTable  string
	ActionOrientation string
	Definer           string
	SQLMode           string
	CharSet           string
	Collation         string
}

Trigger...

type TriggerGraph 

type TriggerGraph []*Trigger

type View 

type View struct {
	Name         string
	Columns      ColumnMap
	CreateSQL    string
	CharSet      string
	Collation    string
	SecurityType string
	Definer      string
}

View...

type ViewGraph 

type ViewGraph []*View

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.