dsutil - Google Cloud DataStore import/export utilities

Installing
go get github.com/rustyx/dsutil
Command Line Usage
dsutil [options] command <args>
command:
export <filename> - export records from DataStore
import <filename>... - import records into DataStore
delete - delete records from DataStore
set <field> <type> <value> - update records in DataStore (type is: string, int, double)
convert <in> <out> - convert exported records from JSON to Go object notation
Note: <filename> ending with ".gz" will be automatically g(un)zipped
-project string
Google Cloud project name (deduced if not provided)
-kind string
DataStore table name (required for export)
-filter string
Filter field name (optional)
-from string
Filter >= value (optional)
-to string
Filter < value (optional)
-eq string
Filter = value (optional)
API Usage
It is possible to read an export file and process each entity programmatically.
There are two interfaces: ImportFile
, based on key-value pairs, and ImportFileReflect
, which is useful for ORM. Here's an example of how to load an export file into a PostgreSQL database using go-pg
ORM API:
type MyEntity struct {
Id int `datastore:"-"`
SomeColumn string `pg:"type:varchar(40)"`
SomeColumn2 string `pg:"type:varchar(40)"`
// . . .
}
inputFile := "my-export.ds"
log.Printf("Importing %v", inputFile)
insertFunc := func(kind string, rows []interface{}) error {
log.Printf("Inserting %v %s(s)", len(rows), kind)
_, err := pgdb.Model(rows...).Insert()
return err
}
modelMap := []dsio.ModelMapping{
{Kind: "MyEntity", TypePtr: &MyEntity{}, ImportFunc: insertFunc, BatchSize: 200},
}
if err := dsio.ImportFileReflect(inputFile, modelMap); err != nil {
log.Fatalf("import %v failed: %v", inputFile, err)
}
For more documentation refer to API Docs.