dgman

package module
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2019 License: Apache-2.0 Imports: 14 Imported by: 0

README

Build Status Coverage Status GoDoc

Dgman is a schema manager for Dgraph using the Go Dgraph client (dgo), which manages Dgraph schema and indexes from Go tags in struct definitions

Features

  • Create schemas and indexes from struct tags.
  • Detect conflicts from existing schema and defined schema.
  • Mutate Helpers (Create, Update, Upsert).
  • Autoinject node type from struct.
  • Field unique checking (e.g: emails, username).
  • Query helpers.
  • Delete helper.

Table of Contents

Installation

Using go get:

go get github.com/dolan-in/dgman

Usage

import(
	"github.com/dolan-in/dgman"
)

Schema Definition

Schemas are defined using Go structs which defines the predicate name from the json tag, indices and directives using the dgraph tag.

CreateSchema

Using the CreateSchema function, it will install the schema, and detect schema and index conflicts within the passed structs and with the currently existing schema in the specified Dgraph database.

// User is a node, nodes have a uid field
type User struct {
	UID      string     `json:"uid,omitempty"`
	Name     string     `json:"name,omitempty" dgraph:"index=term"` // use term index 
	Username string     `json:"username,omitempty" dgraph:"index=hash"` // use hash index
	Email    string     `json:"email,omitempty" dgraph:"index=hash upsert"` // use hash index, use upsert directive
	Password string     `json:"password,omitempty"`
	Height   *int       `json:"height,omitempty"`
	Dob      *time.Time `json:"dob,omitempty"` // will be inferred as dateTime schema type
	Status   EnumType   `json:"status,omitempty" dgraph="type=int"`
	Created  time.Time  `json:"created,omitempty" dgraph:"index=day"` // will be inferred as dateTime schema type, with day index
	Mobiles  []string   `json:"mobiles,omitempty"` // will be inferred as using the  [string] schema type, slices with primitive types will all be inferred as lists
	Schools  []School   `json:"schools,omitempty" dgraph:"count reverse"` // defines an edge to other nodes, add count index, add reverse edges
}

// School is another node, that will be connected to User node using the schools predicate
type School struct {
	UID      string 	`json:"uid,omitempty"`
	Name     string 	`json:"name,omitempty"`
	Location *GeoLoc 	`json:"location,omitempty" dgraph:"type=geo"` // for geo schema type, need to specify explicitly
}

type GeoLoc struct {
	Type  string    `json:"type"`
	Coord []float64 `json:"coordinates"`
}

func main() {
	d, err := grpc.Dial("localhost:9080", grpc.WithInsecure())
	if err != nil {
		panic(err)
	}

	c := dgo.NewDgraphClient(api.NewDgraphClient(d))

	// create the schema, 
	// it will only install non-existing schema in the specified database
	schema, err := dgman.CreateSchema(c, &User{})
	if err != nil {
		panic(err)
	}
	// Check the generated schema
	fmt.Println(schema)
}

On an empty database, the above code will return the generated schema string used to create the schema, logging the conflicting schemas in the process:

2018/12/14 02:23:48 conflicting schema name, already defined as "name: string @index(term) .", trying to define "name: string ."
username: string @index(hash) .
status: int .
created: dateTime @index(day) .
mobiles: [string] .
schools: uid @count @reverse .
user: string .
email: string @index(hash) @upsert .
password: string .
height: int .
dob: dateTime .
school: string .
location: geo .
name: string @index(term) .

When schema conflicts is detected with the existing schema already installed in the database, it will only log the differences. You would need to manually correct the conflicts by dropping or updating the schema manually.

MutateSchema

To overwrite/update index definitions, you can use the MutateSchema function, which will update the schema indexes.

// update the schema indexes
schema, err := dgman.MutateSchema(c, &User{})
if err != nil {
	panic(err)
}
// Check the generated schema
fmt.Println(schema)

Mutate Helpers

Mutate

Using the Mutate function, before sending a mutation, it will marshal a struct into JSON and injecting a node type, for easier labelling nodes, or in SQL it would refer to the table.

user := User{
	Name: "Alexander",
	Email: "alexander@gmail.com",
	Username: "alex123",
}

if err := dgman.Mutate(context.Background(), c.NewTxn(), &user, dgman.MutateOptions{CommitNow: true}); err != nil {
	panic(err)
}

// UID will be set
fmt.Println(user.UID)

The above will insert a node with the following JSON string, with the field "user":"" added in:

{"user":"","email":"alexander@gmail.com","username":"alex123"}
Node Types

Node types will be inferred from the struct name and converted into snake_case, so the User struct above would use user as its node type.

If you need to define a custom name for the node type, you can define the NodeType() string method on the struct.

type CustomNodeType struct {
	UID 	string `json:"uid,omitempty"`
	Name 	string `json:"name,omitempty"`
}

func (c CustomNodeType) NodeType() string {
	return "node_type"
}
Prefixed Node Types

To add a prefix for all subsequent node types, use SetTypePrefix, which will add a prefix, resulting a prefixed node type with the following format: prefix.node_type.

SetTypePrefix("type")

// node type becomes type.node_type
Create (Mutate with Unique Checking)

If you need unique checking for a particular field of a node with a certain node type, e.g: Email of users, you can use the Create function.

To define a field to be unique, add unique in the dgraph tag on the struct definition.

type User struct {
	UID 			string `json:"uid,omitempty"`
	Name 			string `json:"name,omitempty" dgraph:"index=term"`
	Email 		string `json:"email,omitempty" dgraph:"index=hash unique"`
	Username 	string `json:"username,omitempty" dgraph:"index=term unique"`
}

...
	user := User{
		Name: "Alexander",
		Email: "alexander@gmail.com",
		Username: "alex123",
	}

	if err := dgman.Create(context.Background(), c.NewTxn(), &user, dgman.MutateOptions{CommitNow: true}); err != nil {
		panic(err)
	}
	
	// try to create user with a duplicate email
	duplicateEmail := User{
		Name: "Alexander",
		Email: "alexander@gmail.com",
		Username: "alexa",
	}

	// will return a dgman.UniqueError
	if err := dgman.Create(context.Background(), c.NewTxn(), &duplicateEmail, dgman.MutateOptions{CommitNow: true}); err != nil {
		if uniqueErr, ok := err.(dgman.UniqueError); ok {
			// check the duplicate field
			fmt.Println(uniqueErr.Field, uniqueErr.Value)
		}
	}
Update (Mutate existing node with Unique Checking)

This is similar to Create, but for existing nodes. So the uid field must be specified.

type User struct {
	UID 			string 		`json:"uid,omitempty"`
	Name 			string 		`json:"name,omitempty"`
	Email 			string 		`json:"email,omitempty" dgraph:"index=hash unique"`
	Username 		string 		`json:"username,omitempty" dgraph:"index=term unique"`
	Dob			time.Time	`json:"dob" dgraph:"index=day"`
}

...
	users := []User{
		User{
			Name: "Alexander",
			Email: "alexander@gmail.com",
			Username: "alex123",
		},
		User{
			Name: "Fergusso",
			Email: "fergusso@gmail.com",
			Username: "fergusso123",
		},
	}

	if err := dgman.Create(context.Background(), c.NewTxn(), &users, dgman.MutateOptions{CommitNow: true}); err != nil {
		panic(err)
	}
	
	// try to update the user with existing username
	alexander := users[0]
	alexander.Username = "fergusso123"
	// UID should have a value
	fmt.Println(alexander.UID)

	// will return a dgman.UniqueError
	if err := dgman.Update(context.Background(), c.NewTxn(), &alexander, dgman.MutateOptions{CommitNow: true}); err != nil {
		if uniqueErr, ok := err.(dgman.UniqueError); ok {
			// will return duplicate error for username
			fmt.Println(uniqueErr.Field, uniqueErr.Value)
		}
	}

	// try to update the user with non-existing username
	alexander.Username = "wildan"

	if err := dgman.Update(context.Background(), c.NewTxn(), &alexander, dgman.MutateOptions{CommitNow: true}); err != nil {
		panic(err)
	}

	// should be updated
	fmt.Println(alexander)
Update (Mutate existing node with Unique Checking)

This is similar to Create, but for existing nodes. So the uid field must be specified.

type User struct {
	UID 			string 		`json:"uid,omitempty"`
	Name 			string 		`json:"name,omitempty"`
	Email 			string 		`json:"email,omitempty" dgraph:"index=hash unique"`
	Username 		string 		`json:"username,omitempty" dgraph:"index=term unique"`
	Dob			time.Time	`json:"dob" dgraph:"index=day"`
}

...
	users := []User{
		User{
			Name: "Alexander",
			Email: "alexander@gmail.com",
			Username: "alex123",
		},
		User{
			Name: "Fergusso",
			Email: "fergusso@gmail.com",
			Username: "fergusso123",
		},
	}

	if err := dgman.Create(context.Background(), c.NewTxn(), &users, dgman.MutateOptions{CommitNow: true}); err != nil {
		panic(err)
	}
	
	// try to update the user with existing username
	alexander := users[0]
	alexander.Username = "fergusso123"
	// UID should have a value
	fmt.Println(alexander.UID)

	// will return a dgman.UniqueError
	if err := dgman.Update(context.Background(), c.NewTxn(), &alexander, dgman.MutateOptions{CommitNow: true}); err != nil {
		if uniqueErr, ok := err.(dgman.UniqueError); ok {
			// will return duplicate error for username
			fmt.Println(uniqueErr.Field, uniqueErr.Value)
		}
	}

	// try to update the user with non-existing username
	alexander.Username = "wildan"

	if err := dgman.Update(context.Background(), c.NewTxn(), &alexander, dgman.MutateOptions{CommitNow: true}); err != nil {
		panic(err)
	}

	// should be updated
	fmt.Println(alexander)
Upsert

Upsert inserts node(s) if it does not violate any unique key, otherwise update the node.

type User struct {
	UID 			string 		`json:"uid,omitempty"`
	Name 			string 		`json:"name,omitempty"`
	Email 			string 		`json:"email,omitempty" dgraph:"index=hash unique"`
	Username 		string 		`json:"username,omitempty" dgraph:"index=term unique"`
	Dob			time.Time	`json:"dob" dgraph:"index=day"`
}

...
	users := []User{
		User{
			Name: "Alexander",
			Email: "alexander@gmail.com",
			Username: "alex123",
		},
		User{
			Name: "Fergusso",
			Email: "fergusso@gmail.com",
			Username: "fergusso123",
		},
	}

	if err := dgman.Upsert(context.Background(), c.NewTxn(), &users, dgman.MutateOptions{CommitNow: true}); err != nil {
		panic(err)
	}
Create Or Get

CreateOrGet will create any nodes that does not violate any keys, otherwise just get the existing node.

Update On Conflict

UpdateOnConflict is the base for Upsert and CreateOrGet, which defines a callback when there is a unique key conflict.

users := []*User{
	&User{
		Name:     "ajiba",
		Username: "wildanjing",
		Email:    "wildan2711@gmail.com",
	},
	&User{
		Name:     "PooDiePie",
		Username: "wildansyah",
		Email:    "wildanodol2711@gmail.com",
	},
	&User{
		Name:     "lalap",
		Username: "lalap",
		Email:    "lalap@gmail.com",
	},
}

tx = c.NewTxn()

cb := func(uniqueErr UniqueError, found, excluded interface{}) interface{} {
	switch uniqueErr.Field {
	case "email":
		f := found.(*TestUnique)
		e := excluded.(*TestUnique)
		// just modify the username when email found
		f.Username = e.Username
		return found
	}
	// return nil to not update the node
	return nil
}
if err := UpdateOnConflict(context.Background(), tx, &testDuplicate, cb); err != nil {

}

Query Helpers

Get by UID
// Get by UID
user := User{}
if err := dgman.Get(ctx, tx, &user).UID("0x9cd5").Node(); err != nil {
	if err == dgman.ErrNodeNotFound {
		// node not found
	}
}

// struct will be populated if found
fmt.Println(user)
Get by Filter
user := User{}
// get node with node type `user` that matches filter
err := dgman.Get(ctx, tx, &user).
	Vars("getUser($name: string)", map[string]string{"$name": "wildan"}). // function defintion and Graphql variables
	Filter("allofterms(name, $name)"). // dgraph filter
	All(1). // returns all predicates, expand on 1 level of edge predicates
	Node() // get single node from query
if err != nil {
	if err == dgman.ErrNodeNotFound {
		// node using the specified filter not found
	}
}

// struct will be populated if found
fmt.Println(user)
Get by query
users := []User{}
query := `@filter(allofterms(name, $name)) {
	uid
	expand(_all_) {
		uid
		expand(_all_)
	}
}`
// get nodes with node type `user` that matches filter
err := dgman.Get(ctx, tx, &users).
	Vars("getUsers($name: string)", map[string]string{"$name": "wildan"}). // function defintion and Graphql variables
	Query(query). // dgraph query portion (without root function)
	OrderAsc("name"). // ordering ascending by predicate
	OrderDesc("dob"). // multiple ordering is allowed
	First(10). // get first 10 nodes from result
	Nodes() // get all nodes from the prepared query
if err != nil {
}

// slice will be populated if found
fmt.Println(users)
Get by filter query

You can also combine Filter with Query.

users := []User{}
// get nodes with node type `user` that matches filter
err := dgman.Get(ctx, tx, &users).
	Vars("getUsers($name: string)", map[string]string{"$name": "wildan"}). // function defintion and Graphql variables
	Filter("allofterms(name, $name)").
	Query(`{
		uid
		expand(_all_) {
			uid
			expand(_all_)
		}
	}`). // dgraph query portion (without root function)
	OrderAsc("name"). // ordering ascending by predicate
	OrderDesc("dob"). // multiple ordering is allowed
	First(10). // get first 10 nodes from result
	Nodes() // get all nodes from the prepared query
if err != nil {
}

// slice will be populated if found
fmt.Println(users)

Delete Helper

Delete Nodes

Delete helpers can be used to simplify deleting nodes that matches a query, using the same query format as Query Helpers.

query := `@filter() {
	uid
	expand(_all_) {
		uid
	}
}`
// delete all nodes with node type `user` that matches query
// all edge nodes that are specified in the query will also be deleted
deletedUids, err := dgman.Delete(ctx, tx, &User{}, dgman.MutateOptions{CommitNow: true}).
	Vars("getUsers($name: string)", map[string]string{"$name": "wildan"}). // function defintion and Graphql variables
	Query(query). // dgraph query portion (without root function)
	OrderAsc("name"). // ordering ascending by predicate
	OrderDesc("dob"). // multiple ordering is allowed
	First(10). // get first 10 nodes from result
	Nodes() // delete all nodes from the prepared query
if err != nil {
}

// check the deleted uids
fmt.Println(deletedUids)
Delete Edges

For deleting edges, you only need to specify node UID, edge predicate, and edge UIDs

err := dgman.Delete(ctx, tx, &User{}, dgman.MutateOptions{CommitNow: true}).
	Edge("0x12", "schools", "0x13", "0x14")

If no edge UIDs are specified, all edges of the specified predicate will be deleted.

err := dgman.Delete(ctx, tx, &User{}, dgman.MutateOptions{CommitNow: true}).
	Edge("0x12", "schools")

TODO

  • Filter generator

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrNodeNotFound = errors.New("node not found")
)

Functions

func Create 

func Create(ctx context.Context, tx *dgo.Txn, data interface{}, options ...MutateOptions) error

Create create node(s) with field unique checking

func CreateOrGet 

func CreateOrGet(ctx context.Context, tx *dgo.Txn, data interface{}, options ...MutateOptions) error

CreateOrGet creates a node, or returns a node if it exists with a certain unique key

func GetNodeType 

func GetNodeType(data interface{}) string

GetNodeType gets node type from NodeType() method of Node interface if it doesn't implement it, get it from the struct name and convert to snake case

func Mutate 

func Mutate(ctx context.Context, tx *dgo.Txn, data interface{}, options ...MutateOptions) error

Mutate is a shortcut to create mutations from data to be marshalled into JSON, it will inject the node type from the Struct name converted to snake_case

func Node 

func Node(jsonData []byte, model interface{}) error

Node marshals a single node to a single object of model, returns error if no nodes are found, query root must be data(func ...)

func Nodes 

func Nodes(jsonData []byte, model interface{}) error

Nodes marshals multiple nodes to a slice of model, query root must be data(func ...)

func SetTypePrefix 

func SetTypePrefix(prefix string)

SetTypePrefix sets the global node type prefix, with the following format: prefix.node_type

func Update 

func Update(ctx context.Context, tx *dgo.Txn, data interface{}, options ...MutateOptions) error

Update updates a node by their UID with field unique checking

func UpdateOnConflict 

func UpdateOnConflict(ctx context.Context, tx *dgo.Txn, data interface{}, cb OnConflictCallback, options ...MutateOptions) error

UpdateOnConflict updates a node

func Upsert 

func Upsert(ctx context.Context, tx *dgo.Txn, data interface{}, options ...MutateOptions) error

Upsert updates the node when a node with a certain unique key exists

Types

type Deleter 

type Deleter struct {
	// contains filtered or unexported fields
}

func Delete 

func Delete(ctx context.Context, tx *dgo.Txn, model interface{}, opt ...MutateOptions) *Deleter

Delete prepares a delete mutation using a query

func (*Deleter) After 

func (d *Deleter) After(uid string) *Deleter

func (*Deleter) All 

func (d *Deleter) All(depthParam ...int) *Deleter

All returns expands all predicates, with a depth parameter that specifies how deep should edges be expanded

func (*Deleter) Edge 

func (d *Deleter) Edge(uid, edgePredicate string, edgeUIDs ...string) error

Edge delete edges of a node of specified edge predicate, if no edgeUIDs specified, delete all edges

func (*Deleter) Filter 

func (d *Deleter) Filter(filter string) *Deleter

func (*Deleter) First 

func (d *Deleter) First(n int) *Deleter

func (*Deleter) Node 

func (d *Deleter) Node() (uids []string, err error)

Node deletes the first single root node from the query including edge nodes that may be specified on the query

func (*Deleter) Nodes 

func (d *Deleter) Nodes() (uids []string, err error)

Nodes deletes all nodes matching the delete query including edge nodes that may be specified on the query

func (*Deleter) Offset 

func (d *Deleter) Offset(n int) *Deleter

func (*Deleter) OrderAsc 

func (d *Deleter) OrderAsc(clause string) *Deleter

func (*Deleter) OrderDesc 

func (d *Deleter) OrderDesc(clause string) *Deleter

func (*Deleter) Query 

func (d *Deleter) Query(query string) *Deleter

func (*Deleter) RootFunc 

func (d *Deleter) RootFunc(rootFunc string) *Deleter

RootFunc modifies the dgraph query root function, if not set, the default is "has(node_type)"

func (*Deleter) String 

func (d *Deleter) String() string

func (*Deleter) UID 

func (d *Deleter) UID(uid string) *Deleter

UID returns the node with the specified uid

func (*Deleter) Vars 

func (d *Deleter) Vars(funcDef string, vars map[string]string) *Deleter

Vars specify the GraphQL variables to be passed on the query, by specifying the function definition of vars, and variable map. Example funcDef: getUserByEmail($email: string, $age: number)

type MultiError 

type MultiError struct {
	Errors []error
}

func (*MultiError) Collect 

func (e *MultiError) Collect(err error)

func (*MultiError) Error 

func (e *MultiError) Error() (errorStr string)

type MutateOptions 

type MutateOptions struct {
	DisableInject bool
	CommitNow     bool
}

MutateOptions specifies options for mutating

type NodeType 

type NodeType interface {
	NodeType() string
}

NodeType is an interface for passing node types

type NodeUnique 

type NodeUnique interface {
	UniqueKeys() []string
}

type NotNullError 

type NotNullError struct {
	Field string
}

func (NotNullError) Error 

func (n NotNullError) Error() string

type OnConflictCallback 

type OnConflictCallback func(uniqueErr UniqueError, found, excluded interface{}) (updated interface{})

OnConflictCallback defines the callback to update the data when an existing node matches the unique key

type Query 

type Query struct {
	// contains filtered or unexported fields
}

func Get 

func Get(ctx context.Context, tx *dgo.Txn, model interface{}) *Query

Get prepares a query for a model

func (*Query) After 

func (q *Query) After(uid string) *Query

func (*Query) All 

func (q *Query) All(depthParam ...int) *Query

All returns expands all predicates, with a depth parameter that specifies how deep should edges be expanded

func (*Query) Filter 

func (q *Query) Filter(filter string) *Query

Filter defines a query filter, return predicates at the first depth

func (*Query) First 

func (q *Query) First(n int) *Query

func (*Query) Node 

func (q *Query) Node() (err error)

Node returns the first single node from the query

func (*Query) Nodes 

func (q *Query) Nodes() error

Nodes returns all results from the query

func (*Query) Offset 

func (q *Query) Offset(n int) *Query

func (*Query) OrderAsc 

func (q *Query) OrderAsc(clause string) *Query

func (*Query) OrderDesc 

func (q *Query) OrderDesc(clause string) *Query

func (*Query) Query 

func (q *Query) Query(query string) *Query

Query defines the query portion other than the root function

func (*Query) RootFunc 

func (q *Query) RootFunc(rootFunc string) *Query

RootFunc modifies the dgraph query root function, if not set, the default is "has(node_type)"

func (*Query) String 

func (q *Query) String() string

func (*Query) UID 

func (q *Query) UID(uid string) *Query

UID returns the node with the specified uid

func (*Query) Vars 

func (q *Query) Vars(funcDef string, vars map[string]string) *Query

Vars specify the GraphQL variables to be passed on the query, by specifying the function definition of vars, and variable map. Example funcDef: getUserByEmail($email: string)

type Schema 

type Schema struct {
	Predicate string
	Type      string
	Index     bool
	Tokenizer []string
	Reverse   bool
	Count     bool
	List      bool
	Upsert    bool
	Unique    bool
	NotNull   bool
}

func (Schema) String 

func (s Schema) String() string

type SchemaMap 

type SchemaMap map[string]*Schema

SchemaMap maps the underlying schema defined for a predicate

func CreateSchema 

func CreateSchema(c *dgo.Dgraph, models ...interface{}) (*SchemaMap, error)

CreateSchema generate indexes and schema from struct models, returns the created schemap.

func MutateSchema 

func MutateSchema(c *dgo.Dgraph, models ...interface{}) (*SchemaMap, error)

MutateSchema generate indexes and schema from struct models, attempt updates for schema and indexes.

func (SchemaMap) String 

func (s SchemaMap) String() string

type UniqueError 

type UniqueError struct {
	NodeType string
	Field    string
	Value    interface{}
}

UniqueError returns the field and value that failed the unique node check

func (UniqueError) Error 

func (u UniqueError) Error() string

Source Files

View all Source files

Directories

Path Synopsis
examples
mutate
schema

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.