elk

README

elk

This package aims to extend the awesome entgo.io code generator to generate fully functional code on a defined set of entities.

⚠ This is work in progress: The API may change without further notice!

Features

• Generate http crud handlers
• Generate flutter models and http client to consume the generated http api

How to use

1. Create a new Go file named ent/elk.go, and paste the following content:

// +build ignore

package main

import (
	"log"

	"entgo.io/ent/entc"
	"entgo.io/ent/entc/gen"
	"github.com/masseelch/elk"
)

func main() {
	// ent plus http
	err := entc.Generate("./schema", &gen.Config{
		Templates: elk.HTTPTemplates,
		Hooks: []gen.Hook{
			elk.AddGroupsTag,
		},
	})
	if err != nil {
		log.Fatalf("running ent codegen: %v", err)
	}
	// flutter
	if err := elk.Flutter("./schema", ""); err != nil {
		log.Fatalf("running flutter codegen: %v", err)
	}
}

2. Edit the ent/generate.go file to execute the ent/elk.go file:

package ent

//go:generate go run -mod=mod elk.go

3. Run codegen for your ent project:

go generate ./...

---

Generate fully working Go CRUD HTTP API with Ent

Introduction

One of the major time consumers when setting up a new API is setting up the basic CRUD (Create, Read, Update, Delete) operations that repeat itself for every new entity you add to your graph. Luckily there is an extension to the ent framework aiming to provide such handlers, including level logging, validation of the request body, eager loading relations and serializing, all while leaving reflection out of sight and maintaining type-safety: elk. Let’s dig in!

Setting up elk

First make sure you have the latest release of elk installed in your project:

go get github.com/masseelch/elk

The next step is to enable the elk extension. This requires you to use entc (enc codegen) package as described here. Follow the next 3 steps to enable it and tell the generator to execute the elk templates:

1. Create a new Go file named ent/entc.go, and paste the following content:

// +build ignore

package main

import (
	"entgo.io/ent/entc"
	"entgo.io/ent/entc/gen"
	"github.com/masseelch/elk"
	"log"
)

func main() {
	ex, err := elk.NewExtension()
	if err != nil {
		log.Fatalf("creating elk extension: %v", err)
	}
	err = entc.Generate("./schema", &gen.Config{}, entc.Extensions(ex))
	if err != nil {
		log.Fatalf("running ent codegen: %v", err)
	}
}

1. Edit the ent/generate.go file to execute the ent/entc.go file:

package ent

//go:generate go run -mod=mod entc.go

1. Run codegen for your ent project:

go generate./...

Since now all is set up create a schema, add some data and make use of elk-empowered ent!

Setting up a simple server

To show you what elk can do for you, we use the schema and data ent described in its docs. Head over there and create the schema as mentioned. You should end up with a graph like that below:

The generated handlers use go-chi to parse path and query parameters. However the handlers implement net/https HandleFunc interface and therefore seamlessly integrate in most existing apis.
Furthermore elk uses zap for logging and go-playgrounds validator to validate create / update request bodies. Rendering is done by sheriff and render. To hook up our api with the generated handlers add the following file:

// main.go
package main

import (
	"<project>/ent"
	elk "<project>/ent/http"
	"context"
	"github.com/go-chi/chi/v5"
	"github.com/go-playground/validator/v10"
	_ "github.com/mattn/go-sqlite3"
	"go.uber.org/zap"
	"log"
	"net/http"
)

func main() {
	// Create the ent client.
	c, err := ent.Open("sqlite3", "file:ent?mode=memory&cache=shared&_fk=1")
	if err != nil {
		log.Fatalf("failed opening connection to sqlite: %v", err)
	}
	defer c.Close()
	// Run the auto migration tool.
	if err := c.Schema.Create(context.Background()); err != nil {
		log.Fatalf("failed creating schema resources: %v", err)
	}
	// Create a zap logger to use.
	l, err := zap.NewProduction()
	if err != nil {
		log.Fatalf("failed creating logger: %v", err)
	}
	// Validator used by elks handlers.
	v := validator.New()
	// Create a router.
	r := chi.NewRouter()
	// Hook up our generated handlers.
	r.Route("/pets", func(r chi.Router) {
		elk.NewPetHandler(c, l, v).Mount(r, elk.PetRoutes)
	})
	r.Route("/users", func(r chi.Router) {
		// We dont allow user deletion.
		elk.NewUserHandler(c, l, v).Mount(r, elk.PetRoutes &^ elk.UserDelete)
	})
	r.Route("/groups", func(r chi.Router) {
		// Dont include sub-resource routes.
		elk.NewGroupHandler(c, l, v).Mount(r, elk.GroupCreate | elk.GroupRead | elk.GroupUpdate | elk.GroupDelete | elk.GroupList)
	})
	// Start listen to incoming requests.
	if err := http.ListenAndServe(":8080", r); err != nil {
		log.Fatal(err)
	}
}

You can find a ready to be copied example here.

Examples

You find an extensive list of examples of elks capabilities below.

List a resource

elk provides endpoints to list a resource. Pagination is already set up.

curl 'localhost:8080/pets?itemsPerPage=2&page=2'

[
  {
    "id": 3,
    "name": "Coco",
    "edges": {}
  }
]

Read a resource

To get detailed information about a resource set a path parameter.

curl 'localhost:8080/pets/3

  {
  "id": 3,
  "name": "Coco",
  "edges": {}
}

Create a resource

To create a new resource send an POST request with application/json encoded body.

curl -X 'POST' -H 'Content-Type: application/json' -d '{"name":"Bob","owner":2}' 'localhost:8080/pets'

{
  "id": 4,
  "name": "Bob",
  "edges": {}
}

Update a resource

To update a resources property send an PATCH request with application/json encoded body.

curl -X 'PATCH' -H 'Content-Type: application/json' -d '{"name":"Bobs Changed Name"}' 'localhost:8080/pets/4'

{
  "id": 4,
  "name": "Bobs Changed Name",
  "edges": {}
}

Delete a resource

The handlers return a 204 response.

curl -X 'DELETE' 'localhost:8080/pets/4'

Request validation

elk can validate data sent in POST or PATCH requests. Use the elk.Annotation to set validation rules on fields. Head over to go-playgrounds validator to see what validation rules exist.

// ent/schema/user.go

// Fields of the User.
func (User) Fields() []ent.Field {
	return []ent.Field{
		field.Int("age"),
		field.String("name").
			Annotations(elk.Annotation{
				// No numbers allowed in name and it has to be at least 3 chars long.
				CreateValidation: "alpha,min=3",
				UpdateValidation: "alpha,min=3",
		}),
	}
}

curl -X 'POST' -H 'Content-Type: application/json' -d '{"name":"A"}' 'localhost:8080/users'

{
  "code": 400,
  "status": "Bad Request",
  "errors": {
    "name": "This value failed validation on 'min:3'."
  }
}

Error responses

You get meaningful error responses.

curl -X 'POST' -H 'Content-Type: application/json' -d 'foo bar wtf' 'localhost:8080/pets'

{
  "code": 400,
  "status": "Bad Request",
  "errors": "invalid json string"
}

Subresource routes

elk provides endpoints to fetch a resources edges.

curl 'localhost:8080/users/2/pets'

[
  {
    "id": 1,
    "name": "Pedro",
    "edges": {}
  },
  {
    "id": 2,
    "name": "Xabi",
    "edges": {}
  },
  {
    "id": 4,
    "name": "Bob",
    "edges": {}
  }
]

Eager load edges

You can tell elk to eager load edges on specific routes by the use of serialization groups. Use elk.SchemaAnnotation to define what groups to load on what endpoint and elk.Annotation on fields and edges to tell the serializer what fields and edges are included in which group. elk takes care of eager loading the correct nodes.

// ent/schema/pet.go

package schema

import (
	"entgo.io/ent"
	"entgo.io/ent/schema"
	"entgo.io/ent/schema/edge"
	"entgo.io/ent/schema/field"
	"github.com/masseelch/elk"
)

// Pet holds the schema definition for the Pet entity.
type Pet struct {
	ent.Schema
}

// Fields of the Pet.
func (Pet) Fields() []ent.Field {
	return []ent.Field{
		field.String("name").
			Annotations(elk.Annotation{
				// Include the name on the "pet:list" group.
				Groups: []string{"pet:list"},
			}),
	}
}

// Edges of the Pet.
func (Pet) Edges() []ent.Edge {
	return []ent.Edge{
		edge.To("friends", Pet.Type),
		edge.From("owner", User.Type).
			Ref("pets").
			Unique().
			Annotations(elk.Annotation{
				// Include the owner on the "pet:list" group.
				Groups: []string{"pet:list"},
			}),
	}
}

// Annotations of the Pet.
func (Pet) Annotations() []schema.Annotation {
	return []schema.Annotation{
		elk.SchemaAnnotation{
			// Tell elk to use the "pet:list" group on list routes.
			ListGroups: []string{"pet:list"},
		},
	}
}

curl 'localhost:8080/pets'

[
  {
    "id": 1,
    "name": "Pedro",
    "edges": {
      "owner": {
        "id": 2,
        "age": 30,
        "name": "Ariel",
        "edges": {}
      }
    }
  },
  {
    "id": 2,
    "name": "Xabi",
    "edges": {
      "owner": {
        "id": 2,
        "age": 30,
        "name": "Ariel",
        "edges": {}
      }
    }
  },
  {
    "id": 3,
    "name": "Coco",
    "edges": {
      "owner": {
        "id": 3,
        "age": 37,
        "name": "Alex",
        "edges": {}
      }
    }
  }
]

Skip handlers

elk does always generate all handlers. You can declare what routes to mount.

elk.NewPetHandler(c, l, v).Mount(r, elk.PetCreate | elk.PetList | elk.PetRead)

The compiler will not include the unused handlers since they are never called.

Logging

elk does leveled logging with zap. See the example output below.

2021-07-22T07:22:25.436+0200    INFO    http/create.go:167      pet rendered    {"handler": "PetHandler", "method": "Create", "id": 4}
2021-07-22T07:22:25.450+0200    INFO    http/create.go:198      validation failed       {"handler": "UserHandler", "method": "Create", "error": "Key: 'UserCreateRequest.name' Error:Field validation for 'name' failed on the 'min' tag"}
2021-07-22T07:22:25.463+0200    INFO    http/update.go:239      validation failed       {"handler": "UserHandler", "method": "Update", "error": "Key: 'UserUpdateRequest.name' Error:Field validation for 'name' failed on the 'min' tag"}
2021-07-22T07:22:25.489+0200    INFO    http/create.go:254      user rendered   {"handler": "UserHandler", "method": "Create", "id": 4}
2021-07-22T07:22:25.508+0200    INFO    http/read.go:150        user rendered   {"handler": "UserHandler", "method": "Read", "id": 2}

Future and Known Issues

elk has many cool features already, but there are some issues to address in the near future.

Currently, elk does use this render package on combination with sheriff to render its output to the client. render does use reflection under the hood since it calls json.Marshal / xml.Marshal, as well does sheriff. The mapping of request values does currently only work for application/json bodies and uses json.Unmarshal. The goal is to have elk provide interfaces Renderer and Binder which will be implemented by the generated nodes / request structs. This allows type safe and reflection-free transformation between json / xml / protobuf and go structs.

ent already has some builtin validation which is not yet reflected by the request validation elk generates. Validation is only executed if there are tags given.

elks generated bitmask to choose what handlers to mount are not typesafe yet.

Another not yet implemented feature is to give the developer the possibility to customize the generated code by providing custom templates to elk like ent already does with External Templates.

In the future elk is meant to provide fully working and easily extendable integration tests for the generated code.

Initial work to generate a fully working flutter frontend has been done and will hopefully lead to a release soon.

Documentation

Index

• Variables
• func AddGroupsTag(next gen.Generator) gen.Generator
• type Annotation
  • func (a *Annotation) Decode(o interface{}) error
  • func (a *Annotation) EnsureDefaults()
  • func (Annotation) Name() string
  • func (a Annotation) ValidationTags(action string) string
• type Extension
  • func NewExtension(opts ...ExtensionOption) (*Extension, error)
  • func (e *Extension) Hooks() []gen.Hook
  • func (e *Extension) Templates() []*gen.Template
• type ExtensionOption
• type Groups
  • func (gs *Groups) Add(g ...string)
  • func (gs Groups) HasGroup(g string) bool
  • func (gs Groups) Match(other Groups) bool
  • func (gs Groups) StructTag() string
• type SchemaAnnotation
  • func (a *SchemaAnnotation) Decode(o interface{}) error
  • func (SchemaAnnotation) Name() string

Variables

var (
	// HTTPTemplates holds all templates for generating http handlers.
	HTTPTemplates = []*gen.Template{
		parse("template/http/handler.tmpl"),
		parse("template/http/helpers.tmpl"),
		parse("template/http/create.tmpl"),
		parse("template/http/read.tmpl"),
		parse("template/http/update.tmpl"),
		parse("template/http/delete.tmpl"),
		parse("template/http/list.tmpl"),
		parse("template/http/relations.tmpl"),
	}
	// TemplateFuncs contains the extra template functions used by elk.
	TemplateFuncs = template.FuncMap{
		"edgesToLoad":    edgesToLoad,
		"kebab":          strcase.KebabCase,
		"stringSlice":    stringSlice,
		"validationTags": validationTags,
	}
)

Functions

func AddGroupsTag

func AddGroupsTag(next gen.Generator) gen.Generator

AddGroupsTag adds the serialization groups defined by the annotation of each field to the generated entity struct.

Types

type Annotation

type Annotation struct {
	// Skip tells the generator to skip this field / edge.
	Skip bool `json:"Skip,omitempty"`
	// Groups holds the serialization groups to use on this field / edge.
	Groups Groups `json:"Groups,omitempty"`
	// MaxDepth tells the generator the maximum depth of this field when there is a cycle possible.
	MaxDepth uint
	// Validation holds the struct tags to use for github.com/go-playground/validator/v10. Used when no specific
	// validation tags are given in CreateValidation or UpdateValidation.
	Validation string
	// CreateValidation holds the struct tags to use for github.com/go-playground/validator/v10
	// when creating a new model.
	CreateValidation string
	// UpdateValidation holds the struct tags to use for github.com/go-playground/validator/v10
	// when updating an existing model.
	UpdateValidation string
}

Annotation annotates fields and edges with metadata for templates.

func (*Annotation) Decode

func (a *Annotation) Decode(o interface{}) error

Decode from ent.

func (*Annotation) EnsureDefaults

func (a *Annotation) EnsureDefaults()

EnsureDefaults ensures defaults are set.

func (Annotation) Name

func (Annotation) Name() string

Name implements ent.Annotation interface.

func (Annotation) ValidationTags

func (a Annotation) ValidationTags(action string) string

ValidationTags returns the tags to use for the given action.

type Extension

type Extension struct {
	entc.DefaultExtension
	// contains filtered or unexported fields
}

Extension implements entc.Extension interface for providing http handler code generation.

func NewExtension

func NewExtension(opts ...ExtensionOption) (*Extension, error)

func (*Extension) Hooks

func (e *Extension) Hooks() []gen.Hook

func (*Extension) Templates

func (e *Extension) Templates() []*gen.Template

type ExtensionOption

type ExtensionOption func(*Extension) error

ExtensionOption allows to manage Extension configuration using functional arguments.

type Groups

type Groups []string

Groups are used to determine what properties to load and serialize.

func (*Groups) Add

func (gs *Groups) Add(g ...string)

Add adds a group to the Groups. If the group is already present it does nothing.

func (Groups) HasGroup

func (gs Groups) HasGroup(g string) bool

HasGroup checks if the given group is present.

func (Groups) Match

func (gs Groups) Match(other Groups) bool

Match check if at least one of the given Groups is present.

func (Groups) StructTag

func (gs Groups) StructTag() string

StructTag returns the struct tag representation of the Groups.

type SchemaAnnotation

type SchemaAnnotation struct {
	// Skip tells the generator to skip this model.
	Skip bool `json:"Skip,omitempty"`
	// CreateGroups holds the serializations Groups to use on the create handler.
	CreateGroups Groups `json:"CreateGroups,omitempty"`
	// ReadGroups holds the serializations Groups to use on the read handler.
	ReadGroups Groups `json:"ReadGroups,omitempty"`
	// UpdateGroups holds the serializations Groups to use on the update handler.
	UpdateGroups Groups `json:"UpdateGroups,omitempty"`
	// DeleteGroups holds the serializations Groups to use on the delete handler.
	DeleteGroups Groups `json:"DeleteGroups,omitempty"`
	// ListGroups holds the serializations Groups to use on the list handler.
	ListGroups Groups `json:"ListGroups,omitempty"`
}

SchemaAnnotation annotates an entity with metadata for templates.

func (*SchemaAnnotation) Decode

func (a *SchemaAnnotation) Decode(o interface{}) error

Decode from ent.

func (SchemaAnnotation) Name

func (SchemaAnnotation) Name() string

Name implements ent.Annotation interface.