atlas-app-toolkit

README

Atlas Application Toolkit

This repository provides common Go utilities and helpers that are reusable from project-to-project. The goal is to prevent code duplication by encouraging teams to use and contribute to toolkit libraries.

The toolkit is not a framework. Rather, it is a set of (mostly gRPC-related) plugins and helpers.

Background

The toolkit's approach is based on the following assumptions.

• An application is composed of one or more independent services (micro-service architecture)
• Each independent service uses gRPC
• The REST API is presented by a separate service (gRPC Gateway) that serves as a reverse-proxy and forwards incoming HTTP requests to gRPC services

To get started with the toolkit, check out the Atlas CLI repository. The Atlas CLI's "bootstrap command" can generate new applications that make use of the toolkit. For Atlas newcomers, this is a great way to get up-to-speed with toolkit best practices.

Toolkit Libraries

The following libraries have how-to guides included at the package level.

Request Handling Tools

requestid - gets the request ID from incoming requests (or creates a unique ID if it doesn't exist)

query - provides query parameter-specific helpers, like sorting, paging, and filtering resources

auth - includes authorization and authentication-related helpers

rpc - provides additional messages that can be utilized in your proto definitions

errors - helps developers create detailed error messages or return multiple error messages

Server Utilities

server - provides a wrapper utility that manages a gRPC server and its REST gateway as a single unit

gateway - creates a gRPC gateway with built-in REST syntax compliancy

health - helps developers add health and readiness checks to their gRPC services

Database Utilities

gorm - offers a set of utilities for GORM library

Testing

integration - provides a set of utilities that help manage integration testing

Core Concepts

If you are new to the gRPC world, the following resources will be helpful to you.

REST API Syntax Specification

To ensure that public REST API endpoints are consistent, the toolkit enforces API syntax requirements that are common for all applications at Infoblox. For more information about making your syntax-compliant (e.g. for error handling), see the errors, gateway, and query packages.

gRPC Protobuf

See official documentation for Protocol Buffer and for gRPC

As an alternative you may use this plugin to generate Golang code. That is the same as official plugin but with gadgets.

gRPC Gateway

See official documentation

gRPC Interceptors

One of the requirements to the API Toolkit is to support a Pipeline model. We recommend to use gRPC server interceptor as middleware. See examples

Example Toolkit Application

An example app that is based on api-toolkit can be found here

Related Tools

The following are toolkit-recommended utilities that are maintained in separate repositories.

Validation

We recommend to use this validation plugin to generate Validate method for your gRPC requests.

As an alternative you may use this plugin too.

Validation can be invoked "automatically" if you add this middleware as a gRPC server interceptor.

Database Migrations

The toolkit does not require any specific method for database provisioning and setup. However, if golang-migrate or the infobloxopen fork of it is used, a couple helper functions are provided here for verifying that the database version matches a required version without having to import the entire migration package.

Documentation

We recommend to use this plugin to generate documentation.

Documentation can be generated in different formats.

Here are several most used instructions used in documentation generation:

Leading comments

Leading comments can be used everywhere.

/**
 * This is a leading comment for a message
*/

message SomeMessage {
  // this is another leading comment
  string value = 1;
}

Trailing comments

Fields, Service Methods, Enum Values and Extensions support trailing comments.

enum MyEnum {
  DEFAULT = 0; // the default value
  OTHER   = 1; // the other value
}

Excluding comments

If you want to have some comment in your proto files, but don't want them to be part of the docs, you can simply prefix the comment with @exclude.

Example: include only the comment for the id field

/**
 * @exclude
 * This comment won't be rendered
 */
message ExcludedMessage {
  string id   = 1; // the id of this message.
  string name = 2; // @exclude the name of this message

  /* @exclude the value of this message. */
  int32 value = 3;
}

Swagger

Optionally you may generate Swagger schema from your proto file. To do so install this plugin.

go get -u github.com/golang/protobuf/protoc-gen-go

Then invoke it as a plugin for Proto Compiler

protoc -I/usr/local/include -I. \
  -I$GOPATH/src \
  -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
  --swagger_out=logtostderr=true:. \
  path/to/your_service.proto

How to add Swagger definitions in my proto scheme?

import "protoc-gen-swagger/options/annotations.proto";

option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger) = {
  info: {
    title: "My Service";
    version: "1.0";
  };
  schemes: HTTP;
  schemes: HTTPS;
  consumes: "application/json";
  produces: "application/json";
};

message MyMessage {
  option (grpc.gateway.protoc_gen_swagger.options.openapiv2_schema) = {
    external_docs: {
      url: "https://infoblox.com/docs/mymessage";
      description: "MyMessage description";
    }
};

For more Swagger options see this scheme

See example contacts app. Here is a generated Swagger schema.

NOTE Well Known Types are generated in a bit unusual way:

    "protobufEmpty": {
      "type": "object",
      "description": "service Foo {\n      rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);\n    }\n\nThe JSON representation for `Empty` is empty JSON object `{}`.",
      "title": "A generic empty message that you can re-use to avoid defining duplicated\nempty messages in your APIs. A typical example is to use it as the request\nor the response type of an API method. For instance:"
    },

Atlas Protoc Gentool

For convenience purposes there is an atlas-gentool image available which contains a pre-installed set of often used plugins. For more details see infobloxopen/atlas-gentool repository.