protoc-gen-httpclient

README

Protoc HTTP Client Generator

• Protoc HTTP Client Generator
  • Overview
  • Features
    • Go
  • Directory Structure
    • example
    • modules
    • pkg
    • proto
    • templates
  • Roadmap
  • Build
    • Dependencies
    • Commands
  • Usage
    • Client Customizations
    • Proto Options
    • Google WellKnown Types

Overview

This library aims to help with language agnostic http-client generation through using protobufs as a single source of truth .

Features

Go

• Automatic Request Retries : The client retries behind the scenes in case a request fails, Heimdall , and returns the result if only one o*f the retries are successful. Currently constant and exponential backoff is implemented for request retries.

• Circuit-Breaking and Custom Client Fallback : In case there are too many failing requests are , or the number of requests sent are above a configured threshold the client "opens the circuit" for a short period of time. This prevents any more requests from being made and gives the downstream systems recovery time.

• Client Customization : all requests are variadic functions. Method used for customizing the request are defined for their respective client types :

• circuit-breaker client : pkg/go/client/circuit-breaker/client.go

• basic client : pkg/go/client/basic/client.go

Directory Structure

example

This directory contains the example proto file

modules

This directory is contains protobuf compiler modules based on different target language implementation . there is also a proto directory in which shared data structure models used by different compiler modules. These data structures fields are populated by the modules for each language and then passed to the approportiate template.

pkg

This directory contains target language specific libraries used in generated methods .

proto

this directory contains the proto file that has the plugin specific options and must be imported in any proto file that is passed to this plugin.

templates

this directory contains language specific templates that the plugin uses to generate the code.

Roadmap

• Generating Go code from proto files
  • Client Features - [x] request auto-retry - [x] circuit breaker - [x] custom fallback
  • authentication - [x] basic authentication - [ ] other authentication schemes
  • Request Types - [x] POST request - [x] GET request - [x] PUT request - [x] DELETE request - [ ] others
• Generating TypeScript code from proto files
• Generating vanilla Javascript code from proto files
• Generating Rust code from proto files
• Generating Java code from proto files
• Generating C++ code from proto files

Build

Dependencies

We use mage as our main build tool and packr to embed static files (the templates).

install mage by running :

go get -u -d -v github.com/magefile/mage
cd $GOPATH/src/github.com/magefile/mage
go run bootstrap.go

install packr by running :

go get -u -v github.com/gobuffalo/packr/v2/...
go get -u -v github.com/gobuffalo/packr/v2/packr2

Commands

• mage build : rebuild the plugin and install the binary
• mage example : rebuild the plugin and generate examples in /example
• mage clean : remove generated go files in example folder and clean up the files packer generates

Usage

you can use example/example.proto and makefile in example/ as a reference point for your own proto definitions.

Client Customizations

In the requests generated by the plugin there is no retry mechanism by default. To set the retry mechansim , first create a new backoff strategy either by implemtent your own by implementing Backoff interface defined in pkg/go/backoff/shared or by calling the New() method on a predefined backoff strategy [ constant | exponential ] pacakge from pkg/go/backoff. use Retrier() and pass it to the rpc method as clientopt

The circuit breaker client does not have a predefined fallback function. use FallbackFunc() and pass it to the rpc method as clientopt.

Proto Options

after writing your proto file and importing github.com/bifrostcloud/protoc-gen-httpclient/proto/types.proto in it , you must define service specific and rpc specific options.

• service specific option :
  • auth : authentication scheme. currently, only basic is supported. if left blank , it would defualt to basic.
  • endpoint : service base endpoint. target defined in RPCs would be appended to this value
• method specific options :
  • client_type : client type can be circuit-breaker or basic . if left blank , it would default to basic
  • target : this string would be appended to endpoint defiend in service option.
  • useragent : this is an optional field that sets request user agend
  • method : currently only get , post , put and delete requests are acceptable values in this field.

Google WellKnown Types

Most often we found ourselves using Google Well-Known Types as defined in https://developers.google.com/protocol-buffers/docs/reference/google.protobuf. the following table is the filename (proto import) to package (go import) for github.com/gogo/protobuf

File Name	Package
google/protobuf/any.proto	github.com/gogo/protobuf/types
google/protobuf/api.proto	google.golang.org/genproto/protobuf/api
google/protobuf/compiler/plugin.proto	github.com/gogo/protobuf/protoc-gen-gogo/plugin
google/protobuf/descriptor.proto	github.com/gogo/protobuf/protoc-gen-gogo/descriptor
google/protobuf/duration.proto	github.com/gogo/protobuf/types
google/protobuf/empty.proto	github.com/gogo/protobuf/types
google/protobuf/field_mask.proto	github.com/gogo/protobuf/types
google/protobuf/source_context.proto	google.golang.org/genproto/protobuf/source_context
google/protobuf/struct.proto	github.com/gogo/protobuf/types
google/protobuf/timestamp.proto	github.com/gogo/protobuf/types
google/protobuf/type.proto	google.golang.org/genproto/protobuf/ptype
google/protobuf/wrappers.proto	github.com/gogo/protobuf/types