garcon

package module
v0.13.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 17, 2022 License: MIT Imports: 24 Imported by: 7

README

Teal.Finance/Garcon

logo Opinionated boilerplate all-in-one HTTP server with rate-limiter, Cookies, JWT, CORS, OPA, web traffic, Prometheus export, PProf… for API and static website.


Go Reference

This library is used by Rainbow and other internal projects at Teal.Finance.

Please propose a Pull Request to add here your project that also uses Garcon.

Features

Garcon includes the following middleware pieces:

  • Logging of incoming requests ;
  • Rate-limiter to prevent requests flooding ;
  • JWT management using HttpOnly cookie or Authorization header ;
  • Cross-Origin Resource Sharing (CORS) ;
  • Authentication rules based on Datalog/Rego files using Open Policy Agent ;
  • Web traffic metrics.

Garcon also provides the following features:

  • HTTP/REST server for API endpoints (compatible with any Go-standard HTTP handlers) ;
  • File server intended for static web files supporting Brotli and AVIF data ;
  • Metrics server exporting data to Prometheus (or other compatible monitoring tool) ;
  • PProf server for debugging purpose ;
  • Error response in JSON format ;
  • Chained middleware (fork of justinas/alice).

CPU profiling

Moreover, Garcon provides a helper feature defer ProbeCPU.Stop() to investigate CPU consumption issues thanks to https://github.com/pkg/profile.

In you code, add defer ProbeCPU.Stop() that will write the cpu.pprof file.

import "github.com/teal-finance/garcon/pprof"

func myFunctionConsumingLotsOfCPU() {
    defer pprof.ProbeCPU.Stop()

    // ... lots of sub-functions
}

Install pprof and browse your cpu.pprof file:

cd ~/go
go get -u github.com/google/pprof
cd -
pprof -http=: cpu.pprof

Examples

See also a complete real example in the repo github.com/teal-finance/rainbow.

High-level example

The following code uses the high-level function Garcon.RunServer().

package main

import "github.com/teal-finance/garcon"

func main() {
    g, _ := garcon.New(
        garcon.WithURLs("http://localhost:8080/myapp"),
        garcon.WithDocURL("/doc"), // URL --> http://localhost:8080/myapp/doc
        garcon.WithServerHeader("MyBackendName-1.2.0"),
        garcon.WithJWT(hmacSHA256, "FreePlan", 10, "PremiumPlan", 100),
        garcon.WithOPA("auth.rego"),
        garcon.WithLimiter(10, 30),
        garcon.WithPProf(8093),
        garcon.WithProm(9093),
        garcon.WithDev(),
    )

    h := handler(g.ResErr, g.Checker)

    g.Run(h, 8080)
}
1. Run the high-level example
go build -race ./examples/high-level && ./high-level

2022/01/29 17:31:26 Prometheus export http://localhost:9093
2022/01/29 17:31:26 CORS: Set origin prefixes: [http://localhost:8080 http://localhost: http://192.168.1.]
2022/01/29 17:31:26 CORS: Methods=[GET POST] Headers=[Origin Accept Content-Type Authorization Cookie] Credentials=true MaxAge=86400
2022/01/29 17:31:26 Enable PProf endpoints: http://localhost:8093/debug/pprof
2022/01/29 17:31:26 Create cookie plan=FreePlan domain=localhost secure=false myapp=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2NzUwMDk4ODZ9.hiQQuFxNghrrCvvzEsXzN1lWTavL09Plx0dhFynrBxc
2022/01/29 17:31:26 Create cookie plan=PremiumPlan domain=localhost secure=false myapp=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiUHJlbWl1bVBsYW4iLCJleHAiOjE2NzUwMDk4ODZ9.iP587iHjhLmX_8yMhuQfKu9q7qLbLE7UX-UgkL_VYhE
2022/01/29 17:31:26 JWT not required for dev. origins: [http://localhost:8080 http://localhost: http://192.168.1.]
2022/01/29 17:31:26 Middleware response HTTP header: Set Server MyApp-1.2.0
2022/01/29 17:31:26 Middleware RateLimiter: burst=100 rate=5/s
2022/01/29 17:31:26 Middleware logger: requester IP and requested URL
2022/01/29 17:31:26 Server listening on http://localhost:8080
2. Embedded PProf server

Visit the PProf server at http://localhost:8093/debug/pprof providing the following endpoints:

PProf is easy to use with curl or wget:

( cd ~ ; go get -u github.com/google/pprof )

curl http://localhost:8093/debug/pprof/allocs > allocs.pprof
pprof -http=: allocs.pprof

wget http://localhost:8093/debug/pprof/heap
pprof -http=: heap

wget http://localhost:8093/debug/pprof/goroutine
pprof -http=: goroutine

See the PProf post (2013) for further explanations.

3. Embedded metrics server

The export port http://localhost:9093/metrics is for the monitoring tools like Prometheus.

4. Static website server

The high-level example is running.

Open http://localhost:8080/myapp with your browser, and play with the API endpoints.

The resources and API endpoints are protected with a HttpOnly cookie. The high-level example sets the cookie to browsers visiting the index.html.

func handler(resErr reserr.ResErr, jc *jwtperm.Checker) http.Handler {
    r := chi.NewRouter()

    // Static website files
    ws := webserver.WebServer{Dir: "examples/www", ResErr: resErr}
    r.With(jc.SetCookie).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
    r.With(jc.SetCookie).Get("/favicon.ico", ws.ServeFile("favicon.ico", "image/x-icon"))
    r.With(jc.ChkCookie).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
    r.With(jc.ChkCookie).Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
    r.With(jc.ChkCookie).Get("/images/*", ws.ServeImages())

    // API
    r.With(jc.ChkJWT).Get("/api/v1/items", items)
    r.With(jc.ChkJWT).Get("/api/v1/ducks", resErr.NotImplemented)

    // Other endpoints
    r.NotFound(resErr.InvalidPath)

    return r
}
5. Enable Authentication

Restart again the high-level example with authentication enabled.

Attention, in this example we use two redundant middleware pieces using the same JWT: jwtperm and opa. This is just an example, don't be confused.

go build -race ./examples/high-level && ./high-level -auth

2021/12/02 08:09:47 Prometheus export http://localhost:9093
2021/12/02 08:09:47 CORS: Set origin prefixes: [http://localhost:8080 http://localhost: http://192.168.1.]
2021/12/02 08:09:47 CORS: Methods=[GET] Headers=[Origin Accept Content-Type Authorization Cookie] Credentials=true MaxAge=86400
2021/12/02 08:09:47 JWT not required for dev. origins: [http://localhost:8080 http://localhost: http://192.168.1.]
2021/12/02 08:09:47 Enable PProf endpoints: http://localhost:8093/debug/pprof
2021/12/02 08:09:47 Create cookie plan=FreePlan domain=localhost secure=false jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ5ODd9.5tJk2NoHxkG0o_owtMleBcUaR8z1vRx4rxRRqtZUc_Q
2021/12/02 08:09:47 Create cookie plan=PremiumPlan domain=localhost secure=false jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiUHJlbWl1bVBsYW4iLCJleHAiOjE2Njk5NjQ5ODd9.ifKhbmxQQ64NweL5aQDb_42tvKHwqiEKD-vxHO3KzsM
2021/12/02 08:09:47 OPA: load "examples/sample-auth.rego"
2021/12/02 08:09:47 Middleware OPA: map[sample-auth.rego:package auth

default allow = false
tokens := {"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y"} { true }
allow = true { __local0__ = input.token; data.auth.tokens[__local0__] }]
2021/12/02 08:09:47 Middleware response HTTP header: Set Server MyBackendName-1.2.0
2021/12/02 08:09:47 Middleware RateLimiter: burst=100 rate=5/s
2021/12/02 08:09:47 Middleware logger: requester IP and requested URL
2021/12/02 08:09:47 Server listening on http://localhost:8080
6. Default HTTP request headers

Test the API with curl:

curl -D - http://localhost:8080/myapp/api/v1/items

HTTP/1.1 401 Unauthorized
Content-Type: application/json
Server: MyBackendName-1.2.0
Vary: Origin
X-Content-Type-Options: nosniff
Date: Thu, 02 Dec 2021 07:06:20 GMT
Content-Length: 84

{"error":"Unauthorized",
"path":"/api/v1/items",
"doc":"http://localhost:8080/myapp/doc"}

The corresponding garcon logs:

2021/12/02 08:06:20 in  127.0.0.1:42888 GET /api/v1/items
[cors] 2021/12/02 08:06:20 Handler: Actual request
[cors] 2021/12/02 08:06:20   Actual request no headers added: missing origin
2021/12/02 08:06:20 OPA unauthorize 127.0.0.1:42888 /api/v1/items
2021/12/02 08:06:20 out 127.0.0.1:42888 GET /api/v1/items 1.426916ms c=1

The CORS logs can be disabled by passing debug=false in cors.Handler(origins, false).

The value c=1 measures the web traffic (current active HTTP connections).

7. With Authorization header
curl -D - http://localhost:8080/myapp/api/v1/items -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y'

HTTP/1.1 200 OK
Content-Type: application/json
Server: MyBackendName-1.2.0
Vary: Origin
Date: Thu, 02 Dec 2021 07:10:37 GMT
Content-Length: 25

["item1","item2","item3"]

The corresponding garcon logs:

2021/12/02 08:10:37 in  127.0.0.1:42892 GET /api/v1/items
[cors] 2021/12/02 08:10:37 Handler: Actual request
[cors] 2021/12/02 08:10:37   Actual request no headers added: missing origin
2021/12/02 08:10:37 Authorization header has JWT: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y
2021/12/02 08:10:37 JWT Claims: {FreePlan  {  [] 2022-12-02 08:01:28 +0100 CET <nil> <nil> invalid cookie}}
2021/12/02 08:10:37 JWT has the FreePlan Namespace
2021/12/02 08:10:37 JWT Permission: {10}
2021/12/02 08:10:37 out 127.0.0.1:42892 GET /api/v1/items 1.984568ms c=1 a=1 i=0 h=0

Low-level example

⚠ WARNING: This chapter is outdated! ⚠

See the low-level example.

The following code is a bit different to the stuff done by the high-level function Garcon.Run() presented in the previous chapter. The following code is intended to show Garcon can be customized to meet your specific requirements.

package main

import (
    "log"
    "net"
    "net/http"
    "time"

    "github.com/go-chi/chi/v5"
    "github.com/teal-finance/garcon"
    "github.com/teal-finance/garcon/chain"
    "github.com/teal-finance/garcon/cors"
    "github.com/teal-finance/garcon/metrics"
    "github.com/teal-finance/garcon/opa"
    "github.com/teal-finance/garcon/pprof"
    "github.com/teal-finance/garcon/quota"
    "github.com/teal-finance/garcon/reserr"
    "github.com/teal-finance/garcon/webserver"
)

// Garcon settings
const apiDoc = "https://my-dns.co/doc"
const allowedProdOrigin = "https://my-dns.co"
const allowedDevOrigins = "http://localhost:  http://192.168.1."
const serverHeader = "MyBackendName-1.2.0"
const authCfg = "examples/sample-auth.rego"
const pprofPort = 8093
const expPort = 9093
const burst, reqMinute = 10, 30
const devMode = true

func main() {
    if devMode {
        // the following line collects the CPU-profile and writes it in the file "cpu.pprof"
        defer pprof.ProbeCPU().Stop()
    }

    pprof.StartServer(pprofPort)

    // Uniformize error responses with API doc
    resErr := reserr.New(apiDoc)

    mw, connState := setMiddlewares(resErr)

    // Handles both REST API and static web files
    h := handler(resErr)
    h = mw.Then(h)

    runServer(h, connState)
}

func setMiddlewares(resErr reserr.ResErr) (mw chain.Chain, connState func(net.Conn, http.ConnState)) {
    // Start a metrics server in background if export port > 0.
    // The metrics server is for use with Prometheus or another compatible monitoring tool.
    metrics := metrics.Metrics{}
    mw, connState = metrics.StartServer(expPort, devMode)

    // Limit the input request rate per IP
    reqLimiter := quota.New(burst, reqMinute, devMode, resErr)

    corsConfig := allowedProdOrigin
    if devMode {
        corsConfig += " " + allowedDevOrigins
    }

    allowedOrigins := garcon.SplitClean(corsConfig)

    mw = mw.Append(
        reqLimiter.Limit,
        garcon.ServerHeader(serverHeader),
        cors.Handler(allowedOrigins, devMode),
    )

    // Endpoint authentication rules (Open Policy Agent)
    files := garcon.SplitClean(authCfg)
    policy, err := opa.New(files, resErr)
    if err != nil {
        log.Fatal(err)
    }

    if policy.Ready() {
        mw = mw.Append(policy.Auth)
    }

    return mw, connState
}

// runServer runs in foreground the main server.
func runServer(h http.Handler, connState func(net.Conn, http.ConnState)) {
    const mainPort = "8080"

    server := http.Server{
        Addr:              ":" + mainPort,
        Handler:           h,
        TLSConfig:         nil,
        ReadTimeout:       1 * time.Second,
        ReadHeaderTimeout: 1 * time.Second,
        WriteTimeout:      1 * time.Second,
        IdleTimeout:       1 * time.Second,
        MaxHeaderBytes:    222,
        TLSNextProto:      nil,
        ConnState:         connState,
        ErrorLog:          log.Default(),
        BaseContext:       nil,
        ConnContext:       nil,
    }

    log.Print("Server listening on http://localhost", server.Addr)

    log.Fatal(server.ListenAndServe())
}

// handler creates the mapping between the endpoints and the handler functions.
func handler(resErr reserr.ResErr) http.Handler {
    r := chi.NewRouter()

    // Static website files
    ws := webserver.WebServer{Dir: "examples/www", ResErr: resErr}
    r.Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
    r.Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
    r.Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
    r.Get("/images/*", ws.ServeImages())

    // API
    r.Get("/api/v1/items", items)
    r.Get("/api/v1/ducks", resErr.NotImplemented)

    // Other endpoints
    r.NotFound(resErr.InvalidPath)

    return r
}

func items(w http.ResponseWriter, _ *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    _, _ = w.Write([]byte(`["item1","item2","item3"]`))
}

KeyStore example

The example KeyStore implements a key/value datastore providing private storage for each client identified by its unique IP.

go build ./examples/keystore
./keystore

Then open http://localhost:8080 to learn more about the implemented features.

✨ Contributions Welcome

This project needs your help to become better. Please propose your enhancements, or even a further refactoring.

We welcome contributions in many forms, and there's always plenty to do!

🗣️ Feedback

If you have some suggestions, or need a new feature, please contact us, using the issues, or at Teal.Finance[à]pm.me or @TealFinance.

Feel free to propose a Pull Request, your contributions are welcome. 😉

Copyright (c) 2022 Teal.Finance contributors

Teal.Finance/Garcon is free software, and can be redistributed and/or modified under the terms of the MIT License. SPDX-License-Identifier: MIT

Teal.Finance/Garcon is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

See the LICENSE file (alongside the source files) or https://opensource.org/licenses/MIT.

See also

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var DevOrigins = []*url.URL{
	{Scheme: "http", Host: "localhost:"},
	{Scheme: "http", Host: "192.168.1."},
}

DevOrigins provides the development origins: - yarn run vite --port 3000 - yarn run vite preview --port 5000 - localhost:8085 on multidevices: web autoreload using https://github.com/synw/fwr - flutter run --web-port=8080 - 192.168.1.x + any port on tablet: mobile app using fast builtin autoreload.

View Source 
var ErrNonPrintable = errors.New("non-printable")

Functions

func AppendPrefixes added in v0.5.1 

func AppendPrefixes(origins []string, prefixes ...string) []string

func AppendURLs added in v0.9.0 

func AppendURLs(urls []*url.URL, prefixes ...*url.URL) []*url.URL

func OriginsFromURLs added in v0.9.0 

func OriginsFromURLs(urls []*url.URL) []string

func ParseURLs added in v0.9.3 

func ParseURLs(origins []string) []*url.URL

func ServerHeader 

func ServerHeader(version string) func(next http.Handler) http.Handler

ServerHeader sets the Server HTTP header in the response.

func SplitClean 

func SplitClean(values string) []string

SplitClean splits the values and trim them.

func Value added in v0.12.3 

func Value(r *http.Request, key, header string) (string, error)

Value returns the /endpoint/{key} (URL path) else the "key" form (HTTP body) else the "key" query string (URL) else the HTTP header.

func Values added in v0.12.3 

func Values(r *http.Request, key string) ([]string, error)

Types

type Garcon 

type Garcon struct {
	ConnState      func(net.Conn, http.ConnState)
	Checker        TokenChecker
	ResErr         reserr.ResErr
	AllowedOrigins []string
	Middlewares    chain.Chain
}

func New added in v0.6.0 

func New(opts ...Option) (*Garcon, error)

func (*Garcon) NewJWTChecker added in v0.6.0 

func (g *Garcon) NewJWTChecker(urls []*url.URL, secretKey []byte, planPerm ...interface{}) *jwtperm.Checker

func (*Garcon) NewSessionToken added in v0.13.1 

func (g *Garcon) NewSessionToken(urls []*url.URL, secretKey []byte, expiry time.Duration, setIP bool) *incorruptible.Incorruptible

func (*Garcon) Run added in v0.4.0 

func (g *Garcon) Run(h http.Handler, port int) error

RunServer runs the HTTP server(s) in foreground. Optionally it also starts a metrics server in background (if export port > 0). The metrics server is for use with Prometheus or another compatible monitoring tool.

type Option added in v0.6.0 

type Option func(*parameters)

func WithDev added in v0.6.0 

func WithDev(enable ...bool) Option

func WithDocURL added in v0.6.0 

func WithDocURL(pathOrURL string) Option

func WithIncorruptible added in v0.13.4 

func WithIncorruptible(secretKeyHex string, expiry time.Duration, setIP bool) Option

WithIncorruptible enables the "session" cookies based on fast and tiny token. WithIncorruptible requires WithURLs() to set the Cookie name, secure, domain and path. WithIncorruptible is not compatible with WithJWT: use only one of them.

func WithJWT added in v0.6.0 

func WithJWT(secretKeyHex string, planPerm ...interface{}) Option

WithJWT requires WithURLs() to set the Cookie name, secure, domain and path. WithJWT is not compatible with WithTkn: use only one of them.

func WithLimiter added in v0.6.0 

func WithLimiter(values ...int) Option

func WithOPA added in v0.6.0 

func WithOPA(opaFilenames ...string) Option

func WithPProf added in v0.6.0 

func WithPProf(port int) Option

func WithProm added in v0.6.0 

func WithProm(port int, namespace string) Option

func WithReqLogs added in v0.6.1 

func WithReqLogs(verbosity ...int) Option

func WithServerHeader added in v0.6.0 

func WithServerHeader(nameVersion string) Option

func WithURLs added in v0.9.0 

func WithURLs(urls ...string) Option

type TokenChecker added in v0.13.0 

type TokenChecker interface {
	// Set sets a cookie in the response when the request has no valid token.
	// Set searches the token in a cookie and in the first "Authorization" header.
	// Finally, Set stores the token attributes in the request context.
	Set(next http.Handler) http.Handler

	// Chk accepts requests only if it has a valid cookie.
	// Chk does not verify the "Authorization" header.
	// See the Vet() function to also verify the "Authorization" header.
	// Chk also stores the token attributes in the request context.
	// In dev. testing, Chk accepts any request but does not store invalid tokens.
	Chk(next http.Handler) http.Handler

	// Vet accepts requests having a valid token either in
	// the cookie or in the first "Authorization" header.
	// Vet also stores the decoded token in the request context.
	// In dev. testing, Vet accepts any request but does not store invalid tokens.
	Vet(next http.Handler) http.Handler
}

Source Files

View all Source files

Directories

Path Synopsis
examples
high-level
keystore
low-level
Package jwtperm delivers and checks the JWT permissions
 Package jwtperm delivers and checks the JWT permissions
package opa manages the Open Policy Agent.
 package opa manages the Open Policy Agent.
Package pprof serves the /debug/pprof endpoint
 Package pprof serves the /debug/pprof endpoint
Package reqlog logs incoming request URL and browser fingerprints.
 Package reqlog logs incoming request URL and browser fingerprints.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.