go-service

README

Go Service

A framework to build services in go. This came out of out building services over the years and what I have considered good practices in building services. Hence it is highly subjective and opinionated.

This framework stands on the shoulder of giants so we don't reinvent the wheel!

Dependency Injection

This framework heavily relies on DI. We have chosen to use Uber FX. So there is great information online to get you up to speed.

Commands

A service has commands that are configured using Cobra. Each service has the following commands (you can add more):

• Server - This will host your server.
• Client - This will provide your client needs.

These are configured in the main function.

Configuration

The supported configuration kinds are as follows:

• JSON
• TOML
• YAML

The configuration can be read from multiple sources by specifying a flag called --input or -i. As per the following:

• env:CONFIG_FILE - Read from an env variable called CONFIG_FILE. This is the default if nothing is passed. The env variable can be file path or the configuration. If it is the config, we expect the format of extension:ENV_VARIABLE, where extension is the supported kinds and ENV_VARIABLE contains the contents of the config that are base64 encoded. This can be overridden.
• file:path - Read from the path.

The reason for this is that we want to be able to separate how configuration is retrieved. This way we can use and application configuration system.

This is the configuration. We will outline the config required in each section. The following configuration examples will use YAML.

Environment

You can specify the environment of the service.

Configuration

To configure, please specify the following:

environment: development

Compression

We support the following:

• None
• Zstd
• S2
• Snappy

Marshalling

We support the following:

• JSON
• TOML
• YAML
• Proto
• GOB

Caching

The framework currently supports the following caching solutions:

• Redis Cache
• Ristretto

Configuration

To configure, please specify the following:

cache:
  redis:
    compressor: snappy
    marshaller: proto
    addresses:
      server: localhost:6379
    username: test
    password: test
    db: 0
  ristretto:
    num_counters: 10000000
    max_cost: 100000000
    buffer_items: 64

Feature

The framework supports OpenFeature.

Configuration

To configure, please specify the following:

feature:
  kind: flipt
  host: localhost:9000
  retry:
      timeout: 1s
      attempts: 3

Hooks

The framework supports Standard Webhooks.

Configuration

To configure, please specify the following:

hooks:
  secret: Base64 encoded secret

Runtime

We enhance the runtime with the following:

• Automaxprocs
• Automemlimit

SQL

For SQL databases we support the following:

• Postgres

We also support master, slave combinations with the awesome mssqlx.

Configuration

To configure, please specify the following:

sql:
  pg:
    masters:
      -
        url: postgres://test:test@localhost:5432/test?sslmode=disable
    slaves:
      -
        url: postgres://test:test@localhost:5432/test?sslmode=disable
    max_open_conns: 5
    max_idle_conns: 5
    conn_max_lifetime: 1h

Health

The health package is based on go-health. This package allows us to create all sorts of ways to check external and internal systems.

We also provide ways to integrate into container integration systems. So we provide the following endpoints:

• /healthz - This allows us to check any external dependency and provide a breakdown of what is not functioning. This should only be used for verification.
• /livez: Can be used for k8s liveness.
• /readyz: Can be used for k8s readiness.

This is modelled around Kubernetes API health endpoints.

Telemetry

Telemetry is broken down in the following sections:

Logging

For logging we use Uber Zap.

Configuration

To configure, please specify the following:

telemetry:
  logger:
    level: info

Metrics

For metrics we support the following:

• OpenTelemetry
• Prometheus

Configuration

Below is the configuration for each system.

Prometheus

To configure, please specify the following:

telemetry:
  metrics:
    kind: prometheus

OTLP

To configure, please specify the following:

telemetry:
  metrics:
    kind: otlp
    host: http://localhost:9009/otlp/v1/metrics
    key: base64 (username:password)

Trace

For distributed tracing we support the following:

• OpenTelemetry

Configuration

Below is the configuration for each system.

OTLP

To configure, please specify the following:

telemetry:
  tracer:
    kind: otlp
    host: localhost:4318
    key: base64 (username:password)

Token

The framework allows you to define different token generators and verifiers. We provide the config, how these are defined are up to you.

Configuration

To configure, please specify the following:

token:
  Kind: none

Limiter

The framework allows you to define a limiter. This will be applied to the different transports.

The different kinds are:

• user-agent
• ip

Configuration

To configure, please specify the following:

limiter:
  kind: user-agent
  pattern: 10-S

Time

The framework allows you use network time services. We use:

• ntp
• nts

Configuration

To configure, please specify the following:

time:
  kind: nts
  host: time.cloudflare.com

Transport

The transport layer provides ways to abstract communication for in/out of the service. So we have the following integrations:

• gRPC - The author truly believes in IDLs.
• REST - This is achieved with gRPC-Gateway.
• CloudEvents - A specification for describing event data in a common way.

gRPC

Below is list of the provided interceptors:

• Limiter

REST

Below is list of the provided handlers:

• CORS
• Limiter

Configuration

To configure, please specify the following:

transport:
  http:
    port: 8000
    retry:
      timeout: 1s
      attempts: 3
  grpc:
    port: 9000
    retry:
      timeout: 1s
      attempts: 3

If you would like to enable TLS, do the following:

transport:
  http:
    tls:
      cert: PEM certificate base64 encoded
      key: PEM certificate base64 encoded
  grpc:
    tls:
      cert: PEM certificate base64 encoded
      key: PEM certificate base64 encoded

Cryptography

The crypto package provides sensible defaults for symmetric, asymmetric, hashing and randomness.

We rely on the following libraries:

• argon2

Configuration

To configure, please specify the following:

crypto:
  aes:
    key: certificate base64 encoded
  ed25519:
    public: certificate base64 encoded
    private: certificate base64
  hmac:
    key: certificate base64 encoded
  rsa:
    public: certificate base64 encoded
    private: certificate base64 encoded

Debug

This section outlines all utilities added for you troubleshooting abilities.

statsviz

GET http://localhost:6060/debug/statsviz

Check out statsviz.

pprof

GET http://localhost:6060/debug/pprof/
GET http://localhost:6060/debug/pprof/cmdline
GET http://localhost:6060/debug/pprof/profile
GET http://localhost:6060/debug/pprof/symbol
GET http://localhost:6060/debug/pprof/trace

Check out pprof.

fgprof

GET http://localhost:6060/debug/fgprof?seconds=10

Check out fgprof.

gopsutil

GET http://localhost:6060/debug/psutil

Check out gopsutil.

Configuration

To configure, please specify the following:

debug:
  port: 6060

If you would like to enable TLS, do the following:

debug:
  tls:
    cert: PEM certificate base64 encoded
    key: PEM certificate base64 encoded

Development

This section describes how to run and contribute to the project, if you are interested.

Style

We favour what is defined in the Uber Go Style Guide.

Dependencies

Please setup the following:

• https://github.com/FiloSottile/mkcert

Setup

To get yourself setup, please run:

git submodule sync
git submodule update --init

mkcert -install
make create-certs

make dep

Environment

As we rely on external services these need to be configured:

Starting

Please run:

make start

Stopping

Please run:

make stop

Testing

To be able to test locally, please run:

make specs