serverutils

package module
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2021 License: MIT Imports: 31 Imported by: 65

README

Build Status Maintained Linting and Tests Coverage Status

Server Utils Library

serverutils are utilities used by several backend services. These includes starting up and running the server.

Installing it

serverutils is compatible with modern Go releases in module mode, with Go installed:

go get -u github.com/savannahghi/serverutils

will resolve and add the package to the current development module, along with its dependencies.

Alternatively the same can be achieved if you use import in a package:

import "github.com/savannahghi/serverutils"

and run go get without parameters.

The package name is serverutils

Developing

The default branch library is main

We try to follow semantic versioning ( https://semver.org/ ). For that reason, every major, minor and point release should be tagged.

git tag -m "v0.0.1" "v0.0.1"
git push --tags

Continuous integration tests must pass on Travis CI. Our coverage threshold is 90% i.e you must keep coverage above 90%.

Environment variables

In order to run tests, you need to have an env.sh file similar to this one:

# Application settings
export DEBUG=true
export IS_RUNNING_TESTS=true
export SENTRY_DSN=<a Sentry Data Source Name>
export GOOGLE_CLOUD_PROJECT="Google Cloud project id"

This file must not be committed to version control.

It is important to export the environment variables. If they are not exported, they will not be visible to child processes e.g go test ./....

These environment variables should also be set up on Travis CI environment variable section.

Contributing

I would like to cover the entire GitHub API and contributions are of course always welcome. The calling pattern is pretty well established, so adding new methods is relatively straightforward. See CONTRIBUTING.md for details.

Versioning

In general, serverutils follows semver as closely as we can for tagging releases of the package. For self-contained libraries, the application of semantic versioning is relatively straightforward and generally understood. We've adopted the following versioning policy:

  • We increment the major version with any incompatible change to non-preview functionality, including changes to the exported Go API surface or behavior of the API.
  • We increment the minor version with any backwards-compatible changes to functionality, as well as any changes to preview functionality in the GitHub API. GitHub makes no guarantee about the stability of preview functionality, so neither do we consider it a stable part of the go-github API.
  • We increment the patch version with any backwards-compatible bug fixes.

License

This library is distributed under the MIT license found in the LICENSE file.

Documentation

Index

Constants

View Source 
const (
	// AppName is the name of "this server"
	AppName = "api-gateway"

	// DSNEnvVarName is the Sentry reporting config
	DSNEnvVarName = "SENTRY_DSN"

	// AppVersion is the app version (used for StackDriver error reporting)
	AppVersion = "0.0.1"

	// PortEnvVarName is the name of the environment variable that defines the
	// server port
	PortEnvVarName = "PORT"

	// DefaultPort is the default port at which the server listens if the port
	// environment variable is not set
	DefaultPort = "8080"

	// GoogleCloudProjectIDEnvVarName is used to determine the ID of the GCP project e.g for setting up StackDriver client
	GoogleCloudProjectIDEnvVarName = "GOOGLE_CLOUD_PROJECT"

	// DebugEnvVarName is used to determine if we should print extended tracing / logging (debugging aids)
	// to the console
	DebugEnvVarName = "DEBUG"

	// IsRunningTestsEnvVarName is used to determine if we are running in a test environment
	IsRunningTestsEnvVarName = "IS_RUNNING_TESTS"

	// Environment points to where this service is running e.g staging, testing, prod
	Environment = "ENVIRONMENT"

	// StagingEnv runs the service under staging
	StagingEnv = "staging"

	// DemoEnv runs the service under demo
	DemoEnv = "demo"

	// TestingEnv runs the service under testing
	TestingEnv = "testing"

	// ProdEnv runs the service under production
	ProdEnv = "prod"
)
View Source 
const (
	ResolverSuccessValue = "OK"
	ResolverFailureValue = "FAILED"
)

Resolver status values

Variables

View Source 
var (
	GraphqlResolverLatency = stats.Float64(
		"graphql_resolver_latency",
		"The Latency in milliseconds per graphql resolver execution",
		"ms",
	)

	// Resolver is the Graphql resolver used when making a GraphQl request
	ResolverName = tag.MustNewKey("resolver.name")

	// Error is the error recorded if an error occurs
	ResolverErrorMessage = tag.MustNewKey("resolver.error")

	// ResolverStatus is used to tag whether passed or failed
	// it is either pass/fail...make constants
	ResolverStatus = tag.MustNewKey("resolver.status")

	GraphqlResolverLatencyView = &view.View{
		Name:        "graphql_resolver_latency_distribution",
		Description: "Time taken by a graphql resolver",
		Measure:     GraphqlResolverLatency,

		Aggregation: view.Distribution(LatencyBounds...),
		TagKeys:     []tag.Key{ResolverName, ResolverErrorMessage, ResolverStatus},
	}

	GraphqlResolverCountView = &view.View{
		Name:        "graphql_resolver_request_count",
		Description: "The number of times a graphql resolver is executed",
		Measure:     GraphqlResolverLatency,
		Aggregation: view.Count(),
		TagKeys:     []tag.Key{ResolverName, ResolverErrorMessage, ResolverStatus},
	}
)

Server HTTP measures used to record metrics

View Source 
var (
	HTTPRequestLatency = stats.Float64(
		"http_request_latency",
		"The Latency in milliseconds per http request execution",
		"ms",
	)

	// Path is the URL path (not including query string) in the request.
	HTTPPath = tag.MustNewKey("http.path")

	// StatusCode is the numeric HTTP response status code.
	HTTPStatusCode = tag.MustNewKey("http.status")

	// Method is the HTTP method of the request.
	HTTPMethod = tag.MustNewKey("http.method")

	ServerRequestLatencyView = &view.View{
		Name:        "http_request_latency_distribution",
		Description: "Time taken to process a http request",
		Measure:     HTTPRequestLatency,

		Aggregation: view.Distribution(LatencyBounds...),
		TagKeys:     []tag.Key{HTTPPath, HTTPStatusCode, HTTPMethod},
	}

	ServerRequestCountView = &view.View{
		Name:        "http_request_count",
		Description: "The number of HTTP requests",
		Measure:     HTTPRequestLatency,
		Aggregation: view.Count(),
		TagKeys:     []tag.Key{HTTPPath, HTTPStatusCode, HTTPMethod},
	}
)

Views for the collected metrics i.e how they are exported to the various backends

View Source 
var DefaultServiceViews = []*view.View{GraphqlResolverLatencyView, GraphqlResolverCountView, ServerRequestLatencyView, ServerRequestCountView}

DefaultServiceViews are the default/common server views provided by base package The views can be used by the various services

View Source 
var LatencyBounds = GenerateLatencyBounds(60000, 200) //1 min in intervals of 200ms

LatencyBounds used in aggregating latency should be in ms i.e seconds written in ms eg 1s --> 1000ms [>=0ms, >=10ms, >=20ms, >=30ms,...., >=4s, >=5s, >=6s >=7s]

Disclaimer: The interval value should be reasonable so as to avoid many buckets. If the distribution metrics has many buckets, it will not export the metrics.

Functions

func BoolEnv 

func BoolEnv(envVarName string) bool

BoolEnv gets and parses a boolean environment variable

func CloseStackDriverErrorClient 

func CloseStackDriverErrorClient(errorClient *errorreporting.Client)

CloseStackDriverErrorClient closes a StackDriver error client and logs any arising error.

It was written to be defer()'d in servrer initialization code.

func CloseStackDriverLoggingClient 

func CloseStackDriverLoggingClient(loggingClient *logging.Client)

CloseStackDriverLoggingClient closes a StackDriver logging client and logs any arising error.

It was written to be defer()'d in servrer initialization code.

func ConvertStringToInt 

func ConvertStringToInt(w http.ResponseWriter, val string) int

ConvertStringToInt converts a supplied string value to an integer. It writes an error to the JSON response writer if the conversion fails.

func CustomHTTPRequestMetricsMiddleware 

func CustomHTTPRequestMetricsMiddleware() func(http.Handler) http.Handler

CustomHTTPRequestMetricsMiddleware is used to implement custom metrics for our http requests The custom middleware used to collect any custom http request stats It will also be used to capture distributed trace requests and propagate them through context

func DecodeJSONToTargetStruct 

func DecodeJSONToTargetStruct(w http.ResponseWriter, r *http.Request, targetStruct interface{})

DecodeJSONToTargetStruct maps JSON from a HTTP request to a struct. TODO: Move to common helpers

func EnableStatsAndTraceExporters 

func EnableStatsAndTraceExporters(ctx context.Context, service string) (func(), error)

EnableStatsAndTraceExporters a wrapper for initializing metrics exporters TODO:Look into improvements

func ErrorMap 

func ErrorMap(err error) map[string]string

ErrorMap turns the supplied error into a map with "error" as the key

func GenerateLatencyBounds 

func GenerateLatencyBounds(max, interval int) []float64

GenerateLatencyBounds is used in generating latency bounds The arguments provided should be in millisecond format e.g 1s == 1000ms interval will be used as an increment value [>=0ms, >=100ms, >=200ms, >=300ms,...., >=1000ms]

func GetEnvVar 

func GetEnvVar(envVarName string) (string, error)

GetEnvVar retrieves the environment variable with the supplied name and fails if it is not able to do so

func GetRunningEnvironment 

func GetRunningEnvironment() string

GetRunningEnvironment returns the environment where the service is running. Important so as to point to the correct deps

func HealthStatusCheck 

func HealthStatusCheck(w http.ResponseWriter, r *http.Request)

HealthStatusCheck endpoint to check if the server is working.

func InitOtelSDK added in v0.0.3 

func InitOtelSDK(ctx context.Context, serviceName string) (*tracesdk.TracerProvider, error)

InitOtelSDK returns an OpenTelemetry TracerProvider configured to use the Jaeger exporter for sending traces/spans. The returned TracerProvider will also use a Resource configured with all the information about the service deployment.

func IsDebug 

func IsDebug() bool

IsDebug returns true if debug has been turned on in the environment

func IsRunningTests 

func IsRunningTests() bool

IsRunningTests returns true if debug has been turned on in the environment

func ListenAddress 

func ListenAddress() string

ListenAddress determines what port to listen on and falls back to a default port if the environment does not supply a port

func LogStartupError 

func LogStartupError(ctx context.Context, err error)

LogStartupError is used to e.g log fatal startup errors. It logs, attempts to report the error to StackDriver then panics/crashes.

func MetricsCollectorService 

func MetricsCollectorService(serviceName string) string

MetricsCollectorService returns name of service suffixed by it's running environment this helps identify metrics from different services at the backend/metrics viewer. e.g namespace in prometheus exporter

func MustGetEnvVar 

func MustGetEnvVar(envVarName string) string

MustGetEnvVar returns the value of the environment variable with the indicated name or panics. It is intended to be used in the INTERNALS of the server when we can guarantee (through orderly coding) that the environment variable was set at server startup. Since the env is required, kill the app if the env is not set. In the event a variable is not super required, set a sensible default or don't call this method

func RecordGraphqlResolverMetrics 

func RecordGraphqlResolverMetrics(ctx context.Context, startTime time.Time, name string, e error)

RecordGraphqlResolverMetrics records the metrics for a specific graphql resolver It should be deferred until the execution of the resolver function is completed

func RecordHTTPStats 

func RecordHTTPStats(w *MetricsResponseWriter, r *http.Request)

RecordHTTPStats adds tags and records the metrics for a request

func RequestDebugMiddleware 

func RequestDebugMiddleware() func(http.Handler) http.Handler

RequestDebugMiddleware dumps the incoming HTTP request to the log for inspection

func Sentry 

func Sentry() error

Sentry initializes Sentry, for error reporting

func StackDriver 

func StackDriver(ctx context.Context) *errorreporting.Client

StackDriver initializes StackDriver logging, error reporting, profiling etc

func StartTestServer 

func StartTestServer(ctx context.Context, prepareServer PrepareServer, allowedOrigins []string) (*http.Server, string, error)

StartTestServer starts up test server

func WriteJSONResponse 

func WriteJSONResponse(w http.ResponseWriter, source interface{}, status int)

WriteJSONResponse writes the content supplied via the `source` parameter to the supplied http ResponseWriter. The response is returned with the indicated status. TODO: Move to common helpers

Types

type ErrorResponseWriter 

type ErrorResponseWriter struct {
	// contains filtered or unexported fields
}

ErrorResponseWriter is a http.ResponseWriter that always errors on attempted writes.

It is necessary for tests.

func NewErrorResponseWriter 

func NewErrorResponseWriter(err error) *ErrorResponseWriter

NewErrorResponseWriter returns an initialized ErrorResponseWriter

func (*ErrorResponseWriter) Header 

func (w *ErrorResponseWriter) Header() http.Header

Header delegates reading of headers to the underlying response writer

func (*ErrorResponseWriter) Write 

func (w *ErrorResponseWriter) Write([]byte) (int, error)

Write always returns the supplied error on any attempt to write.

func (*ErrorResponseWriter) WriteHeader 

func (w *ErrorResponseWriter) WriteHeader(statusCode int)

WriteHeader delegates writing of headers to the underlying response writer

type MetricsResponseWriter 

type MetricsResponseWriter struct {
	StatusCode int
	StartTime  time.Time
	// contains filtered or unexported fields
}

MetricsResponseWriter implements the http.ResponseWriter Interface it is a wrapper of http.ResponseWriter and enables obtaining measures

func NewMetricsResponseWriter 

func NewMetricsResponseWriter(w http.ResponseWriter) *MetricsResponseWriter

NewMetricsResponseWriter new http.ResponseWriter wrapper

func (*MetricsResponseWriter) Header 

func (m *MetricsResponseWriter) Header() http.Header

Header ...

func (*MetricsResponseWriter) Write 

func (m *MetricsResponseWriter) Write(b []byte) (int, error)

Write ...

func (*MetricsResponseWriter) WriteHeader 

func (m *MetricsResponseWriter) WriteHeader(code int)

WriteHeader ...

type PrepareServer 

type PrepareServer func(ctx context.Context, port int, allowedOrigins []string) *http.Server

PrepareServer is the signature of a function that Knows how to prepare & initialise the server

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.