fxapp

package
v0.0.0-...-11620cc Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2019 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

Package fxapp builds upon https://godoc.org/go.uber.org/fx to provide a standardized functional driven application container.

DevOps Application Aspects

  • all app deployments must have an identity
  • each app is assigned a unique ID
  • application names may change, but the app ID is immutable
  • each app deployment is assigned a release ID, which maps to related app information, e.g.,
  • release notes
  • who were the persons involved - developers, testers, product managers, etc
  • discussions
  • test reports
  • unit test reports
  • acceptance test reports
  • performance test reports
  • performance profiles
  • etc
  • all running application deployment instances must be identified via an instance ID
  • used for troubleshooting, e.g., querying for application instance logs, metrics, etc
  • application logging is structured
  • zerolog is used to provided structured JSON logging
  • log events are strongly typed, i.e., domain specific
  • metrics
  • health checks

Index

Examples

Constants

View Source 
const (
	// 	type Data struct {
	//		StartTimeout 	uint `json:"start_timeout"`
	//		StopTimeout  	uint `json:"stop_timeout"`
	//		Provides     	[]string
	//		Invokes      	[]string
	//		DependencyGraph string `json:"dot_graph"` // DOT language visualization of the app dependency graph
	//	}
	InitializedEvent = "01DE4STZ0S24RG7R08PAY1RQX3"
	// 	type Data struct {
	//		Err string `json:"e"`
	//	}
	InitFailedEvent = "01DE4SWMZXD1ZB40QRT7RGQVPN"

	StartingEvent = "01DE4SXMG8W3KSPZ9FNZ8Z17F8"
	// 	type Data struct {
	//		Err string `json:"e"`
	//	}
	StartFailedEvent = "01DE4SY6RYCD0356KYJV7G7THW"

	// 	type Data struct {
	//		Duration uint
	//	}
	StartedEvent = "01DE4X10QCV1M8TKRNXDK6AK7C"

	ReadyEvent = "01DEJ5RA8XRZVECJDJFAA2PWJF"

	StoppingEvent = "01DE4SZ1KY60JQTF7XP4DQ8WGC"
	// 	type Data struct {
	//		Err string `json:"e"`
	//	}
	StopFailedEvent = "01DE4T0W35RPD6QMDS42WQXR48"

	// 	type Data struct {
	//		Duration uint
	//	}
	StoppedEvent = "01DE4T1V9N50BB67V424S6MG5C"
)

app lifecycle event IDs

View Source 
const (
	//  sample event data:
	//  {
	//    "id": "01DF3MNDKPB69AJR7ZGDNB3KA1",
	//    "desc_id": "01DF3MNDKP8DS3B04E2TKFHXD9",
	//    "description": "Foo",
	//    "red_impact": "app is unavailable",
	//    "yellow_impact": "app response times are slow",
	//    "timeout": 5000,
	//    "run_interval": 15000
	//  }
	HealthCheckRegisteredEvent = "01DF3FV60A2J1WKX5NQHP47H61"

	//  sample event data:
	//  {
	//    "id": "01DF3MNDKPB69AJR7ZGDNB3KA1",
	//	  "t": 155454546546,
	//	  "d": 9
	//  }
	HealthCheckResultEvent = "01DF3X60Z7XFYVVXGE9TFFQ7Z1"

	HealthCheckGaugeRegistrationErrorEvent = "01DF6M0T7K3DNSFMFQ26TM7XX4"
)

health check related events

View Source 
const (
	// HTTPServerError indicates an error occurred while handling a metrics scrape HTTP request.
	//
	// 	type Data struct {
	//		Err string `json:"e"`
	//	}
	HTTPServerError = "01DEDRH8A9X3SCSJRCJ4PM7749"

	// 	type Data struct {
	//		Addr      string
	//		Endpoints []string
	//	}
	HTTPServerStarting = "01DEFM9FFSH58ZGNPSR7Z4C3G2"
)

HTTP server related events

View Source 
const (
	AppIDLabel         = "a"
	AppReleaseIDLabel  = "r"
	AppInstanceIDLabel = "i"

	EventLabel = "z"
)

standard application labels used for metrics and logging

View Source 
const (
	// EnvconfigPrefix is the standard env var name prefix.
	// "APP12X" was chosen to represent 12-factor apps.
	EnvconfigPrefix = "APP12X"
)

envconfig related constants

View Source 
const HealthCheckMetricID = "U01DF4CVSSF4RT1ZB4EXC44G668"

HealthCheckMetricID is used as the prometheus metric name

View Source 
const (
	// 	type Data struct {
	//		Duration uint
	//	}
	LivenessProbeEvent = "01DF91XTSXWVDJQ4XJ432KQFXY"
)

probe related events

View Source 
const MetricsEndpoint = "01DF9JKZ73Y3V1AJN89B58D9HY"

MetricsEndpoint is used to construct the default metrics HTTP endpoint

View Source 
const PrometheusHTTPError = "01DEARG17HNQ606ARQNYFY7PG5"

PrometheusHTTPError indicates an error occurred while handling a metrics scrape HTTP request. 

type Data struct {
	Err string `json:"e"`
}

Variables

This section is empty.

Functions

func FindMetricFamilies 

func FindMetricFamilies(mfs []*dto.MetricFamily, accept func(mf *dto.MetricFamily) bool) []*dto.MetricFamily

FindMetricFamilies returns first metric families that match the filter

func FindMetricFamily 

func FindMetricFamily(mfs []*dto.MetricFamily, accept func(mf *dto.MetricFamily) bool) *dto.MetricFamily

FindMetricFamily returns the first metric family that matches the filter

func LoadIDsFromEnv 

func LoadIDsFromEnv() (ID, ReleaseID, error)

LoadIDsFromEnv tries to load the app descriptor from env vars:

  • APP12X_ID
  • APP12X_RELEASE_ID

func RegisterProcessMetricsCollector 

func RegisterProcessMetricsCollector(registerer prometheus.Registerer) error

RegisterProcessMetricsCollector is used to register the prometheus built in process metrics collector.

Types

type App 

type App interface {
	ID() ID
	ReleaseID() ReleaseID
	InstanceID() InstanceID

	Options
	LifeCycle

	// Run will start running the application and blocks until the app is shutdown.
	// It waits to receive a SIGINT or SIGTERM signal to shutdown the app.
	Run() error

	// StopAsync signals the app to shutdown. This method does not block, i.e., application shutdown occurs async.
	//
	// StopAsync can only be called after the app has been started - otherwise an error is returned.
	Shutdown() error
}

App represents a functional application container, leveraging fx (https://godoc.org/go.uber.org/fx) as the underlying framework. Functional means, the application behavior is defined via functions.

The key is understanding the application life cycle. The application transitions through the following lifecycle states:

  1. Initialized
  2. Starting
  3. Started
  4. Ready
  5. Stopping
  6. Done

When building an application, functions are registered which specify how to:

Function arguments are provided via dependency injection by registering provider constructor functions with the application. Provider constructor functions are lazily invoked when needed inject function dependencies.

Application Descriptor

The application descriptor is another way to say application metadata (see `Desc`). Every application has the following metadata:

  • ID - represented as a XID
  • release ID - an application has many versions, but not all versions are released.
  • can be used to look up additional release artifacts, e.g., release notes, test reports, etc

Application Logging

Zerolog (https://godoc.org/github.com/rs/zerolog) is used as the structured JSON logging framework. A `*zerolog.Logger` is automatically provided when building the application and available for dependency injection. The application logger context is augmented with application metadata and an event ID, e.g., 

{"a":"01DE2GCMX5ZSVZXE2RTY7DCB88","r":"01DE2GCMX570BXG6468XBXNXQT","x":"01DE2GCMX5Q9S44S8166JX10WV","z":"01DE30RAEQGQBS0THBCVKVHFSW","t":1561304912,"m":"[Fx] RUNNING"}

where a -> app ID
	  r -> app release ID
	  x -> app instance ID
	  z -> event ID
	  t -> timestamp - in Unix time format
	  m -> message

The zerolog application logger is plugged in as the go standard log, where log events are logged with no level and logged using a component logger named 'log' ("c":"log")

All application log events should be defined as an `Event` and logged via `Event` logger functions. This makes it easy to document and understand application logs. All events are assigned a unique identifier - it is recommended to use a XID as the event name.

Prometheus Metrics

The following are automatically provided for the app:

  • prometheus.Registerer
  • prometheus.Gatherer

Prometheus metrics are automatically exposed via HTTP (using https://godoc.org/github.com/prometheus/client_golang/prometheus/promhttp#HandlerFor). `PrometheusHTTPHandlerOpts` is used to configure the Prometheus HTTP handler. By default the following options are used:

  • Endpoint = /01DF9JKZ73Y3V1AJN89B58D9HY
  • Timeout = 5 secs
  • ErrorHandling = promhttp.HTTPErrorOnError (HTTP status code 500 is returned upon the first error encountered)

If a `PrometheusHTTPHandlerOpts` is provided, then it will be used instead. However, if the provided endpoint is blank, then it will be set to '/metrics' and if timeout is zero, then it will be set to 5 secs.

TODO: Metrics are logged on a scheduled basis. By default, every minute - but is configurable.

Health Checks

The application provides support to register health checks, which will be automatically run on a schedule.

  • Health checks are integrated with the readiness and liveliness probes. Any Red health checks will cause the probes to fail.
  • Health check results are logged
  • Health checks are integrated with metrics as gauges, using the health check status as the gauge value.
  • the health check gauge is designed as a gauge vec, where the health check name is "U01DF4CVSSF4RT1ZB4EXC44G668" (defined by the `HealthCheckMetricID` const)
  • health check gauges have the following labels:
  • "h" - health check ID
  • "d" - health check descriptor ID
  • health checks are registered with the app readiness probe. The app is not ready until all health checks are pass green. If any health checks fail, i.e., not green, then the app will fail to start up.
  • TODO: health check GRPC API

Readiness Probe

A readiness probe indicates whether the application is ready to service requests. A wait group mechanism is used to implement application readiness functionality via `ReadinessWaitGroup`. During application initialization, components can register with the `ReadinessWaitGroup` and notify the app when it is ready.

A readiness probe HTTP endpoint is exposed:

  • endpoint: /01DEJ5RA8XRZVECJDJFAA2PWJF - corresponds to `ReadyEvent`
  • the handler is linked to `ReadinessWaitGroup`
  • if the app is ready, then HTTP 200 is returned
  • if the app is not ready, then HTTP 503 is returned with response returns header `x-readiness-wait-group-count` set to the number of components that the app is waiting on

Liveliness Probe

The application liveness probe fails if any health checks fail with a RED status.

A liveness probe HTTP endpoint is exposed:

  • /01DF91XTSXWVDJQ4XJ432KQFXY - corresponds to `LivenessProbeEvent`
  • HTTP 503 is returned if the probe fails
  • LivenessProbeEvent is logged each time the endpoint handler is invoked
  • the probe duration is logged with the event

HTTP server support

Any HTTPHandler(s) that are discovered, i.e., have been provided, will be registered with the app's HTTP server. HTTP server settings can be provided via an *http.Server (NOTE: http.Server.Handler will be overwritten using http handlers that are provided by the app). If no *http.Server is discovered, then the app will automatically create an HTTP server with the following settings:

  • Addr: ":8008",
  • ReadHeaderTimeout: time.Second,
  • MaxHeaderBytes: 1024,

When building the app, the app HTTP server can be disabled - when using the App in unit testing, it is best to disable the HTTP server if HTTP functionality is not being tested.

Automatically Provided

  • Application Metadata
  • Desc
  • InstanceID
  • fx provided
  • fx.Lifecycle - for components to use to bind to the app lifecycle
  • fx.Shutdowner - used to trigger app shutdown
  • fx.Dotgraph - contains a DOT language visualization of the app dependency graph
  • Prometheus metrics related
  • prometheus.Gatherer
  • prometheus.Registerer
  • Health RegisteredCheck related
  • health.Registry
  • health.Scheduler
  • Probes
  • ReadinessWaitGroup - the readiness probe uses the ReadinessWaitGroup to know when the application is ready to serve requests
  • LivenessProbe - returns an error if any health check is RED
  • Application Infrastructure Related
  • *zerolog.Logger
  • *http.Server
  • can be disabled
  • can be customized by providing it
  • HTTP endpoints
  • /01DF9JKZ73Y3V1AJN89B58D9HY - exposes prometheus metrics
  • /01DEJ5RA8XRZVECJDJFAA2PWJF - readiness probe
  • /01DF91XTSXWVDJQ4XJ432KQFXY - liveness probe

type BuildInfo 

type BuildInfo struct {
	Path string    // The main package Path
	Main Module    // The main module information
	Deps []*Module // Module dependencies
}

BuildInfo represents the build information read from the running binary.

func ReadBuildInfo 

func ReadBuildInfo() (*BuildInfo, error)

ReadBuildInfo returns the build information embedded in the running binary. The information is available only in binaries built with module support.

func (*BuildInfo) MarshalZerologObject 

func (b *BuildInfo) MarshalZerologObject(e *zerolog.Event)

MarshalZerologObject implements zerolog.LogObjectMarshaler interface

type Builder 

type Builder interface {
	// Provide is used to provide dependency injection
	Provide(constructors ...interface{}) Builder
	// Invoke is used to register application functions, which will be invoked to to initialize the app.
	// The functions are invoked in the order that they are registered.
	Invoke(funcs ...interface{}) Builder

	SetStartTimeout(timeout time.Duration) Builder
	SetStopTimeout(timeout time.Duration) Builder

	// LogWriter is used as the zerolog writer.
	//
	// By default, stderr is used.
	LogWriter(w io.Writer) Builder
	LogLevel(level LogLevel) Builder

	// Error handlers
	HandleInvokeError(errorHandlers ...func(error)) Builder
	HandleStartupError(errorHandlers ...func(error)) Builder
	HandleShutdownError(errorHandlers ...func(error)) Builder
	// HandleError will handle any app error, i.e., app function invoke errors, app startup errors, and app shutdown errors.
	HandleError(errorHandlers ...func(error)) Builder

	// Populate sets targets with values from the dependency injection container during application initialization.
	// All targets must be pointers to the values that must be populated.
	// Pointers to structs that embed fx.In are supported, which can be used to populate multiple values in a struct.
	//
	// NOTE: this is useful for unit testing
	Populate(targets ...interface{}) Builder

	// DisableHTTPServer disables the HTTP server
	//
	// Uses cases for disabling the HTTP server:
	//  - when using the App for running tests the HTTP server can be disabled to reduce overhead. It also enables tests
	//    to be run in parallel
	//  - for CLI based apps
	DisableHTTPServer() Builder

	Build() (App, error)
}

Builder is used to construct a new App instance.

func NewBuilder 

func NewBuilder(id ID, releaseID ReleaseID) Builder

NewBuilder constructs a new Builder

type HTTPEndpoint 

type HTTPEndpoint struct {
	Path    string
	Handler func(http.ResponseWriter, *http.Request)
}

HTTPEndpoint maps an HTTP handler to an HTTP path

type HTTPHandler 

type HTTPHandler struct {
	fx.Out

	HTTPEndpoint `group:"HTTPHandler"`
}

HTTPHandler is used to group HTTPEndpoint(s) together. The HTTPEndpoint(s) are automatically registered with the app's HTTP server.

func NewHTTPHandler 

func NewHTTPHandler(path string, handler func(http.ResponseWriter, *http.Request)) HTTPHandler

NewHTTPHandler constructs a new HTTPHandler

type ID 

type ID ulid.ULID

ID corresponds to an application

type InstanceID 

type InstanceID ulid.ULID

InstanceID corresponds to an application instance

type Interval 

type Interval time.Duration

Interval indicates the duration is meant to be used as an interval

type LifeCycle 

type LifeCycle interface {
	// Starting signals that the app is starting.
	// Closing the channel is the signal.
	Starting() <-chan struct{}
	// Started signals that the app has fully started
	Started() <-chan struct{}
	// Ready means the app is ready to serve requests
	Ready() <-chan struct{}
	// Stopping signals that app is stopping.
	// The channel is closed after the stop signal is sent.
	Stopping() <-chan os.Signal
	// Done signals that the app has shutdown.
	// The channel is closed after the stop signal is sent.
	// If the app fails to startup, then the channel is simply closed, i.e., no stop signal will be sent on the channel.
	Done() <-chan os.Signal
}

LifeCycle defines the application lifecycle.

type LivenessProbe 

type LivenessProbe func() error

LivenessProbe checks if the app is healthy. It returns an error if probe fails, indicating the app is unhealthy.

type LogLevel 

type LogLevel uint

LogLevel defines the supported app log levels 

const (
	DebugLogLevel LogLevel = iota
	InfoLogLevel
	WarnLogLevel
	ErrorLogLevel
)

LogLevel enum

func (LogLevel) ZerologLevel 

func (level LogLevel) ZerologLevel() zerolog.Level

ZerologLevel maps LogLevel to a zerolog.Level

type MetricDesc 

type MetricDesc struct {
	Name string
	Help string
	MetricType
	Labels []string // label names
}

MetricDesc is used to describe the metric

func DescsFromMetricFamilies 

func DescsFromMetricFamilies(mfs []*dto.MetricFamily) []*MetricDesc

DescsFromMetricFamilies extracts metric descriptors from gathered metrics

func NewMetricDesc 

func NewMetricDesc(mf *dto.MetricFamily) *MetricDesc

NewMetricDesc extracts the metric descriptor from the gathered metrics

type MetricType 

type MetricType uint8

MetricType represents a metric type enum 

const (
	Untyped MetricType = iota
	Counter
	Gauge
	Histogram
	Summary
)

metric type enum values

type Module 

type Module struct {
	Path     string
	Version  string
	Checksum string
}

Module represents an app module dependency

func NewModule 

func NewModule(m *debug.Module) *Module

NewModule constructs a new Module

func (*Module) MarshalZerologObject 

func (m *Module) MarshalZerologObject(e *zerolog.Event)

MarshalZerologObject implements zerolog.LogObjectMarshaler interface

type Options 

type Options interface {
	// StartTimeout returns the app start timeout. If the app takes longer than the specified timeout, then the app will
	// fail to run.
	StartTimeout() time.Duration
	// StopTimeout returns the app shutdown timeout. If the app takes longer than the specified timeout, then the app shutdown
	// will be aborted.
	StopTimeout() time.Duration

	// ConstructorTypes returns the registered constructor types
	ConstructorTypes() []reflect.Type
	// FuncTypes returns the registered function types
	FuncTypes() []reflect.Type
}

Options represent application options that were used to configure and build app.

type PrometheusHTTPHandlerOpts 

type PrometheusHTTPHandlerOpts struct {
	// Timeout returns the handler response timeout.
	//
	// If handling a request takes longer than Timeout, it is responded to with 503 ServiceUnavailable and a suitable Message.
	// No timeout is applied if Timeout is 0 or negative. Note that with the current implementation, reaching the timeout
	// simply ends the HTTP requests as described above (and even that only if sending of the body hasn't started yet), while
	// the bulk work of gathering all the metrics keeps running in the background (with the eventual result to be thrown away).
	// Until the implementation is improved, it is recommended to implement a separate timeout in potentially slow Collectors.
	Timeout  time.Duration
	Endpoint string
	// ErrorHandling defines how errors are handled.
	//
	// Note: errors are always logged regardless of the configured ErrorHandling
	ErrorHandling promhttp.HandlerErrorHandling
}

PrometheusHTTPHandlerOpts PrometheusHTTPServer options

Example 
app, err := fxapp.NewBuilder(fxapp.ID(ulids.MustNew()), fxapp.ReleaseID(ulids.MustNew())).
	// Provide custom PrometheusHTTPHandlerOpts, which will be used to configure the Prometheus HTTP handler
	Provide(func() fxapp.PrometheusHTTPHandlerOpts {
		opts := fxapp.DefaultPrometheusHTTPHandlerOpts()
		opts.Timeout = 10 * time.Second
		return opts
	}).
	Invoke(func() {}).
	Build()

if err != nil {
	log.Panic(err)
}

go app.Run()
<-app.Ready()
defer func() {
	app.Shutdown()
	<-app.Done()
}()

resp, err := retryablehttp.Get(fmt.Sprintf("http://:8008/%s", fxapp.MetricsEndpoint))
switch {
case err != nil:
	log.Panic(err)
case resp.StatusCode != http.StatusOK:
	log.Panicf("HTTP request failed: %v : %v", resp.StatusCode, resp.Status)
default:
	reader := bufio.NewReader(resp.Body)
	for {
		line, err := reader.ReadString('\n')
		if err != nil {
			break
		}
		log.Println(line)
	}
}

Output:

func DefaultPrometheusHTTPHandlerOpts 

func DefaultPrometheusHTTPHandlerOpts() PrometheusHTTPHandlerOpts

DefaultPrometheusHTTPHandlerOpts constructs a new PrometheusHTTPHandlerOpts with the following options:

  • timeout: 5 secs
  • endpoint: /metrics
  • error handling: promhttp.HTTPErrorOnError
  • Serve an HTTP status code 500 upon the first error encountered. Report the error message in the body.

type ReadinessWaitGroup 

type ReadinessWaitGroup interface {
	Add(delta uint)
	Inc()

	// Count returns the wait group counter value. When the count is zero, it means the wait group is done.
	Count() uint

	// Done decrements the wait group counter by one
	Done()

	// Ready returns a chan that is used to signal when the wait group counter is zero.
	Ready() <-chan struct{}
}

ReadinessWaitGroup is used by application components to signal when they are ready to service requests

func NewReadinessWaitgroup 

func NewReadinessWaitgroup(count uint) ReadinessWaitGroup

NewReadinessWaitgroup returns a new ReadinessWaitGroup initialized with the specified count

type ReleaseID 

type ReleaseID ulid.ULID

ReleaseID corresponds to an application release

type Timeout 

type Timeout time.Duration

Timeout indicates the duration is meant to be used as a timeout

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.