oom

package module
v0.0.0-...-23692ba Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 9, 2015 License: MIT Imports: 12 Imported by: 0

README

oom

GoDoc Go Report Card

oom is a small collection of middlewares for chi that help you handle your system running out of memory

In theory memory use should be more or less stable or at least predictable and you should be scaling our by adding new servers.

In practice you have:

  • leaky GCO libs
  • spiky load or memory use
  • thundering herd issues
  • not enough time to track memory leak
  • all of the above
Stop processing new requests until GC catches up

Put oom.Protect(t float64) early in your middleware chain to terminate request processing when total system memory use exceeds t and return HTTP 503 response

Example

func main() {
  r := chi.NewRouter()

  // Start returning 503 if memory use exceeds 90% of total system memory
  r.Use(oom.Protect(0.9))
  r.Get("/", GetResource)

  // This endpoint uses a lot of memory, use lower threshold
  r.Post("/", oom.Protect(0.7), CreateResource)

  http.ListenAndServe(":80", r)
}
Restart process

Put oom.Selfdestruct(t float64) in your middleware chain to terminate process (not just goroutine!) when total system memory use exceeds t. Termination will be handled by sending SIGTERM to the process letting you gracefully stop it

Example

func main() {
  r := chi.NewRouter()

  // This is one leaky http handler that will make process eventually consume
  // all the memory available and crash.
  // Gracefully terminate it when system memory use exceeds 95% and have docker
  // --restart=always start it again
  r.Use(oom.Selfdestruct(0.95))
  r.Get("/", GetResource)
  r.Post("/", CreateResource)

  graceful.AddSignal(syscall.SIGTERM)
  graceful.Timeout(10 * time.Second) // Wait timeout for handlers to finish.
  graceful.ListenAndServe(":80", r)
  graceful.Wait()
}
Call a custom function to handle restart/cleanup for you

Put oom.SelfdestructFn(t float64, fn func(context.Context) bool) in your middleware chain to terminate process (not just goroutine!) when total system memory use exceeds t by calling your own function that will handle the termination, or do something completely different.

Example

func main() {
  r := chi.NewRouter()

  // This is one leaky http handler that will make process eventually consume
  // all the memory available and crash.
  // Handle that with a custom function
  r.Use(oom.SelfdestructFn(0.95, customKillFunction))
  r.Get("/", GetResource)
  r.Post("/", CreateResource)

  http.ListenAndServe(":80", r)
}
Change memory use sampling frequency

Use oom.SetUpdateInteval(i time.Duration) to only update current memory use at most every i. By default it's updated on every request.

Example

func main() {
  r := chi.NewRouter()

  // This app receives a very high volume of requests that have a tiny impact on
  // memory use. Sampling system memory use once a second is enough
  oom.SetUpdateInterval(time.Second)

  // Start returning 503 if memory use exceeds 90% of total system memory
  r.Use(oom.Protect(0.9))
  r.Get("/", GetResource)

  // This endpoint uses a lot of memory, use lower threshold
  r.Post("/", oom.Protect(0.7), CreateResource)

  http.ListenAndServe(":80", r)
}
OS compatibility

oom middlewares are only able to get memory usage data on Linux. Kernels 3.14 and newer offer better accuracy. Pull requests adding other OS support are very welcome!

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MemoryUsage

func MemoryUsage() float64

MemoryUsage usage returns used to total memory ratio

func Protect

func Protect(limit float64) func(chi.Handler) chi.Handler

Protect is a middleware that cancels incoming requests and returns a 503 HTTP code if system memory use exceeds the threshold set.

This middleware should be inserted fairly early in the middleware stack to ensure that request is cancelled early

Protect accepts a single parameter - a float64 defining a fraction of the system memory process can use before it starts cancelling requests

Right now it relies on memory usage data provided by /proc/meminfo so it is Linux specific. For convenience (dev envs) it will do nothing on other archs

func Selfdestruct

func Selfdestruct(limit float64) func(chi.Handler) chi.Handler

Selfdestruct is a middleware that terminates current process if system memory use exceeds the threshold set.

Most common use case is automatic restart of leaky *cough*GCO*cough service all you need is this middleware and something that will start the process again after it's been terminated (docker --restart=always for example )

OOMSelfdestruct accepts one parameter - a float64 defining a fraction of the system memory that can be used before process self-terminates using system signal (SIG_TERM). As long as the app handles shutdown gracefully on SIG_TERM it should just work

Right now it relies on memory usage data provided by /proc/meminfo so it is Linux specific. For convenience (dev envs) it will do nothing on other archs

func SelfdestructFn

func SelfdestructFn(limit float64, fn func(ctx context.Context) bool) func(chi.Handler) chi.Handler

SelfdestructFn is a version of Selfdestruct that accepts a custom termination function ( func(context.Context) bool ) that handles the process shutdown. Use it if your app requires extra steps to gracefully shut down.

func SetUpdateInteval

func SetUpdateInteval(i time.Duration)

SetUpdateInteval updates the interval between MemInfo updates. By default Updates happen on every request

func SignalSelfdestructGroup

func SignalSelfdestructGroup(ctx context.Context) bool

SignalSelfdestructGroup sends SIGTERM to whole process group

func SignalSelfdestructProcess

func SignalSelfdestructProcess(ctx context.Context) bool

SignalSelfdestructProcess sends SIGTERM to current process

Types

type MemInfo

type MemInfo struct {
	Values map[string]uint64
	// contains filtered or unexported fields
}

MemInfo contains data feched from /proc/meminfo, mutex preventing races and update frequency related fields

func (*MemInfo) Available

func (m *MemInfo) Available() uint64

Available returns the amount of still RAM available It uses new process mem available estimation - factors in reclaiming file buffers and cache. On Linux kernels 3.14+ it tens to err on the side of caution On Linux kernels older than 3.14 it's a tiny bit too optimistic, unless you are using next to no file buffers

func (*MemInfo) Total

func (m *MemInfo) Total() uint64

Total returns total RAM of system

func (*MemInfo) Update

func (m *MemInfo) Update() error

Update get fresh data from /proc/meminfo

func (*MemInfo) Used

func (m *MemInfo) Used() uint64

Used returns the amount of non-reclaimable memory used

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL