README
¶
oom
oom is a small collection of middlewares for chi that help you handle your system running out of memory
In theory memory use should be more or less stable or at least predictable and you should be scaling our by adding new servers.
In practice you have:
- leaky GCO libs
- spiky load or memory use
- thundering herd issues
- not enough time to track memory leak
- all of the above
Stop processing new requests until GC catches up
Put oom.Protect(t float64) early in your middleware chain to terminate request processing when total system memory use exceeds t and return HTTP 503 response
Example
func main() {
r := chi.NewRouter()
// Start returning 503 if memory use exceeds 90% of total system memory
r.Use(oom.Protect(0.9))
r.Get("/", GetResource)
// This endpoint uses a lot of memory, use lower threshold
r.Post("/", oom.Protect(0.7), CreateResource)
http.ListenAndServe(":80", r)
}
Restart process
Put oom.Selfdestruct(t float64) in your middleware chain to terminate process (not just goroutine!) when total system memory use exceeds t. Termination will be handled by sending SIGTERM to the process letting you gracefully stop it
Example
func main() {
r := chi.NewRouter()
// This is one leaky http handler that will make process eventually consume
// all the memory available and crash.
// Gracefully terminate it when system memory use exceeds 95% and have docker
// --restart=always start it again
r.Use(oom.Selfdestruct(0.95))
r.Get("/", GetResource)
r.Post("/", CreateResource)
graceful.AddSignal(syscall.SIGTERM)
graceful.Timeout(10 * time.Second) // Wait timeout for handlers to finish.
graceful.ListenAndServe(":80", r)
graceful.Wait()
}
Call a custom function to handle restart/cleanup for you
Put oom.SelfdestructFn(t float64, fn func(context.Context) bool) in your middleware chain to terminate process (not just goroutine!) when total system memory use exceeds t by calling your own function that will handle the termination, or do something completely different.
Example
func main() {
r := chi.NewRouter()
// This is one leaky http handler that will make process eventually consume
// all the memory available and crash.
// Handle that with a custom function
r.Use(oom.SelfdestructFn(0.95, customKillFunction))
r.Get("/", GetResource)
r.Post("/", CreateResource)
http.ListenAndServe(":80", r)
}
Change memory use sampling frequency
Use oom.SetUpdateInteval(i time.Duration) to only update current memory use at most every i. By default it's updated on every request.
Example
func main() {
r := chi.NewRouter()
// This app receives a very high volume of requests that have a tiny impact on
// memory use. Sampling system memory use once a second is enough
oom.SetUpdateInterval(time.Second)
// Start returning 503 if memory use exceeds 90% of total system memory
r.Use(oom.Protect(0.9))
r.Get("/", GetResource)
// This endpoint uses a lot of memory, use lower threshold
r.Post("/", oom.Protect(0.7), CreateResource)
http.ListenAndServe(":80", r)
}
OS compatibility
oom middlewares are only able to get memory usage data on Linux. Kernels 3.14 and newer offer better accuracy. Pull requests adding other OS support are very welcome!
Documentation
¶
Index ¶
- func MemoryUsage() float64
- func Protect(limit float64) func(chi.Handler) chi.Handler
- func Selfdestruct(limit float64) func(chi.Handler) chi.Handler
- func SelfdestructFn(limit float64, fn func(ctx context.Context) bool) func(chi.Handler) chi.Handler
- func SetUpdateInteval(i time.Duration)
- func SignalSelfdestructGroup(ctx context.Context) bool
- func SignalSelfdestructProcess(ctx context.Context) bool
- type MemInfo
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Protect ¶
Protect is a middleware that cancels incoming requests and returns a 503 HTTP code if system memory use exceeds the threshold set.
This middleware should be inserted fairly early in the middleware stack to ensure that request is cancelled early
Protect accepts a single parameter - a float64 defining a fraction of the system memory process can use before it starts cancelling requests
Right now it relies on memory usage data provided by /proc/meminfo so it is Linux specific. For convenience (dev envs) it will do nothing on other archs
func Selfdestruct ¶
Selfdestruct is a middleware that terminates current process if system memory use exceeds the threshold set.
Most common use case is automatic restart of leaky *cough*GCO*cough service all you need is this middleware and something that will start the process again after it's been terminated (docker --restart=always for example )
OOMSelfdestruct accepts one parameter - a float64 defining a fraction of the system memory that can be used before process self-terminates using system signal (SIG_TERM). As long as the app handles shutdown gracefully on SIG_TERM it should just work
Right now it relies on memory usage data provided by /proc/meminfo so it is Linux specific. For convenience (dev envs) it will do nothing on other archs
func SelfdestructFn ¶
SelfdestructFn is a version of Selfdestruct that accepts a custom termination function ( func(context.Context) bool ) that handles the process shutdown. Use it if your app requires extra steps to gracefully shut down.
func SetUpdateInteval ¶
SetUpdateInteval updates the interval between MemInfo updates. By default Updates happen on every request
func SignalSelfdestructGroup ¶
SignalSelfdestructGroup sends SIGTERM to whole process group
func SignalSelfdestructProcess ¶
SignalSelfdestructProcess sends SIGTERM to current process
Types ¶
type MemInfo ¶
MemInfo contains data feched from /proc/meminfo, mutex preventing races and update frequency related fields
func (*MemInfo) Available ¶
Available returns the amount of still RAM available It uses new process mem available estimation - factors in reclaiming file buffers and cache. On Linux kernels 3.14+ it tens to err on the side of caution On Linux kernels older than 3.14 it's a tiny bit too optimistic, unless you are using next to no file buffers
Source Files