profiler

README

Profiler

profiler is an abstraction layer around the golang profiling stdlib, namely runtime/pprof and runtime/trace to simplify and make profiling configuration easy and fast. It offers an assortment of options and configurations as well as various non stdlib options such as the fgprof wall clock option, which pieces together a single .pprof file which includes both cpu ON and OFF data. A basic example for each of the profilers is outlined below.

By default profiler will listen for SIGTERM & SIGINT in order to close out the pprof/trace files correctly, however in some cases this is undesirable. If you would like full control of cleaning up then use the WithoutSignalHandling() functional option to any invocation of .Start().

---

1⃣ CPU Profiling

Profiling CPU can help identify areas of your code where most CPU cycle execution time is spent. For programs with a lot of IO wait etc, the graph may not be extremely useful, trace may be of better benefit there.

import (
    "github.com/symonk/profiler"
)

func main() {
    defer profiler.Start(profiler.WithCPUProfiler()).Stop()
    /* your code here */
}

---

2⃣ Heap Profiling

Heap profiling reports memory allocation samples, useful for monitoring current and historical memory usage and to check for potential memory leaks in your program. Heap profiling tracks the allocation sites for all live objects in the application memory and for all objects allocated since the program has started. By default heap profiling will display live objects, scaled by size.

package main

import (
    "github.com/symonk/profiler"
)

func main() {
    // additional options available here are: `profiler.WithMemoryProfileRate(...)`
    defer profiler.Start(profiler.WithHeapProfiler()).Stop()
    /* your code here */
}

---

3⃣ Alloc Profiling

Alloc profiling is essentially the same as heap profiling except rather than the default of live objects scaled by size, it reports the -alloc_space data, that is the total number of bytes allocated since the program has began (including garbage collected bytes).

package main

import (
    "github.com/symonk/profiler"
)

func main() {
    // additional options available here are: `profiler.WithMemoryProfileRate(...)`
    defer profiler.Start(profiler.WithAllocProfiler()).Stop()
    /* your code here */
}

---

4⃣ Block Profiling

Block profiling captures how long a program spends off CPU blocked by either a mutex or a channel operation. The following events are recorded:

• select operations
• channel send operations
• channel receive operations
• semacquire operations (sync -> Mutex.Lock(), sync -> RWMutex.RLock(), sync -> RWMutex.Lock(), sync -> WaitGroup.Wait())
• notify list operations (sync -> Conf.Wait())

[!CAUTION] Block profiles do not include time in sleep, IO wait etc and block events are only recorded when the block has cleared, as such they are not appropriate to see why a program is currently hanging.

package main

import (
    "github.com/symonk/profiler"    
)

func main() {
    defer profiler.Start(profiler.WithBlockProfiler()).Stop()
    /* your code here */
}

---

5⃣ Thread Profiling

Thread profiling shows stack traces of code that caused new OS level threads to be created by the go scheduler. This is implemented in this library but it has been broken since 2013. Should it be fixed in future or if you wish to use it now, consider it experimental.

[!CAUTION] This has been broken since 2013, do not use it!

package main

import (
    "github.com/symonk/profiler"
)

func main() {
    defer profiler.Start(profiler.WithThreadProfiler()).Stop()
    /* your code here */
}

---

6⃣ Goroutine Profiling

The go runtime keeps track of all goroutines internally in a slice. This tracks both active and dead goroutines (dead goros are reused later if requested). Active goroutines are routines waiting on io, blocking or executing. Profiling goroutines causes a 'stop-the-world' event which is o(n) depending on how many goroutines you have so caution is advised when profiling goroutines at scale in production.

package main

import (
    "github.com/symonk/profiler"
)

func main() {
    defer profiler.Start(profiler.WithGoroutineProfiler()).Stop()
    /* your code here */
}

---

7⃣ Mutex Profiling

Mutex profiling differs from block profiling in the sense that block profiling records how long a goroutine took to acquire a .Lock() but the mutex profile tracks how long a goroutine was waiting for an .Unlock(). Contention is not recorded until the mutexes are unlocked, so again this is not viable for checking why a program is currently hanging/blocking 'now'.

package main

import (
    "github.com/symonk/profiler"
)

func main() {
    defer profiler.Start(profiler.WithMutexProfiler()).Stop()
    /* your code here */
}

---

8⃣ Clock Profiling

Clock profiling allows monitoring of both off and on CPU time collectively, this paints a nice picture of where CPU cycles are spent, but also where IO perhaps is causing delays too.

package main

import (
    "github.com/symonk/profiler"
)

func main() {
    defer profiler.Start(profiler.WithClockProfiling()).Stop()
    /* your code here */
}

---

Available Options

• WithAllocProfiler => Enables allocation (memory) profiling.
• WithBlockProfiler => Enables block profiling.
• WithCPUProfiler => Enables CPU profiling (default).
• WithCallback => User defined callback that has the profiler in scope, invoked after teardown.
• WithClockProfiling => Enables CPU on & off profiling (non stdlib).
• WithHeapProfiler => Enables heap (memory) profiling.
• WithMemoryProfilingRate => Sets the profiling rate for memory related profiling samples.
• WithMutexFraction => Sets the fraction rate used in conjunction with mutex profiling.
• WithProfileFileLocation => Sets the custom folder location for the pprof / trace files.
• WithQuietOutput => Suppresses writing to stdout/printing.
• WithRealTimeData => Spins a http server for the lifetime of the profiling for real curl/fetching if desired.
• WithThreadProfiler => Enables the os thread creation profiling.
• WithTracing => Enables the tracing.
• WithoutSignalHandling => Prevents the profiler tool signal handling, allow more fine grained user control.

Documentation

Index

• Constants
• Variables
• func CreateProfileFile(folder string, name string) (*os.File, error)
• type CallbackFunc
• type FinalizerFunc
• type Mode
• type ProfileOption
  • func WithAllocProfiler() ProfileOption
  • func WithBlockProfiler() ProfileOption
  • func WithCPUProfiler() ProfileOption
  • func WithCallback(callback CallbackFunc) ProfileOption
  • func WithClockProfiling() ProfileOption
  • func WithHeapProfiler() ProfileOption
  • func WithMemoryProfilingRate(rate int) ProfileOption
  • func WithMutexFraction(rate int) ProfileOption
  • func WithProfileFileLocation(path string) ProfileOption
  • func WithQuietOutput() ProfileOption
  • func WithRealTimeData() ProfileOption
  • func WithThreadProfiler() ProfileOption
  • func WithTracing() ProfileOption
  • func WithoutSignalHandling() ProfileOption
• type Profiler
  • func New(options ...ProfileOption) *Profiler
  • func Start(options ...ProfileOption) *Profiler
  • func (p *Profiler) SetProfileFile(name string)
  • func (p *Profiler) Stop()
• type StrategyFunc

Constants

const (
	CPUFileName          = "cpu.pprof"
	MemoryFileName       = "memory.pprof" // Covers heap and alloc
	BlockFileName        = "block.pprof"
	GoroutineFileName    = "goroutine.pprof"
	MutexFileName        = "mutex.pprof"
	ThreadCreateFileName = "threadcreate.pprof"
	TraceFileName        = "trace.out"
	ClockFileName        = "clock.pprof"
)

Variables

var StrategyMap = map[Mode]StrategyFunc{
	CPUMode:          cpuStrategyFn,
	MemoryHeapMode:   heapStrategyFn,
	MemoryAllocMode:  allocStrategyFn,
	MutexMode:        mutexStrategyFn,
	BlockMode:        blockStrategyFn,
	GoroutineMode:    goroutineStrategyFn,
	ThreadCreateMode: threadCreateStrategyFn,
	TraceMode:        traceStrategyFn,
	ClockMode:        clockStrategyFn,
}

Functions

func CreateProfileFile

func CreateProfileFile(folder string, name string) (*os.File, error)

CreateProfileFile takes the user defined folder (or working dir) if omitted and attempts to make the full folder tree. If the folder creation fails, a temp folder is created and the file is written to that location. File names are currently not customisable and are provided by the caller based on the profile mode selected.

Types

type CallbackFunc

type CallbackFunc func(p *Profiler)

CallbackFunc is a function that can be supplied with the WithCallback option to be executed when the profiling instance is performing teardown. It has access to the *Profiler instance.

type FinalizerFunc

type FinalizerFunc func() error

FinalizerFunc is a function that is invokved during the teardown period of the profiling instance.

type Mode

type Mode int

const (
	// List of available runtime profiles
	CPUMode Mode = iota
	MemoryHeapMode
	MemoryAllocMode
	BlockMode
	GoroutineMode
	MutexMode
	ThreadCreateMode
	TraceMode
	ClockMode
)

type ProfileOption

type ProfileOption func(*Profiler)

ProfileOption is a functional option to configure profiler instances.

func WithAllocProfiler

func WithAllocProfiler() ProfileOption

WithAllocProfiler enables the Alloc Profiler. Alloc Profiling is useful for determining where memory is being allocated and where it is being retained. This is different to Heap Profiling as it will show you where memory is being allocated, but not necessarily where it is being retained. This is useful for finding memory leaks. This is only available in Go 1.12 and later. The rate at which the profiler samples memory allocations can be set with the WithMemoryProfilingRate option.

func WithBlockProfiler

func WithBlockProfiler() ProfileOption

TODO: Doc

func WithCPUProfiler

func WithCPUProfiler() ProfileOption

WithCPUProfiler enables the CPU Profiler. CPU Profiling is useful for determining where a program is spending CPU cycles (as opposed) to sleeping or waiting for IO.

func WithCallback

func WithCallback(callback CallbackFunc) ProfileOption

WithCallback executes a user defined function when clean up occurs. This function is also fired on sigterm handling when the option is enabled. Callbacks have access to the underlying *Profiler instance, this is typically useful if you wanted to do some logic with the profile files that are written as the callback is only fired when the profile is complete, such as persisting a profile file to a central store etc.

func WithClockProfiling

func WithClockProfiling() ProfileOption

WithClockProfiling utilises wall clock profiling powered by https://github.com/felixge/fgprof. This allows you to profile both CPU ON and OFF wait in tandem, painting a nice picture. Go runtimes built in CPU profiler only displays cpu ON time.

func WithHeapProfiler

func WithHeapProfiler() ProfileOption

WithHeapProfiler enables the Heap Profiler. Heap Profiling is useful for determining where memory is being allocated and where it is being retained.

func WithMemoryProfilingRate

func WithMemoryProfilingRate(rate int) ProfileOption

WithMemoryProfilingRate sets the rate at which the memory profiler samples memory allocations for both Heap and Alloc profiling. By default this is set to the runtime.MemProfileRate value which is 512 * 1024. This can be set to a higher value to increase the resolution of the memory profile.

func WithMutexFraction

func WithMutexFraction(rate int) ProfileOption

WithMutexFraction sets the rate at which the mutex profiler samples mutex contention. By default this is set to 1.

func WithProfileFileLocation

func WithProfileFileLocation(path string) ProfileOption

WithProfileFileLocation allows a custom output path for the profile file that is written to disk.

func WithQuietOutput

func WithQuietOutput() ProfileOption

WithQuietOutput prevents the profiling from writing logger events.

func WithRealTimeData

func WithRealTimeData() ProfileOption

WithLiveTracing enables live tracing of the program as it runs for cases which allow it. This exposes trace data via the runtime/pprof http server.

func WithThreadProfiler

func WithThreadProfiler() ProfileOption

TODO: Doc

func WithTracing

func WithTracing() ProfileOption

WithTracing enables the tracing profiler. Tracing is useful for determining the flow of a program and where it is spending time. Utilising the trace api within your code can add some extra context to the trace output (logs, tasks etc). but is not the responsibility of this package.

func WithoutSignalHandling

func WithoutSignalHandling() ProfileOption

WithoutSignalHandling disables the signal handling for the profiler. This is useful for cases where you want to handle the signal yourself. Be sure to invoke profiler.Stop() yourself in your code and handle the os.Exit() yourself etc.

type Profiler

type Profiler struct {
	// contains filtered or unexported fields
}

Profiler encapsulates a profiling instance.

func New

func New(options ...ProfileOption) *Profiler

New returns a new instance of the Profiler.

func Start

func Start(options ...ProfileOption) *Profiler

Start starts a new profiling instance. If no mode option is provided, the default behavious is to perform CPU profiling. Start returns the underlying profile instance typically deferred in simple scenarios. In more complex scenarios keeping a handle to the stop function and calling it yourself in some of your own signal handling code for example is wise, this should be used with the option: WithNoSignalShutdownHandling.

func (*Profiler) SetProfileFile

func (p *Profiler) SetProfileFile(name string)

SetProfileFile sets the profile file for the profiler instance. not to be confused with the folder location provided by the functional options.

func (*Profiler) Stop

func (p *Profiler) Stop()

Stop stops the profiling instance. If no profiling instance is active, this function will cause an exit.

type StrategyFunc

type StrategyFunc func(p *Profiler) (FinalizerFunc, error)

StrategyFunc is the custom type for an implementation that controls pre/post profiling setup and teardown.