deadlock

README

Preface

Based on https://github.com/sasha-s/go-deadlock.

Changes from that package:

• Uses build tags to eliminate all overhead when not enabled
• Tests now pass race checker and improves code coverage
• Improved performance when using go 1.18+
• Uses significantly less memory
• Diagnostic output matches -race style and uses runtime.CallersFrames to get correct line numbers
• Adds deadlock.Enabled and deadlock.Debug constants
• Adds Try(R)Lock() when using go 1.18+
• Drops the dummy implementations for types other than Mutex and RWMutex

Installation

go get github.com/linkdata/deadlock

Usage

The package enables itself when either the deadlock or race build tag is set, and the nodeadlock build tag is not set. The easiest way is to simply use deadlock.(RW)Mutex and run or test your code with the race detector.

import "github.com/linkdata/deadlock"

var mu deadlock.Mutex
mu.Lock()
defer mu.Unlock()

var rw deadlock.RWMutex
rw.RLock()
defer rw.RUnlock()

go run -race .

Deadlocks

Taking the same lock twice in the same goroutine will deadlock:

A.RLock() // or A.Lock()
...
A.Lock() // or A.RLock()

Those cases will be reported immediately when they occur. Also, in case we wait for a lock for more than deadlock.Opts.DeadlockTimeout (30 seconds by default), we also report that as a potential deadlock. Setting the DeadlockTimeout to zero disables this detection.

Sample output

POTENTIAL DEADLOCK:
goroutine 624 have been trying to lock 0xc0009a20d8 for more than 20ms:
  github.com/linkdata/deadlock.(*DeadlockMutex).Lock()
      /home/user/src/deadlock/deadlock.go:26 +0x113
  github.com/linkdata/deadlock.TestHardDeadlock.func2()
      /home/user/src/deadlock/deadlock_test.go:154 +0x92

goroutine 622 previously locked it from:
  github.com/linkdata/deadlock.(*DeadlockMutex).Lock()
      /home/user/src/deadlock/deadlock.go:26 +0x164
  github.com/linkdata/deadlock.TestHardDeadlock()
      /home/user/src/deadlock/deadlock_test.go:150 +0xe6

goroutine 622 current stack:
goroutine 622 [sleep]:
time.Sleep(0xf4240)
        /usr/local/go/src/runtime/time.go:195 +0x135
github.com/linkdata/deadlock.spinWait(0xc000988340, 0x0?, 0x1)
        /home/user/src/deadlock/deadlock_test.go:25 +0x3e
github.com/linkdata/deadlock.TestHardDeadlock(0xc000988340)
        /home/user/src/deadlock/deadlock_test.go:157 +0x265
testing.tRunner(0xc000988340, 0x6187e8)
        /usr/local/go/src/testing/testing.go:1576 +0x217
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1629 +0x806

Inconsistent lock ordering

One of the most common sources of deadlocks is inconsistent lock ordering. If you have two mutexes A and B, and in one goroutine you have:

A.Lock() // defer A.Unlock() or similar.
...
B.Lock() // defer B.Unlock() or similar.

And in another goroutine the order of locks is reversed:

B.Lock() // defer B.Unlock() or similar.
...
A.Lock() // defer A.Unlock() or similar.

This does not guarantee a deadlock (maybe the goroutines above can never be running at the same time), but it is bad practice. Detection is enabled by default, but can be disabled by setting deadlock.Opts.MaxMapSize to zero.

Sample output

POTENTIAL DEADLOCK: Inconsistent locking:
in one goroutine: happened before
  github.com/linkdata/deadlock.(*DeadlockRWMutex).Lock()
      /home/user/src/deadlock/deadlock.go:55 +0xa8
  github.com/linkdata/deadlock.TestLockOrder.func2()
      /home/user/src/deadlock/deadlock_test.go:120 +0x34

happened after
  github.com/linkdata/deadlock.(*DeadlockMutex).Lock()
      /home/user/src/deadlock/deadlock.go:26 +0x11a
  github.com/linkdata/deadlock.TestLockOrder.func2()
      /home/user/src/deadlock/deadlock_test.go:121 +0xa9

in another goroutine: happened before
  github.com/linkdata/deadlock.(*DeadlockMutex).Lock()
      /home/user/src/deadlock/deadlock.go:26 +0xa5
  github.com/linkdata/deadlock.TestLockOrder.func3()
      /home/user/src/deadlock/deadlock_test.go:129 +0x34

happened after
  github.com/linkdata/deadlock.(*DeadlockRWMutex).RLock()
      /home/user/src/deadlock/deadlock.go:74 +0x11a
  github.com/linkdata/deadlock.TestLockOrder.func3()
      /home/user/src/deadlock/deadlock_test.go:130 +0xa6

Debugging constants

It's often helpful to run extra runtime checks during development and testing, but you don't want to have that code around in a production environment. Since these are constants, if the constant is false, code that depends on it being true gets removed entirely.

We define two:

• deadlock.Debug is true if either race or debug are set.
• deadlock.Enabled is true if either race or deadlock are set and nodeadlock is not set.

if deadlock.Debug {
    // extra checks or logging go here
}

Configuring

Options are stored in the global variable deadlock.Opts. See Options.

• Opts.DeadlockTimeout: blocking on mutex for longer than DeadlockTimeout is considered a deadlock, ignored if zero
• Opts.OnPotentialDeadlock: callback for when a deadlock is detected, or panic if nil
• Opts.MaxMapSize: size of happens before // happens after table, disables inconsistent locking order detection if zero
• Opts.PrintAllCurrentGoroutines: if true, dump stacktraces of all goroutines when inconsistent locking is detected
• Opts.LogBuf: where to write deadlock info/stacktraces, default is os.Stderr

Documentation

Index

• Constants
• Variables
• type DeadlockMutex
  • func (m *DeadlockMutex) Lock()
  • func (m *DeadlockMutex) TryLock() bool
  • func (m *DeadlockMutex) Unlock()
• type DeadlockRWMutex
  • func (m *DeadlockRWMutex) Lock()
  • func (m *DeadlockRWMutex) RLock()
  • func (m *DeadlockRWMutex) RLocker() sync.Locker
  • func (m *DeadlockRWMutex) RUnlock()
  • func (m *DeadlockRWMutex) TryLock() bool
  • func (m *DeadlockRWMutex) TryRLock() bool
  • func (m *DeadlockRWMutex) Unlock()
• type Mutex
• type Options
  • func (opts *Options) Flush() error
  • func (opts *Options) PotentialDeadlock()
  • func (opts *Options) ReadLocked(fn func())
  • func (opts *Options) Write(b []byte) (int, error)
  • func (opts *Options) WriteLocked(fn func())
• type RWMutex

Constants

const Debug = false

const Enabled = false

Enabled is true if deadlock checking is enabled

Variables

var Opts = Options{
	DeadlockTimeout: time.Second * 30,
	MaxMapSize:      1024 * 64,
	LogBuf:          os.Stderr,
}

Opts control how deadlock detection behaves. To safely read or change options during runtime, use Opts.ReadLocked() and Opts.WriteLocked()

Types

type DeadlockMutex

type DeadlockMutex struct {
	// contains filtered or unexported fields
}

A DeadlockMutex is a drop-in replacement for sync.Mutex.

func (*DeadlockMutex) Lock

func (m *DeadlockMutex) Lock()

Lock locks the mutex. If the lock is already in use, the calling goroutine blocks until the mutex is available.

Logs potential deadlocks to Opts.LogBuf, calling Opts.OnPotentialDeadlock on each occasion.

func (*DeadlockMutex) TryLock

func (m *DeadlockMutex) TryLock() bool

func (*DeadlockMutex) Unlock

func (m *DeadlockMutex) Unlock()

Unlock unlocks the mutex. It is a run-time error if m is not locked on entry to Unlock.

A locked Mutex is not associated with a particular goroutine. It is allowed for one goroutine to lock a Mutex and then arrange for another goroutine to unlock it.

type DeadlockRWMutex

type DeadlockRWMutex struct {
	// contains filtered or unexported fields
}

An DeadlockRWMutex is a drop-in replacement for sync.RWMutex.

func (*DeadlockRWMutex) Lock

func (m *DeadlockRWMutex) Lock()

Lock locks rw for writing. If the lock is already locked for reading or writing, Lock blocks until the lock is available. To ensure that the lock eventually becomes available, a blocked Lock call excludes new readers from acquiring the lock.

Logs potential deadlocks to Opts.LogBuf, calling Opts.OnPotentialDeadlock on each occasion.

func (*DeadlockRWMutex) RLock

func (m *DeadlockRWMutex) RLock()

RLock locks the mutex for reading.

Logs potential deadlocks to Opts.LogBuf, calling Opts.OnPotentialDeadlock on each occasion.

func (*DeadlockRWMutex) RLocker

func (m *DeadlockRWMutex) RLocker() sync.Locker

RLocker returns a Locker interface that implements the Lock and Unlock methods by calling RLock and RUnlock.

func (*DeadlockRWMutex) RUnlock

func (m *DeadlockRWMutex) RUnlock()

RUnlock undoes a single RLock call; it does not affect other simultaneous readers. It is a run-time error if rw is not locked for reading on entry to RUnlock.

func (*DeadlockRWMutex) TryLock

func (m *DeadlockRWMutex) TryLock() bool

func (*DeadlockRWMutex) TryRLock

func (m *DeadlockRWMutex) TryRLock() bool

func (*DeadlockRWMutex) Unlock

func (m *DeadlockRWMutex) Unlock()

Unlock unlocks the mutex for writing. It is a run-time error if rw is not locked for writing on entry to Unlock.

As with Mutexes, a locked RWMutex is not associated with a particular goroutine. One goroutine may RLock (Lock) an RWMutex and then arrange for another goroutine to RUnlock (Unlock) it.

type Mutex

type Mutex struct{ sync.Mutex }

Mutex is sync.Mutex wrapper

type Options

type Options struct {
	// Waiting for a lock for longer than a non-zero DeadlockTimeout is considered a deadlock.
	// Set to 30 seconds by default.
	DeadlockTimeout time.Duration
	// OnPotentialDeadlock is called each time a potential deadlock is detected -- either based on
	// lock order or on lock wait time. If nil, panics instead.
	OnPotentialDeadlock func()
	// Sets the maximum size of the map that tracks lock ordering.
	// Setting this to zero disables tracking of lock order. Default is a reasonable size.
	MaxMapSize int
	// Will dump stacktraces of all goroutines when inconsistent locking is detected.
	PrintAllCurrentGoroutines bool
	// Where to write reports, set to os.Stderr by default.
	LogBuf io.Writer
}

func (*Options) Flush

func (opts *Options) Flush() error

Flush will flush the LogBuf if it is a *bufio.Writer

func (*Options) PotentialDeadlock

func (opts *Options) PotentialDeadlock()

PotentialDeadlock calls OnPotentialDeadlock if it is set, or panics if not.

func (*Options) ReadLocked

func (opts *Options) ReadLocked(fn func())

ReadLocked calls the given function with Opts locked for reading.

func (*Options) Write

func (opts *Options) Write(b []byte) (int, error)

Write implements io.Writer for Options.

func (*Options) WriteLocked

func (opts *Options) WriteLocked(fn func())

WriteLocked calls the given function with Opts locked for writing.

type RWMutex

type RWMutex struct{ sync.RWMutex }

RWMutex is sync.RWMutex wrapper