fdooze

README

fdooze

fdooze complements Gomega with tests to detect leaked ("oozed") file descriptors.

Note: fdooze is available on Linux only, as discovering file descriptor information requires using highly system-specific APIs and the descriptor information varies across different systems (if available at all).

For devcontainer instructions, please see the section "DevContainer" below.

Usage

In your tests, using Ginkgo:

import . "github.com/thediveo/fdooze"

var _ = Describe("...", func() {

    BeforeEach(func() {
        goodfds := Filedescriptors()
        DeferCleanup(func() {
            Expect(Filedescriptors()).NotTo(HaveLeakedFds(goodfds))
        })
    })

})

This takes a snapshot of "good" file descriptors before each test and then after each test it checks to see if there are any leftover file descriptors that weren't already in use before a test. fdooze does not blindly just compare fd numbers, but takes as much additional detail information as possible into account: like file paths, socket domains, types, protocols and addresses, et cetera.

On finding leaked file descriptors, fdooze dumps these leaked fds in the failure message of the HaveLeakedFds matcher. For instance:

Expected not to leak 1 file descriptors:
    fd 7, flags 0xa0000 (O_RDONLY,O_CLOEXEC)
        path: "/home/leaky/module/oozing_test.go"

For other types of file descriptors, such as pipes and sockets, several details will differ: instead of a path, other parameters will be shown, like pipe inode numbers or socket addresses. Due to the limitations of the existing fd discovery API, it is not possible to see where the file descriptor was opened (which might be deep inside some 3rd party package anyway).

Expect or Eventually?

In case you are already familiar with Gomega's gleak goroutine leak detection package, then please note that typical fdooze usage doesn't require Eventually, so Expect is fine most of the time. However, in situations where goroutines open file descriptors it might be a good idea to first wait for goroutines to terminate and not leak and only then test for any file descriptor leaks.

When using Eventually() make sure to pass the Filedescriptors function itself to it, not the result of calling Filedescriptors.

// Correct
Eventually(Filedescriptors).ShouldNot(HaveLeakedFds(...))

WRONG: Eventually(Filedescriptors()).ShouldNot(HaveLeakedFds(...))

Leak Tests on Launched Processes

The session package implements retrieving the open file descriptors from a Gomega gexec.Session (Linux only). This allows checking processes launched by a test suite for file descriptor leaks, subject to normal process access control.

It is recommended to dot-import the session package, as this keeps the "session" identifier free to be used by test writers as they see fit.

session, err := gexec.Start(cmd, GinkgoWriter, GinkgoWriter)
Expect(err).NotTo(HaveOccurred())

// Optional, please see note below.
client.DoWarmupAPIThing()

// When using Eventually, make sure to pass the function, not its result!
sessionFds := func ([]FileDescriptor, error) {
    return FiledescriptorsFor(session)
}

goodfds := sessionFds()

client.DoSomeAPIThing()
Expect(session).Should(gbytes.Say("I did the thing"))

Eventually(sessionFds).ShouldNot(HaveLeakedFds(goodfds))
Eventually(session.Interrupt()).Should(gexec.Exit(0))

In case the launched process is implemented in Go, fd leak tests need to be carefully designed as to not fail with false positive fd leaks caused by Go's netpoll runtime (for instance, see The Go netpoller for more background information).

For instance, when opening a file or network socket for the first time, Go's runtime creates an internal epoll fd as well as a non-blocking pipe fd for use in its internal asynchronous I/O handling.

Unfortunately, it is not possible to easily filter out the file descriptors belonging to the Go runtime netpoller: fds in general don't record who created them and for what purpose. An epoll fd might be used in an application itself and thus quite often be ambigous. Also, the exact fd number will depend on a Go application highly specific initialization process.

It is thus mandatory to take a "reference" snapshot of baseline fds only after the launched process has opened its first file or network socket. In case of network-facing services this will be when the listening transport port has become available.

DevContainer

[!CAUTION]

Do not use VSCode's "Dev Containers: Clone Repository in Container Volume" command, as it is utterly broken by design, ignoring .devcontainer/devcontainer.json.

1. git clone https://github.com/thediveo/enumflag
2. in VSCode: Ctrl+Shift+P, "Dev Containers: Open Workspace in Container..."
3. select enumflag.code-workspace and off you go...

Go Version Support

fdooze supports versions of Go that are noted by the Go release policy, that is, major versions N and N-1 (where N is the current major version).

Goigi the Gopher

Goigi the gopher mascot clearly has been inspired by the Go gopher art work of Renee French. It seems as if Goigi has some issues with plumbing file descriptors properly.

Copyright and License

fdooze is Copyright 2022, 2025 Harald Albrecht, and licensed under the Apache License, Version 2.0.

Documentation

Overview

Package fdooze complements the Gingko/Gomega testing and matchers framework with matchers for file descriptor leakage detection.

Please note that due to technical restrictions this (experimental) package is available only for Linux.

Basic Usage

In your project (with a go.mod) run

go get github.com/thediveo/fdooze

to get and install the latest stable release.

A typical usage in your tests and using Ginkgo then is, after dot-importing the fdooze package:

import . "github.com/thediveo/fdooze"

var _ = Describe("...", func() {
    BeforeEach(func() {
        goodfds := Filedescriptors()
        DeferCleanup(func() {
            Expect(Filedescriptors()).NotTo(HaveLeakedFds(goodfds))
        })
    })
})

This takes a snapshot of “good” file descriptors before each test and then after each test it checks to see if there are any leftover file descriptors that weren't already in use before a test. The fdooze package does not blindly just compare fd numbers, but takes as much additional detail information as possible into account: like file paths, socket domains, types, protocols and addresses, et cetera.

On finding leaked file descriptors, fdooze dumps these leaked fds in the failure message of the HaveLeakedFds matcher. For instance:

Expected not to leak 1 file descriptors:
    fd 7, flags 0xa0000 (O_RDONLY,O_CLOEXEC)
        path: "/home/leaky/module/oozing_test.go"

For other types of file descriptors, such as pipes and sockets, several details will differ: instead of a path, other parameters will be shown, like pipe inode numbers or socket addresses. Due to the limitations of the existing fd discovery API, it is not possible to see where the file descriptor was opened (which might be deep inside some 3rd party package anyway).

Expect or Eventually

In case you are already familiar with Gomega's github.com/onsi/gomega/gleak goroutine leak detection package, then please note that typical fdooze usage doesn't require Eventually, so Expect is fine most of the time. However, in situations where goroutines open file descriptors it might be a good idea to first wait for goroutines to terminate and not leak and only then test for any file descriptor leaks.

When using Eventually make sure to pass the Filedescriptors function itself, but not the result of calling Filedescriptors.

// ✔✔✔ Correct ✔✔✔
Eventually(Filedescriptors).ShouldNot(HaveLeakedFds(...))

// ❌❌❌ WRONG WRONG WRONG ❌❌❌
Eventually(Filedescriptors()).ShouldNot(HaveLeakedFds(...))

Index

• func HaveLeakedFds(fds []FileDescriptor, ignoring ...types.GomegaMatcher) types.GomegaMatcher
• func IgnoringFiledescriptors(fds []FileDescriptor) types.GomegaMatcher
• type FileDescriptor
  • func Filedescriptors() []FileDescriptor

Functions

func HaveLeakedFds

func HaveLeakedFds(fds []FileDescriptor, ignoring ...types.GomegaMatcher) types.GomegaMatcher

HaveLeakedFds succeeds if after filtering out expected file descriptors from the list of actual file descriptors the remaining list is non-empty. The file descriptors not filtered out are considered to have been leaked.

Optional additional filter matchers can be specified that can filter out use case-specific file descriptors based on various fd properties. Please refer to the github.com/thediveo/fdooze/filedesc package for details about the defined github.com/thediveo/fdooze/filedesc.FileDescriptor implementations for various types of file descriptors.

File descriptors are identified not only based on the fd number, but also additional associated information, such as a file path they link to, type, socket inode number, et cetera. As fd numbers tend to quickly get reused this allows detecting changed fds in many (if not most) situations with enough accuracy.

HaveLeakedFds does not assume any well-known fds, and in particular, it does not make any assumptions about fds with numbers 0, 1, 2.

A typical way to check for leaked (“oozed”) file descriptors is as follows, after dot-importing the fdooze package:

 import . "github.com/thediveo/fdooze"

 var _ = Describe("...", {
	  BeforeEach(func() {
	      goodfds := Filedescriptors()
	      DeferCleanup(func() {
	          Expect(Filedescriptors()).NotTo(HaveLeakedFds(goodfds))
	      })
	  })
 })

HaveLeakedFds accepts optional Gomega matchers (of type types.GomegaMatcher) that it will repeatedly pass FileDescriptor values to: if a matcher succeeds, the particular file descriptor is considered not to be leaked and thus filtered out. Especially Gomega's HaveField matcher can be quite useful in covering specific use cases where the otherwise straightforward before-after fd comparism isn't enough.

func IgnoringFiledescriptors

func IgnoringFiledescriptors(fds []FileDescriptor) types.GomegaMatcher

IgnoringFiledescriptors succeeds if an actual FileDescriptor in contained in a slice of expected file descriptors. An actual FileDescriptor is considered to be contained, if the slice must contains a FileDescriptor with the same fd number and filedesc.FileDescriptor.Equal considers both file descriptors to be equal.

Please note that fd flags and file offsets are ignored when testing for equality, in order to avoid spurious false positives.

Types

type FileDescriptor

type FileDescriptor = filedesc.FileDescriptor

FileDescriptor describes a Linux “fd” file descriptor in more detail than just its fd int number; it is a type alias of filedesc.FileDescriptor.

func Filedescriptors

func Filedescriptors() []FileDescriptor

Filedescriptors returns the list of currently open file descriptors for this process.