err2

README

err2

The package provides functions for automatic error propagation.

go get github.com/lainio/err2

• Structure
• Automatic Error Propagation And Stack Tracing
• Error handling
  • Automatic And Optimized Stack Tracing
  • Manual Stack Tracing
  • Error Handler
• Error checks
  • Filters for non-errors like io.EOF
• Backwards Compatibility Promise for the API
• Assertion (design by contract)
  • Assertion Package for Unit Testing
• Background
• Learnings by so far
• Support
• Roadmap

Structure

err2 has the following package structure:

• The err2 (main) package includes declarative error handling functions.
• The try package offers error checking functions.
• The assert package implements assertion helpers for both unit-testing and design-by-contract.

Automatic Error Propagation And Stack Tracing

The current version of Go tends to produce too much error checking and too little error handling. This package helps us fix that.

1. It helps to declare error handlers with defer.
2. It helps to check and transport errors to the nearest (the defer-stack) error handler.
3. It helps us use design-by-contract type preconditions.
4. It offers automatic stack tracing for every error, runtime error, or panic.

You can use all of them or just the other. However, if you use try for error checks you must remember use Go's recover() by yourself, or your error isn't transformed to an error return value at any point.

Error handling

Package err2 relies on Go's declarative programming structure defer. The err2 helps to set deferred functions (error handlers) which are only called if err != nil.

In every function which uses err2 for error-checking should have at least one error handler. If there are no error handlers and error occurs the current function panics. However, if any function above in the call stack has err2 error handler it will catch the error.

This is the simplest form of err2 error handler

defer err2.Return(&err)

which is the helper handler for cases that don't need to annotate the error. If you need to annotate the error you can use Returnf. There is a wrapping version Returnw even you will not need it.

Our general guideline is:

Do not wrap an error when doing so would expose implementation details.

Automatic And Optimized Stack Tracing

err2 offers optional stack tracing. It's automatic and optimized. Optimized means that call stack is processes before output. That means that stack trace starts from where the actual error/panic is occurred and not from where the error is caught. You don't need to search your self the actual line where the pointer was nil or error was received. That line is in the first one you are seeing.

---
runtime error: index out of range [0] with length 0
---
goroutine 1 [running]:
main.test2({0x0, 0x0, 0x40XXXXXf00?}, 0x2?)
	/home/.../go/src/github.com/lainio/ic/main.go:43 +0x14c
main.main()
	/home/.../go/src/github.com/lainio/ic/main.go:77 +0x248

Without optimization call stack would have at least two more call stack entries:

goroutine 1 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x68
panic({0x12e3e0, 0x188f50})
	/usr/local/go/src/runtime/panic.go:838 +0x20c
main.test2({0x0, 0x0, 0x40XXXXXf00?}, 0x2?)
	/home/.../go/src/github.com/lainio/ic/main.go:43 +0x14c
main.main()
	/home/.../go/src/github.com/lainio/ic/main.go:77 +0x248

Just set the err2.SetErrorTracer or err2.SetPanicTracer to the stream you want traces to be written:

err2.SetErrorTracer(os.Stderr) // write error stack trace to stderr
  or, for example:
err2.SetPanicTracer(log.Writer()) // stack panic trace to std logger

If no Tracer is set no stack tracing is done. This is the default because in the most cases proper error messages are enough and panics are handled immediately anyhow.

Manual Stack Tracing

The err2 offers two error catchers for manual stack tracing: CatchTrace and CatchAll. The first one lets you handle errors and it will print the stack trace to stderr for panic and runtime.Error. The second is the same but you have a separate handler function for panic and runtime.Error so you can decide by yourself where to print them or what to do with them.

Error Handler

The err2.Handle is a helper function to add actual error handlers which are called only if an error has occurred. In most real-world cases, we have multiple error checks and only one or just a few error handlers. However, you can have as many error handlers per function as you need.

Read the package documentation for more information.

Error checks

The try package provides convenient helpers to check the errors. Since the Go 1.18 we have been using generics to have fast and convenient error checking.

For example, instead of

b, err := ioutil.ReadAll(r)
if err != nil {
        return err
}
...

we can call

b := try.To1(ioutil.ReadAll(r))
...

but not without an error handler (Returnf, Handle) or it just panics your app if you don't have a recovery call in the current call stack. However, you can put your error handlers where ever you want in your call stack. That can be handy in the internal packages and certain types of algorithms.

We think that panicking for the errors at the start of the development is far better than not checking errors at all.

Filters for non-errors like io.EOF

When error values are used to transport some other information instead of actual errors we have functions like try.Is and even try.IsEOF for convenience.

With these you can write code where error is translated to boolean value:

notExist := try.Is(r2.err, plugin.ErrNotExist)

// real errors are cought and the returned boolean tells if value
// dosen't exist returnend as `plugin.ErrNotExist`

For more information see the examples of both functions.

Backwards Compatibility Promise for the API

The err2 package's API will be backwards compatible. Before the version 1.0.0 is released the API changes time to time, but we promise to offer automatic conversion scripts for your repos to update them for the latest API. We also mark functions deprecated before they become obsolete. Usually one released version before. We have tested this in our systems with large code base and it works wonderfully.

More information can be found from scripts' readme file.

Assertion (design by contract)

The assert package is meant to be used for design-by-contract-type of development where you set preconditions for your functions. It's not meant to replace normal error checking but speed up incremental hacking cycle. That's the reason why default mode (var D Asserter) is to panic. By panicking developer get immediate and proper feedback which allows cleanup the code and APIs before actual production release.

func marshalAttestedCredentialData(json []byte, data *protocol.AuthenticatorData) []byte {
	assert.SLen(data.AttData.AAGUID, 16, "wrong AAGUID length")
	assert.NotEmpty(data.AttData.CredentialID, "empty credential id")
	assert.SNotEmpty(data.AttData.CredentialPublicKey, "empty credential public key")
	...

Previous code block shows the use of the default asserter for developing.

assert.DefaultAsserter = AsserterDebug

If any of the assertion fails, code panics. These type of assertions can be used without help of the err2 package if wanted.

During the software development lifecycle, it isn't crystal clear what preconditions are for a programmer and what should be translated to end-user errors as well. The assert package uses a concept called Asserter to have different types of asserter for different phases of a software project.

The following code block is a sample where the production time asserter is used to generate proper error messages.

func (ac *Cmd) Validate() (err error) {
	defer err2.Return(&err)

	assert.P.NotEmpty(ac.SubCmd, "sub command needed")
	assert.P.Truef(ac.SubCmd == "register" || ac.SubCmd == "login",
		"wrong sub command: %s: want: register|login", ac.SubCmd)
	assert.P.NotEmpty(ac.UserName, "user name needed")
	assert.P.NotEmpty(ac.Url, "connection URL cannot be empty")
	assert.P.NotEmpty(ac.AAGUID, "authenticator ID needed")
	assert.P.NotEmpty(ac.Key, "master key needed")

	return nil
}

When assert statements are used to generate end-user error messages instead of immediate panics, err2 handlers are needed to translate asserts to errors in a convenient way. That's why we decided to build assert as a sub package of err2 even though there are no actual dependencies between them. See the assert package's documentation and examples for more information.

Assertion Package for Unit Testing

Same asserts can be used during the unit tests:

func TestWebOfTrustInfo(t *testing.T) {
	assert.PushTester(t)
	defer assert.PopTester()

	common := dave.CommonChains(eve.Node)
	assert.SLen(common, 2)

	wot := dave.WebOfTrustInfo(eve.Node)
	assert.Equal(0, wot.CommonInvider)
	assert.Equal(1, wot.Hops)

	wot = NewWebOfTrust(bob.Node, carol.Node)
	assert.Equal(-1, wot.CommonInvider)
	assert.Equal(-1, wot.Hops)
	...

Especially powerful feature is that even if some assertion violation happens during the execution of called functions like above NewWebOfTrust() function instead of the actual Test function, it's reported as normal test failure. That means that we don't need to open our internal preconditions just for testing.

Background

err2 implements similar error handling mechanism as drafted in the original check/handle proposal. The package does it by using internally panic/recovery, which some might think isn't perfect. We have run many benchmarks to try to minimise the performance penalty this kind of mechanism might bring. We have focused on the happy path analyses. If the performance of the error path is essential, don't use this mechanism presented here. But be aware that if your code uses the error path as a part of algorithm itself something is wrong.

For happy paths by using try.ToX error check functions there are no performance penalty at all. However, the mandatory use of the defer might prevent some code optimisations like function inlining. If you have a performance-critical use case, we recommend you to write performance tests to measure the effect. As a general guideline for maximum performance we recommend to put error handlers as high in the call stack as possible, and use only error checking (try.To() calls) in the inner loops. And yes, that leads to non-local control structures, but it's the most performant solution of all.

The original goal was to make it possible to write similar code that the proposed Go2 error handling would allow and do it right now (summer 2019). The goal was well aligned with the Go2 proposal, where it would bring a try macro and let the error handling be implemented in defer blocks. The try-proposal was canceled at its latest form. Nevertheless, we have learned that using panics for early-stage error transport isn't bad but the opposite. It seems to help:

• to draft algorithms much faster,
• still maintains the readability,
• and most importantly, it keeps your code more refactorable because you don't have to repeat yourself.

Learnings by so far

We have used the err2 and assert packages in several projects. The results have been so far very encouraging:

• If you forget to use handler, but you use checks from the package, you will get panics (and optionally stack traces) if an error occurs. That is much better than getting unrelated panic somewhere else in the code later. There have also been cases when code reports error correctly because the 'upper' handler catches it.

• Because the use of err2.Returnf is so relatively easy, error messages much better and informative.

• When error handling is based on the actual error handlers, code changes have been much easier.

• You don't seem to need '%w' wrapping. See the Go's official blog post what are cons for that.

  Do not wrap an error when doing so would expose implementation details.

Support

The package has been in experimental mode quite long time. Since the Go generics we are transiting towards more official mode. Currently we offer support by GitHub Discussions. Naturally, any issues are welcome as well!

Roadmap

Version history:

• 0.1, first draft (Summer 2019)
• 0.2, code generation for type helpers
• 0.3, Returnf added, not use own transport type anymore but just error
• 0.4, Documentation update
• 0.5, Go modules are in use
• 0.6.1, assert package added, and new type helpers
• 0.7.0 filter functions for non-errors like io.EOF
• 0.8.0 try.To() & assert.That(), etc. functions with the help of the generics
• 0.8.1 bug-fix: runtime.Error types are treated as panics now (Issue #1)
• 0.8.3 try.IsXX() bug fix, lots of new docs, and automatic stack tracing!
• 0.8.4 Optimized Stack Tracing, documentation, benchmarks, etc.
• 0.8.5 Typo in StackTraceWriter fixed
• 0.8.6 Stack Tracing bug fixed, URL helper restored until migration tool
• 0.8.7 Auto-migration tool to convert deprecated API usage for your repos, err2.Throwf added
• 0.8.8 Assertion package integrates with Go's testing system. Type variables removed.
• 0.8.9 Bug fixes, deprecations, new Tracer API, preparing err2 for 1.0
• 0.8.10 New assertion functions and helpers for tests
• 0.8.11 Remove deprecations, new global err values and try.IsXX functions, more documentation.

Documentation

Overview

Package err2 provides three main functionality:

1. err2 package includes helper functions for error handling & automatic stack tracing
2. try package is for error checking
3. assert package is for design-by-contract and preconditions both for normal runtime and for testing

The traditional error handling idiom in Go is roughly akin to

if err != nil { return err }

which applied recursively.

The err2 package drives programmers to focus more on error handling rather than checking errors. We think that checks should be so easy that we never forget them. The CopyFile example shows how it works:

// CopyFile copies source file to the given destination. If any error occurs it
// returns error value describing the reason.
func CopyFile(src, dst string) (err error) {
     // Add first error handler just to annotate the error properly.
     defer err2.Returnf(&err, "copy %s %s", src, dst)

     // Try to open the file. If error occurs now, err will be annotated and
     // returned properly thanks to above err2.Returnf.
     r := try.To1(os.Open(src))
     defer r.Close()

     // Try to create a file. If error occurs now, err will be annotated and
     // returned properly.
     w := try.To1(os.Create(dst))
     // Add error handler to clean up the destination file. Place it here that
     // the next deferred close is called before our Remove call.
     defer err2.Handle(&err, func() {
     	os.Remove(dst)
     })
     defer w.Close()

     // Try to copy the file. If error occurs now, all previous error handlers
     // will be called in the reversed order. And final return error is
     // properly annotated in all the cases.
     try.To1(io.Copy(w, r))

     // All OK, just return nil.
     return nil
}

Error checks and Automatic Error Propagation

The try package provides convenient helpers to check the errors. For example, instead of

b, err := ioutil.ReadAll(r)
if err != nil {
   return err
}

we can write

b := try.To1(ioutil.ReadAll(r))

Note that try.ToX functions are as fast as if err != nil statements. Please see the try package documentation for more information about the error checks.

Automatic Stack Tracing

err2 offers optional stack tracing. And yes, it's fully automatic. Just set the tracers to the stream you want traces to be written:

err2.SetErrorTracer(os.Stderr) // write error stack trace to stderr
 or
err2.SetPanicTracer(log.Writer()) // panic stack trace to std logger

Error handling

Package err2 relies on declarative control structures to achieve error and panic safety. In every function which uses err2 or try package for error-checking has to have at least one declarative error handler if it returns error value. If there are no error handlers and error occurs it panics. We think that panicking for the errors is much better than not checking errors at all. Nevertheless, if the call stack includes any err2 error handlers like err2.Handle the error is handled where the handler is saved to defer-stack. (defer is not lexically scoped)

err2 includes many examples to play with like previous CopyFile. Please see them for more information.

Example

package main

import (
	"fmt"
	"io"
	"os"

	"github.com/lainio/err2"
	"github.com/lainio/err2/try"
)

func CopyFile(src, dst string) (err error) {
	defer err2.Returnf(&err, "copy file %s->%s", src, dst)

	// These try.To() checkers are as fast as `if err != nil {}`

	r := try.To1(os.Open(src))
	defer r.Close()

	w := try.To1(os.Create(dst))
	defer err2.Handle(&err, func() {
		os.Remove(dst)
	})
	defer w.Close()
	try.To1(io.Copy(w, r))
	return nil
}

func main() {
	// To see how automatic stack tracing works please run this example with:
	//   go test -v -run='^Example$'
	err2.SetErrorTracer(os.Stderr)

	err := CopyFile("/notfound/path/file.go", "/notfound/path/file.bak")
	if err != nil {
		fmt.Println(err)
	}
}

Output:

copy file /notfound/path/file.go->/notfound/path/file.bak: open /notfound/path/file.go: no such file or directory

Index

• Variables
• func Catch(f func(err error))
• func CatchAll(errorHandler func(err error), panicHandler func(v any))
• func CatchTrace(errorHandler func(err error))
• func ErrorTracer() io.Writer
• func Handle(err *error, handlerFn func())
• func PanicTracer() io.Writer
• func Return(err *error)
• func Returnf(err *error, format string, args ...any)
• func Returnw(err *error, format string, args ...any)
• func SetErrorTracer(w io.Writer)
• func SetPanicTracer(w io.Writer)
• func SetTracers(w io.Writer)
• func Throwf(format string, args ...any)

Examples

• Package
• Handle
• Return
• Returnf
• Returnf (DeferStack)
• Returnf (Empty)
• Throwf

Variables

var (
	// NotFound is similar *no-error* like io.EOF for those who really want to
	// use error return values to transport non errors. It's far better to have
	// discriminated unions as errors for function calls. But if you insist the
	// related helpers are in they try package: try.IsNotFound(), ... These
	// 'global' errors and their helper functions in try package are for
	// experimenting now.
	NotFound  = errors.New("not found")
	NotExist  = errors.New("not exist")
	Exist     = errors.New("already exist")
	NotAccess = errors.New("permission denied")
)

Functions

func Catch

func Catch(f func(err error))

Catch is a convenient helper to those functions that doesn't return errors. There can be only one deferred Catch function per non error returning function like main(). It doesn't catch panics and runtime errors. If that's important use CatchAll or CatchTrace instead. See Handle for more information.

func CatchAll

func CatchAll(errorHandler func(err error), panicHandler func(v any))

CatchAll is a helper function to catch and write handlers for all errors and all panics thrown in the current go routine. It and CatchTrace are preferred helpers for go workers on long running servers, because they stop panics as well.

Note, if any Tracer is set stack traces are printed automatically. If you want to do it in the handlers by yourself, auto tracers should be nil.

func CatchTrace

func CatchTrace(errorHandler func(err error))

CatchTrace is a helper function to catch and handle all errors. It also recovers a panic and prints its call stack. CatchTrace and CatchAll are preferred helpers for go-workers on long-running servers because they stop panics as well.

CatchTrace prints only panic and runtime.Error stack trace if ErrorTracer isn't set. If it's set it prints both. The panic trace is printed to stderr. If you need panic trace to be printed to some other io.Writer than os.Stderr, you should use CatchAll or Catch with tracers.

func ErrorTracer

func ErrorTracer() io.Writer

ErrorTracer returns current io.Writer for automatic error stack tracing.

func Handle

func Handle(err *error, handlerFn func())

Handle is for adding an error handler to a function by deferring. It's for functions returning errors themself. For those functions that don't return errors, there is a CatchXxxx functions. The handler is called only when err != nil. There is no limit how many Handle functions can be added to defer stack. They all are called if an error has occurred and they are in deferred.

Example

package main

import (
	"fmt"

	"github.com/lainio/err2"
	"github.com/lainio/err2/try"
)

func throw() (string, error) {
	return "", fmt.Errorf("this is an ERROR")
}

func main() {
	doSomething := func(a, b int) (err error) {
		defer err2.Handle(&err, func() {
			// Example for just annotating current err. Normally Handle is
			// used for cleanup. See CopyFile example for more information.
			// Use err2.Returnf for err annotation.
			err = fmt.Errorf("error with (%d, %d): %v", a, b, err)
		})
		try.To1(throw())
		return err
	}
	err := doSomething(1, 2)
	fmt.Printf("%v", err)
}

Output:

error with (1, 2): this is an ERROR

func PanicTracer

func PanicTracer() io.Writer

PanicTracer returns current io.Writer for automatic panic stack tracing. Note that runtime.Error types which are transported by panics are controlled by this.

func Return

func Return(err *error)

Return is the same as Handle but it's for functions that don't wrap or annotate their errors. It's still needed to break panicking which is used for error transport in err2. If you want to annotate errors see Returnf and Returnw functions for more information.

Example

package main

import (
	"github.com/lainio/err2"
	"github.com/lainio/err2/try"
)

func noThrow() (string, error) { return "test", nil }

func main() {
	var err error
	defer err2.Return(&err)
	try.To1(noThrow())
}

func Returnf

func Returnf(err *error, format string, args ...any)

Returnf builds an error. It's similar to fmt.Errorf, but it's called only if error != nil. It uses '%v' to wrap the error not '%w'. Use Returnw for that.

Example

package main

import (
	"fmt"

	"github.com/lainio/err2"
	"github.com/lainio/err2/try"
)

func throw() (string, error) {
	return "", fmt.Errorf("this is an ERROR")
}

func main() {
	annotated := func() (err error) {
		defer err2.Returnf(&err, "annotated: %s", "err2")
		try.To1(throw())
		return err
	}
	err := annotated()
	fmt.Printf("%v", err)
}

Output:

annotated: err2: this is an ERROR

Example (DeferStack)

package main

import (
	"fmt"

	"github.com/lainio/err2"
	"github.com/lainio/err2/try"
)

func throw() (string, error) {
	return "", fmt.Errorf("this is an ERROR")
}

func main() {
	annotated := func() (err error) {
		defer err2.Returnf(&err, "annotated 2nd")
		defer err2.Returnf(&err, "annotated 1st")
		try.To1(throw())
		return err
	}
	err := annotated()
	fmt.Printf("%v", err)
}

Output:

annotated 2nd: annotated 1st: this is an ERROR

Example (Empty)

package main

import (
	"fmt"

	"github.com/lainio/err2"
	"github.com/lainio/err2/try"
)

func throw() (string, error) {
	return "", fmt.Errorf("this is an ERROR")
}

func main() {
	annotated := func() (err error) {
		defer err2.Returnf(&err, "annotated")
		try.To1(throw())
		return err
	}
	err := annotated()
	fmt.Printf("%v", err)
}

Output:

annotated: this is an ERROR

func Returnw

func Returnw(err *error, format string, args ...any)

Returnw wraps an error with '%w'. It's similar to fmt.Errorf, but it's called only if error != nil. If you don't want to wrap the error use Returnf instead.

func SetErrorTracer

func SetErrorTracer(w io.Writer)

SetErrorTracer sets a io.Writer for automatic error stack tracing. Note that runtime.Error types which are transported by panics are controlled by this.

func SetPanicTracer

func SetPanicTracer(w io.Writer)

SetPanicTracer sets a io.Writer for automatic panic stack tracing. Note that runtime.Error types which are transported by panics are controlled by this.

func SetTracers

func SetTracers(w io.Writer)

SetTracers a convenient helper to set a io.Writer for error and panic stack tracing.

func Throwf

func Throwf(format string, args ...any)

Throwf builds and throws (panics) an error. For creation it's similar to fmt.Errorf. Because panic is used to transport the error instead of error return value, it's called only if you want to non-local control structure for error handling, i.e. your current function doesn't have error return value. NOTE, Throwf is rarely needed. We suggest to use error return values instead. Throwf is offered for deep recursive algorithms to help readability.

func yourFn() (res any) {
     ...
     if badHappens {
          err2.Throwf("we cannot do that for %v", subject)
     }
     ...
}

Example

package main

import (
	"fmt"

	"github.com/lainio/err2"
)

func main() {
	type fn func(v int) int
	var recursion fn
	const recursionLimit = 77 // 12+11+10+9+8+7+6+5+4+3+2+1 = 78

	recursion = func(i int) int {
		if i > recursionLimit { // simulated error case
			err2.Throwf("helper failed at: %d", i)
		} else if i == 0 {
			return 0 // recursion without error ends here
		}
		return i + recursion(i-1)
	}

	annotated := func() (err error) {
		defer err2.Returnf(&err, "annotated: %s", "err2")

		r := recursion(12) // call recursive algorithm successfully
		recursion(r)       // call recursive algorithm unsuccessfully
		return err
	}
	err := annotated()
	fmt.Printf("%v", err)
}

Output:

annotated: err2: helper failed at: 78