evaltest

Documentation

Overview

Package evaltest supports testing the Elvish interpreter and libraries.

The entrypoint of this package is TestTranscriptsInFS. Typical usage looks like this:

import (
	"embed"
	"src.elv.sh/pkg/eval/evaltest"
)

//go:embed *.elv *.elvts
var transcripts embed.FS

func TestTranscripts(t *testing.T) {
	evaltest.TestTranscriptsInFS(t, transcripts)
}

See src.elv.sh/pkg/transcript for how transcript sessions are discovered.

Setup functions

TestTranscriptsInFS accepts variadic arguments in (name, f) pairs, where name must not contain any spaces. Each pair defines a setup function that may be referred to in the transcripts with the directive "//name".

The setup function f may take a *testing.T, *eval.Evaler and a string argument. All of them are optional but must appear in that order. If it takes a string argument, the directive can be followed by an argument after a space ("//name argument"), and that argument is passed to f. The argument itself may contain spaces.

The following setup functions are predefined:

• skip-test: Don't run this test. Useful for examples in .d.elv files that shouldn't be run as tests.

• in-temp-dir: Run inside a temporary directory.

• set-env $name $value: Run with the environment variable $name set to $value.

• unset-env $name: Run with the environment variable $name unset.

• eval $code: Evaluate the argument as Elvish code.

• only-on $cond: Evaluate $cond like a //go:build constraint and only run the test if the constraint is satisfied.

  The syntax is the same as //go:build constraints, but the set of supported tags is different and consists of: GOARCH and GOOS values, "unix", "32bit" and "64bit".

• deprecation-level $x: Run with deprecation level set to $x.

These setup functions can then be used in transcripts as directives. By default, they only apply to the current session; adding a "each:" prefix makes them apply to descendant sessions too.

//global-setup
//each:global-setup-2

# h1 #
//h1-setup
//each:h1-setup2

## h2 ##
//h2-setup

// All of globa-setup2, h1-setup2 and h2-setup are run for this session, in
// that

~> echo foo
foo

ELVISH_TRANSCRIPT_RUN

The environment variable ELVISH_TRANSCRIPT_RUN may be set to a string $filename:$lineno. If the location falls within the code lines of an interaction, the following happens:

1. Only the session that the interaction belongs to is run, and only up to the located interaction.

2. If the actual output doesn't match what's in the file, the test fails, and writes out a machine readable instruction to update the file to match the actual output.

As an example, consider the following fragment of foo_test.elvts (with line numbers):

12 ~> echo foo
13    echo bar
14 lorem
15 ipsum

Running

env ELVISH_TRANSCRIPT_RUN=foo_test.elvts:12 go test -run TestTranscripts

will end up with a test failure, with a message like the following (the line range is left-closed, right-open):

UPDATE {"fromLine": 14, "toLine": 16, "content": "foo\nbar\n"}

This mechanism enables editor plugins that can fill or update the output of transcript tests without requiring user to leave the editor.

Deterministic output order

When Elvish code writes to both the value output and byte output, or to both stdout and stderr, there's no guarantee which one appears first in the terminal.

To make testing easier, this package guarantees a deterministic order in such cases.

Index

• func TestTranscriptNodes(t *testing.T, nodes []*transcript.Node, setupPairs ...any)
• func TestTranscriptsInFS(t *testing.T, fsys fs.FS, setupPairs ...any)

Functions

func TestTranscriptNodes

func TestTranscriptNodes(t *testing.T, nodes []*transcript.Node, setupPairs ...any)

TestTranscriptsInFS runs parsed Elvish transcript nodes as tests.

func TestTranscriptsInFS

func TestTranscriptsInFS(t *testing.T, fsys fs.FS, setupPairs ...any)

TestTranscriptsInFS extracts all Elvish transcript sessions from .elv and .elvts files in fsys, and runs each of them as a test.