gonix: unix as a Go library
Unix text utilities implemented in pure Go, using
github.com/gomoni/gio/unix
and an excellent github.com/benhoyt/goawk
- ⚠ not yet guaranteed to be stable, API and a project layout MAY change
- ✔ Go library
- ✔ Native pipes in Go
Native filters
- awk - a thin wrapper for goawk
- cat -uses goawk
- cksum - POSIX ctx, md5 and sha check sums, runs concurrently (
-j/--threads
) by default
- head -n/--lines - uses goawk
- wc - word count
Work in progress
- x/tr - translate characters
Go library
Each filter can be called from Go code.
head := head.New().Lines(2)
err := head.Run(context.TODO(), unix.NewStdio(
bytes.NewBufferString("three\nsmall\npigs\n"),
os.Stdout,
os.Stderr,
))
if err != nil {
log.Fatal(err)
}
// Output:
// three
// small
Native pipes in Go
Unix is unix because of a pipe(2)
allowing a seamless combination of all unix filters into longer colons.
gonix
has pipe.Run
allowing to connect and arbitrary number of filters. It connects stdin/stdout
automatically like unix sh(1)
do.
// printf "three\nsmall\npigs\n" | cat | wc -l
err := unix.NewLine().Run(ctx, stdio, cat.New(), wc.New().Lines(true))
if err != nil {
log.Fatal(err)
}
// Output:
// 3
Architecture of a filter
- Each command is represented as Go struct
- New() returns a pointer to zero structure, no default values are passed in
- Optional
FromArgs([]string)(*Struct, error)
provides cli parsing and implements defaults
- It does defer most of runtime errors to
Run
method
Run(context.Context, pipe.Stdio) error
method gets a value receiver so it never changes the configuration
// wc does nothing, as it has all zeroes - an equivalent of &wc.Wc{} or new(Wc)
wc := wc.New()
// wc gets Lines(true) Chars(true) Bytes(true)
wc, err := wc.FromArgs(nil)
// wc gets chars(false)
wc = wc.Chars(false)
// wc is a value receiver, so never changes the configuration
err = wc.Run(...)
Internal helpers
internal.RunFiles
abstracts running a command over stdin (and) or list of
files. Takes a care about opening and proper closing the files, does errors
gracefully, so they do not cancel the code to run, but are propagated to caller
properly. Supports a parallel execution of tasks via internal.PMap
so cksum
run in a parallel by default.
internal.PMap
is a parallel map algorithm. Executes MapFunc, which converts
input slices to output slice and each execution is capped by maximum number of
threads. It maintains the order.
internal.Unit
and internal.Byte
is a fork of time.Duration of stdlib, which
supports bigger ranges (based on float64). New units can be easily defined on
top of Unit type.
Testing
The typical testing is very repetitive, so there is a common structure for build of
table tests. It uses generics to improve a type safety.
import "github.com/gomoni/gonix/internal/test"
testCases := []test.Case[Wc]{
{
Name: "wc -l",
Filter: New().Lines(true),
FromArgs: fromArgs(t, []string{"-l"}),
Input: "three\nsmall\npigs\n",
Expected: "3\n",
},
}
test.RunAll(t, testCases)
Where the struct fields are
- Name is name of test case to be printed by go test
- Input is a string input for a particular command
- Expected is what command is supposed to generate
- Filter is a definition of a filter
- FromArgs is an alternative definition obtained by
FromArgs
helper. It
ensures CLI parsing is tested as a part of regular functional testing
Testing with real files
WIP atm, there is test.TestData
helper and a bunch of code in
cksum/cksum_test.go
to run tests using real files.
Other interesting projects