README
¶
Streamline CLI and configuration code in Go applications using static analysis and code generation. Commands and config are described by convention, allowing automatic generation of flags, command handlers, docs, config loaders, etc.
Status
Beta quality: relatively new and hasn't been fully proven yet.
Usage
Install:
go get github.com/buchanae/cli/cmd/cli
In main_cli.go:
package main
import (
"fmt"
"net/http"
"github.com/buchanae/cli"
)
//go:generate cli .
type ServerOpt struct {
// Server name.
Name string
// Address to listen on.
Addr string
}
func DefaultServerOpt() ServerOpt {
return ServerOpt{
Name: "cli-example",
Addr: ":8080",
}
}
// Run a simple echo server.
// Example: ./server run --name "my-server" "Hello, world!"
func Run(opt ServerOpt, msg string) {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "from server %q: %s\n", opt.Name, msg)
})
cli.Check(http.ListenAndServe(opt.Addr, nil))
}
func main() {
cli.AutoCobra("server", specs())
}
Generate the CLI command handlers and build it:
go generate .
found cli "Run"
generated file generated_specs.go
go build -o server .
Try it out:
./server run 'Hello, world!'
curl localhost:8080
from server "cli-example": Hello, world!
Try it with some different options:
./server run --name "my-server" --addr :8081 'Hello, world!'
curl localhost:8081
from server "my-server": Hello, world!
Try it with a config file:
cat config.yaml
addr: ":8081"
name: "config-server"
./server run 'Hi!'
curl localhost:8081
from server "config-server": Hi!
This example code is in ./examples/server.
How it works
The cli utility parses the source code, looks for exported functions
and their options, then generates Go code containing this metadata (a cli.Spec).
At runtime, this metadata is used to generate commands, flags, docs, loaders, etc.
Conventions used:
- Only files with the suffix
_cli.goare analyzed. - Exported functions are turned into CLI commands.
- If a function has a struct-type argument named
opt, the fields of that type are used to generate flags, docs, file loaders, etc. - If an option type
MyOpthas a matching functionDefaultMyOpt() MyOpt, that function will provide default values for the options. - Command function arguments are coerced from CLI positional arguments,
e.g.Age(name string, age int)maps to./app age "Alex" 33
I'm most familiar with cobra and YAML config files, so I wrote
AutoCobra(appName string, specs []Spec) to handle my common usecase,
but hopefully cli is flexible enough to handle a wide variety of preferences.
Why?
Building powerful configuration and commandline interfaces is important, yet writing the code can be tedious, error-prone, and tricky.
Issues I frequently encounter:
-
Loading and merging config files, defaults, and CLI flags is error-prone and tricky. It's easy to get the order wrong, and correct code gets complex.
-
Config key names and CLI flag names can have inconsistent naming/casing (http.port vs --server_port).
-
Config files and CLI flags can get out of sync, e.g. add a new config option but forget to add a CLI flag.
-
A subset of config is available via CLI flags, a different subset via env. vars, and the rest requires a config file.
-
Names can be misspelled or incorrectly formatted, but the error passes silently, leading to subtle behavior that is difficult to debug.
-
time.Duration (and friends) is not handled well by common marshalers, such as YAML.
-
Wrappers for time.Duration are needed for proper (un)marshling from config files, but the wrappers then invade the whole codebase, leading to code like
time.NewTimer(time.Duration(config.TimeoutDuration)). -
Config docs easily get out of sync with the actual types.
-
Evolving config leads to broken systems when upgrading to newer versions.
-
Writing CLI and config code is often tedious, verbose, and covered in boilerplate. This is especially annoying when you build a CLI with lots of commands.
-
Unit testing is tricky because the CLI/config code usually interacts with the entire application. Organizing mocks or other tricks gets messy.
-
Unit testing is tricky because, again, you want to replace the usual stdin/out/err with something you can test.
-
Every new project develops a new pattern for loading, validation, testing, etc.
Design Goals
-
Configuration should feel natural when created and used in code. Config should be based on struct types, defaults should be provided by instances, or functions that return instances, of those types.
-
Documentation should be written in code, as with nearly all Go documentation. Tools should be provided to generate other forms of documentation.
-
Flags, environment variables, config files, and other types of data sources should use struct types as the source of truth while loading.
-
Common config errors, such as misspelling or non-existent keys, should be caught by the core library.
-
Merging multiple config sources should be handled simply and robustly by the core library.
-
Projects should be consistent in their config, CLI, and test code. The pattern and tools developed here should be robust enough to support many projects.
-
The pattern should allow for easily removing code that is duplicated amongst multiple commands, such as database init code.
-
The pattern should help reduce boilerplate and other sources of verbose code.
Other wants:
- Generate YAML config with dynamic values and docs.
Useful for doc generation, debugging, and bug reports.
Design Decisions
-
Use
cli.Fatalin top-level CLI code to minimize error checks. I totally agree with Go's approach to error handling by value, but panic/recover is useful too. A CLI command is top-level; it isn't expected to be called from other code, so the error values aren't being checked by anything, and panics aren't escaping into general code. CLI commands aren't being called in loops, so panic/recover performance isn't an issue. When an error occurs in a CLI command, you usually want the whole program to stop with an error, which seems like a good fit for Fatal/panic. -
Use code generation to inspect commands and config. The best way to keep config and docs up to date is to have it written right next to the code in the form of code comments.
-
Keep code generation minimal; generate just enough information for libraries to do the rest at runtime. Details such as cobra command building, doc parsing, flag building, etc. could all happen during code generation, but it feels slightly less flexible and more likely to become complex. Also, more strings/data being generated as code means larger binaries for projects with lots of commands. Honestly, I'm on the fence here though.
-
Allow an alternative to struct tags. Sometimes you don't have access to the struct type, or you don't want to modify it (maybe it's generated by protoc).
To do / Known Issues
- be able to hide/ignore fields without using a struct tag, for fields which you don't have access to or don't want to modify with cli tags.
- provide
sensitivetag for passwords and other sensitive fields. - properly marshal yaml/json slices/maps/etc.
- GCE metadata, etcd, consul, openstack provider
- dump json, env, flags
- handle map[string]string via "key=value" flag value
- pull fieldname from json tag
- ignore/alias fields via struct tag
- recognize misspelled env var
- case sensitivity
- manage editing config file
Complex:
- reloading
- multiple config files with merging
Questions:
- how to handle pointers? cycles?
- how are slices handled in env vars?
- how are slices of structs handled in flags?
Documentation
¶
Overview ¶
package cli provides utilities for streamlining CLI and configuration code. See github.com/buchanae/cli for a full example.
Index ¶
- Variables
- func AutoCobra(appname string, specs []Spec) error
- func Check(err error)
- func Coerce(dest interface{}, val interface{}) error
- func Enrich(cmd *Cmd)
- func Fatal(msg string, args ...interface{})
- func Run(spec Spec, l *Loader, raw []string) (err error)
- type Arg
- type Cmd
- type Cobra
- type ErrFatal
- type ErrUsage
- type FileOpts
- type KeyFunc
- type Loader
- type Opt
- type Provider
- type Spec
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // DotKey joins key parts with a "." and converts to lowercase. DotKey KeyFunc = lowerJoin(".") // UnderscoreKey joins key parts with a "_" and converts to lowercase. UnderscoreKey = lowerJoin("_") // DashKey joins key parts with a "-" and converts to lowercase. DashKey = lowerJoin("-") )
var DefaultJSON = FileOpts{ Paths: []string{"config.json"}, OptKey: []string{"config"}, }
DefaultJSON contains the most common configuration for loading options from a JSON config file.
var DefaultTOML = FileOpts{ Paths: []string{"config.toml"}, OptKey: []string{"config"}, }
DefaultTOML contains the most common configuration for loading options from a TOML config file.
var DefaultYAML = FileOpts{ Paths: []string{"config.yaml", "config.yml"}, OptKey: []string{"config"}, }
DefaultYAML contains the most common configuration for loading options from a YAML config file.
Functions ¶
func AutoCobra ¶
AutoCobra sets up a common pattern for apps: commands are built into a tree of cobra subcommands, flags are created using pflag, options are loaded from env. vars, flags, and YAML config files.
func Coerce ¶
func Coerce(dest interface{}, val interface{}) error
Coerce attempts to coerce "val" to the type of "dest".
func Enrich ¶
func Enrich(cmd *Cmd)
Enrich fills in Cmd and Opt fields runtime by parsing names and doc strings, looking for annotations such as Synopsis, Deprecated, Example, etc.
Example ¶
cmd := &Cmd{
RawName: "WordCount",
RawDoc: `Count the number of words in a file.
word-count counts the number of words in a file and writes a single number to stdout.
This is a second line of docs.
Deprecated: please don't use word-count, it will be removed.
Hidden
Aliases: count-words count-word wc
Example: word-count <file.txt>
`,
Opts: []*Opt{
{
RawDoc: `Opt detail synopsis.`,
},
{
DefaultValue: os.Stderr,
},
},
}
Enrich(cmd)
fmt.Println("NAME: ", cmd.Name)
fmt.Println("PATH: ", cmd.Path)
fmt.Println("SYN: ", cmd.Synopsis)
fmt.Println("EX: ", cmd.Example)
fmt.Println("DEP: ", cmd.Deprecated)
fmt.Println("HIDE: ", cmd.Hidden)
fmt.Println("ALIAS:", cmd.Aliases)
fmt.Println("SYN: ", cmd.Opts[0].Synopsis)
fmt.Println("DEF: ", cmd.Opts[1].DefaultString)
Output: NAME: count PATH: [word count] SYN: Count the number of words in a file. EX: word-count <file.txt> DEP: please don't use word-count, it will be removed. HIDE: true ALIAS: [count-words count-word wc] SYN: Opt detail synopsis. DEF: os.Stderr
func Fatal ¶
func Fatal(msg string, args ...interface{})
Fatal panics with an instance of ErrFatal with a formatted message.
Types ¶
type Arg ¶
type Arg struct {
// Name is the name of this argument.
Name string
// Type contains a string describing the type of this field,
// e.g. "[]string".
Type string
// Variadic is true if this function argument is variadic,
// e.g. func example(a string, b ...string)
// "b" is variadic.
Variadic bool
// Value contains a pointer to the value for this argument.
// Used by `cli` machinery to set the value of this argument.
Value interface{}
}
Arg defines a positional argument of a Cmd.
Usually generated by the `cli` code generator.
type Cmd ¶
type Cmd struct {
// RawName is the raw, unprocessed name of the function.
RawName string
// RawDoc is the raw, unprocessed doc string attached to the function.
RawDoc string
// Args describes metadata about the function arguments.
Args []*Arg
// Opts describes metadata about the function options
// (the `opt` argument, by convention).
Opts []*Opt
// Name is the command name,
// normally determined by parsing the raw function name and/or doc.
Name string
// Path is the path to this subcommand,
// normally determined by parsing the raw function name and/or docs.
Path []string
// Doc is the function doc after processing,
// where annotations such as "Deprecated" and "Hidden" are removed.
Doc string
// Synopsis is a short description of the command.
Synopsis string
// Example describes an example of how to use the command.
Example string
// Deprecated marks this command as deprecated and contains
// a message describing why.
Deprecated string
// Hidden marks this command as hidden.
Hidden bool
// Aliases contains aliases for this command.
Aliases []string
}
Cmd holds metadata related to a CLI command.
Usually generated by the `cli` code generator.
type Cobra ¶
Cobra helps build a set of cobra commands. Commands are built into a tree of subcommands based on their common subpaths.
type ErrFatal ¶
type ErrFatal struct {
// contains filtered or unexported fields
}
ErrFatal is used to signal fatal errors coming from `Fatal`.
type ErrUsage ¶
type ErrUsage struct {
// contains filtered or unexported fields
}
ErrUsage is used to signal fatal errors caused by invalid commandline usage.
type FileOpts ¶
type FileOpts struct {
// Paths is a list of paths to look for a config file.
// Environment variables will be expanded using `os.ExpandEnv`.
// Loading will stop on the first path that exists.
Paths []string
// OptKey is used to look for a config file path set by
// an option (e.g. by a flag or env. var). For example,
// an OptKey of ["config", "file"] could load the config
// file from a "--config.file" flag. OptKey is prepended
// to the Paths list, so it takes priority.
OptKey []string
}
FileOpts describes options related to loading options from a file.
type KeyFunc ¶
KeyFunc defines a function which transforms a key.
Example ¶
path := []string{"Foo", "bar", "BAZ"}
fmt.Println(DotKey(path))
fmt.Println(UnderscoreKey(path))
fmt.Println(DashKey(path))
Output: foo.bar.baz foo_bar_baz foo-bar-baz
type Loader ¶
type Loader struct {
// Coerce can be used to override the type coercion
// needed when setting an option value. A coerce function
// must set the value. "dst" is always a pointer to the
// value which needs to be set, e.g. *int for an option
// value of type int. See coerce.go for an example.
Coerce func(dst, src interface{}) error
// contains filtered or unexported fields
}
Loader is used to load, coerce, and set option values at command run time. Loader.Load runs the providers in order. Once an option has been set, it is not overridden by later values.
func NewLoader ¶
NewLoader returns a Loader instance which is configured to load option values from the given providers.
func (*Loader) Load ¶
func (l *Loader) Load()
Load runs the providers, loading and setting option values. Load is meant to be called only once. TODO this is awkward. The code should make it clear that
it only gets called once, and/or make it safe to load multiple times.
func (*Loader) Set ¶
Set sets an option value for the option at the given key. Once an option is set, it will not be overridden by future calls to Set. Set uses Loader.Coerce to set the value.
TODO this is ugly. This checks IsSet, but doesn't set it to true,
that happens somewhere else. The whole set/coerce interface needs cleanup
type Opt ¶
type Opt struct {
// Key is the path of struct fields names from the root to this option.
// e.g. ["Server", "TLS", "KeyPath"]
Key []string
// RawDoc is the raw, unprocessed doc string attached to this field.
RawDoc string
// Doc is the field doc after processing,
// where annotations such as "Deprecated" and "Hidden" are removed.
Doc string
// Synopsis is a short description of this option.
Synopsis string
// Hidden marks this option as hidden.
Hidden bool
// Deprecated marks this option as deprecated and contains
// a message describing why.
Deprecated string
// Type contains a string describing the type of this field,
// e.g. "[]string".
Type string
// Short is the name of the short version of a flag for this option.
Short string
// Value contains a pointer to the value for this option.
// Used by Loader machinery to set the value of this option.
Value interface{}
// DefaultValue contains the default value of this option.
DefaultValue interface{}
// DefaultString contains a more human-friendly description
// of the default value, e.g. os.Stderr instead of io.Writer.
DefaultString string
// IsSet is true if this value has been set.
// Used by Loader machinery.
// TODO ugly.
IsSet bool
}
Opt holds metadata related to an option of a Cmd.
Usually generated by the `cli` code generator.
type Provider ¶
Provider is implemented by types which provide option values, such as from environment variables, config files, CLI flags, etc.
func Env ¶
Env loads option values from environment variables with the given prefix. The prefix and option keys are converted to uppercase.
Example ¶
// Note: Env converts keys to uppercase + underscore.
os.Setenv("CLI_FOO_BAR", "baz")
// TODO gross
val := ""
key := []string{"foo", "bar"}
opts := []*Opt{
{Key: key, Value: &val},
}
l := NewLoader(opts, Env("cli"))
l.Load()
fmt.Println(val)
Output: baz
Source Files
Directories