README
¶
polyester
Bootstrap servers, apps, and secrets. Run applications with secrets mapped to environment variables. Template configuration files and scripts. Continuously deploy. Made for version control.
status
Pretty unstable!
usage
plans
Since we're an ops tool, we need a magic directory structure:
.
├── files
├── plans
│ ├── myapp
│ │ ├── files
│ │ └── templates
│ └── myapp2
├── templates
│ └── systemd
└── vars
agent
run a collection of plans (default usage):
$ make && ./polyester apply testdata/basic
run a single plan in a filesystem sandbox, w/ an overridden state directory:
$ make && ./polyester apply --dir-root /tmp/polytest --state-dir /tmp/polystate testdata/basic/plans/touchy/plan.sh
Normal usage entails running polyester apply on remote servers. To continuously update a git repository containing the polyester manifest, a cron could periodically run a script such as:
$ cd ~/repos/cluster && git pull && polyester apply
how does it work
The key concepts are "plans" and "operators". Plans are sequences of operations, used to execute commands on an environment. Operations are run in order, by plan (which run concurrently). There is a caching strategy for operations where, if an operation's state changes, it and every subsequent operation is executed.
The primary domain language is POSIX shell (though others could be supported without a huge amount of effort). Shell scripts are evaluated to generate the execution plan by outputting it to an intermediate format in the local filesystem. This means variable scope and other behavior may not be what you expect because the script doesn't immediately execute, but rather constructs an intermediate plan.
Operators idempotently execute operations and track state. In many cases they extend common linux tools. Some example operators:
- useradd
- touch
- copy
See the testdata/ directory for example plans.
development
build it:
$ make
run docker tests:
$ go test ./cmd/polyester -run TestDocker
Operators can be partially "sandboxed" using the --dir-root flag, which can be useful for debugging. For example, to override the state directory and use a filesystem sandbox:
$ ./polyester apply --dir-root /tmp/mysandbox --state-dir /tmp/mystate testdata/basic
todo
- operators
- more/better secret management (sops, ?)
- templating (maybe w/ gomplate)
- systemd operators to reenable, reinstall, restart units on state changes (maybe shell is enough).
- apt add source
- certbot
- shell plan script improvements
- validate operator calls in shell scripts pre-execution
- handling variables / scope in shell script plans
- planner improvements
- improve dryrun contract -- ie planner passes --dry-run to commands that have it otherwise execution is skipped.
- use remote, versioned git repo / tar / exported plans
- prune old / unused state files
- output formats
- nice readable apply summaries, what changed etc
- json log for debugging, introspection, integration testing
- operations interface (maybe just stringer) to control its argument formatting
- testing
- parameterize base docker image to run against various distros & oses
- generic table test for operation idempotency
- replace github repo in docker basic test w/ a proper test fixture repo on the local fs
- docs
- write some
- templatized operation docs using go generate / gomplate / operation.Info
- example repo / article
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
polyester/commands
Package commands contains the available polyester cli commands.
|
Package commands contains the available polyester cli commands. |
|
Package compiler contains code to compile manifests into executable plans.
|
Package compiler contains code to compile manifests into executable plans. |
|
shell
Package shell handles parsing shell scripts in plan files.
|
Package shell handles parsing shell scripts in plan files. |
|
Package executil wraps some functions in the exec package to ease testing and common subprocess use cases.
|
Package executil wraps some functions in the exec package to ease testing and common subprocess use cases. |
|
Package manifest contains functions to save and load manifests.
|
Package manifest contains functions to save and load manifests. |
|
Package operator is an interface for defining arbitrary operators, which gather state and make changes to an environment.
|
Package operator is an interface for defining arbitrary operators, which gather state and make changes to an environment. |
|
facts
Package facts manages "facts", which are data gathered from the environment of the agent.
|
Package facts manages "facts", which are data gathered from the environment of the agent. |
|
fileop
Package fileop contains filesystem-related operators.
|
Package fileop contains filesystem-related operators. |
|
gitop
Package gitop contains operators that use git.
|
Package gitop contains operators that use git. |
|
opfs
Package opfs is an fs implementation that supports stat, reads, and globbing.
|
Package opfs is an fs implementation that supports stat, reads, and globbing. |
|
pkgop
Package pkgop contains operators for using package managers.
|
Package pkgop contains operators for using package managers. |
|
planop
Package planop contains a dependency operator and other operations on plans.
|
Package planop contains a dependency operator and other operations on plans. |
|
shellop
Package shellop contains shell operators.
|
Package shellop contains shell operators. |
|
templateop
Package templateop contains template operations.
|
Package templateop contains template operations. |
|
templates
Package templates contains a template manager.
|
Package templates contains a template manager. |
|
userop
Package userop contains user operators.
|
Package userop contains user operators. |
|
Package planner traverses file systems, loading state according to modules, managing execution, and interfacing with the command line.
|
Package planner traverses file systems, loading state according to modules, managing execution, and interfacing with the command line. |
|
execute
Package execute contains the logic to execute plans concurrently, taking into account dependencies and phases.
|
Package execute contains the logic to execute plans concurrently, taking into account dependencies and phases. |
|
format
Package format contains code to control the look of planner output.
|
Package format contains code to control the look of planner output. |
|
Package state manages polyester states.
|
Package state manages polyester states. |
|
Package stdio manages standard io in a way that's easily mockable in tests while also not depending on overriding os.Stdin, os.Stdout, and os.Stderr.
|
Package stdio manages standard io in a way that's easily mockable in tests while also not depending on overriding os.Stdin, os.Stdout, and os.Stderr. |
|
Package testenv contains testing helpers for creating test harnesses and other common tasks.
|
Package testenv contains testing helpers for creating test harnesses and other common tasks. |