nescript

README

nescript    

Add automation to your network emulation workflows with NES & NEScript 🚀

An os.exec wrapper providing extra functionality and cleaner usage when using it heavily.

Added features include:

• Support for handlebar values in scripts (go templates)
• Function chaining for cleaner code
• Complex GitHub Actions style output parsing
• Dynamic evaluation of output using expressions (plugin-friendly 🔌)
• Script execution on the local machine, ssh target, or docker container (plugin-friendly 🔌)

---

Understanding Executive

Executive is divided into 5 core components:

• Script: A script is somewhat self explantory. A script can either be created from a source (string, file, http), and can contain template engine handlebars (awesome for loops, etc...). A script is not executed upon creation, instead further configuration can be set. When executing a script, a specific Executor should be specified (allowing for local & non-local execution).
• ExecFunc: A plugin that allows for scripts to be executed in many ways. Provided is a local executor (that just runs the script on the local machine), ssh executor (that executes the script on a remote SSH target), and a docker executor (for executing scripts on a docker container).
• Process: A process is an executing or executed script instance. Calling for a Result from this will wait for execution to be complete.
• Result: A result is the output of an executed script, including the exit code, stdout and stderr.
• Output: Output is key/value mapping of explicitly set outputs. This is done similarly to github actions, where outputs are picked up from stdout/stderr with a prefix similar to ::set-output name=example::.... As these values can be typed (string, int, JSON), they can also be evaluated based on expressions.

---

Key Features

Templating

Using templates in scripts is easy, just have a handlebar value in the script named as desired. For example:

{{.Command}} "Hello, {{.Name}}"

Then to set the value to replace {{.Name}}, the following go code can be used:

...
script, err := NewScript('{{.Command}} "Hello, {{.Name}}"').WithField("Command", "echo").WithField("Name", "world").Compile()
if err != nil {
	panic(err)
}
...

The templating system powering this supports other features too, such as loops when fields are slices of data etc...

Shebangs (#!/bin/bash etc...) should not be used as these can be hard to use on certain executors. Instead, NEScript allows for a sub-command to be set, for example sh -c, where the script is provided as the last argument. This overall seems to be a more portable approach.

Remote Execution

Scripts require an ExecFunc to actually be executed. There are the 3 provided, but more can easily be created. Executors, such as SSH, can have required configuration parameters.

The limitations are that optional working directories for script execution are not available when executing over SSH, and signals such as SIGINT are not supported when using a remote Docker target.

⚠️ When using env vars over SSH, be sure to allow any (*) env var on the SSH server by setting the AcceptEnv option in sshd

Output Handling & Evaluation

If specific output is desired to be able to evaluate a response to a script, this package allows for specific typed outputs to be set. If a line in StdOut or StdErr has a prefix similar to ::set-output name=example::, the rest of the line is stored as an output value with the key being provided in the name field. For example, the output key/value Hello/world can be set like so if a script is executing via a shell such as bash:

echo ::set-output name=Hello::world

By default, all outputted keys and values are considered to be string types. Optionally, a value can be an int, or a JSON structure. For example:

• INT: echo ::set-output name=example type=int::42
• JSON: echo ::set-output name=example type=json::{"sometext": "json", "anumber": 42}

Script results can then be programmatically evaluated to boolean values using expression functions (currently only a expr plugin is available). For example, to ensure the number set in the above JSON example is 42, the expression could be given:

...
if ok, err := result.CombinedOutput().Evaluate("example.anumber == 42"); err == nil && ok {
	fmt.Println("number is 42")
} else {
	panic(fmt.Errorf("number is not 42: %w", err))
}
...

Documentation

Index

• type EvalFunc
• type ExecFunc
• type Output
  • func NewOutput(source string) Output
  • func (o Output) Evaluate(evaluator EvalFunc, expression string) (bool, error)
• type Process
• type Result
  • func (r Result) CombinedOutput() Output
  • func (r Result) Output(useErr bool) Output
• type Script
  • func NewScript(raw string) *Script
  • func NewScriptFromFile(path string) (*Script, error)
  • func NewScriptFromHTTP(link string) (*Script, error)
  • func (s Script) Compile() (Script, error)
  • func (s Script) CompileExec(executor ExecFunc) (Process, error)
  • func (s Script) Data() map[string]any
  • func (s Script) Env() []string
  • func (s Script) Exec(executor ExecFunc) (Process, error)
  • func (s Script) MustCompile() Script
  • func (s Script) OSCmd(subcommand []string) (*exec.Cmd, error)
  • func (s Script) Raw() string
  • func (s Script) WithEnv(env ...string) Script
  • func (s Script) WithField(key string, value any) Script
  • func (s Script) WithFields(fields map[string]any, overwrite bool) Script
  • func (s Script) WithOSEnv() Script

Types

type EvalFunc

type EvalFunc func(Output, string) (bool, error)

type ExecFunc

type ExecFunc func(Script) (Process, error)

ExecFunc is used to execute a script, in-turn creating a process. If the script fails to be executed for any reason, this function can also error.

type Output

type Output map[string]any

Output is a key/value map of data, where the value can be any type. This should be generated from NewOutput or from the helper methods of a result.

func NewOutput

func NewOutput(source string) Output

NewOutput creates an Output from a given input string (such as stdout). It will type cast select types if a type is given in the set-output message (or a string if not).

func (Output) Evaluate

func (o Output) Evaluate(evaluator EvalFunc, expression string) (bool, error)

Evaluate takes an evaultion function (such as expr) and a string expression and uses the output data given to evaluate to a boolean. This will error is the expression can not be evaluation with the given data, or the output would not be a boolean.

type Process

type Process interface {
	// Kill sends a SIGKILL to the running process. If this fails, for example if
	// the process is not running, this will return an error.
	Kill() error

	// Signal sends a signal (such as SIGINT) to the running process. If this
	// fails, for example if the process is not running, this will return an
	// error.
	Signal(os.Signal) error

	// Write sends a string to the process's STDIN. Note that the string is sent
	// as-is. Thus if the program is looking for a newline before the read is
	// complete, this must be included in the string provided. If the write fails,
	// an error is returned.
	Write(string) error

	// Result waits for a script to complete execution, then a result is returned.
	// If the script returns an unknown error, this will also error.
	Result() (*Result, error)

	// Close should be called on a process, freeing any resources used where
	// appropriate.
	Close()
}

Process is a single instance of the script, either running or exited. A process can be used to control the script and extract results from a script that has completed its execution.

type Result

type Result struct {
	StdOut   string `json:"stdout"`
	StdErr   string `json:"stderr"`
	ExitCode int    `json:"exitCode"`

	TotalTime time.Duration `json:"executionTime"`
}

Result represents the output of a completed script execution. This is only created by the `Result` function of a "Process" and thus the process must have exited.

func (Result) CombinedOutput

func (r Result) CombinedOutput() Output

Output parses the specified outputs from the script's stdOut and stdErr. In the event that stdOut and stdErr specify an output of the same name, the value from stdOut is preferred. This is returned as a map. Any field that is not correctly parsed, will simply be ignored.

func (Result) Output

func (r Result) Output(useErr bool) Output

Output parses the specified outputs from the script's stdOut (or stdErr if specified). This is returned as a map. Any field that is not correctly parsed, will simply be ignored.

type Script

type Script struct {
	// contains filtered or unexported fields
}

Script is some executable string, along with data to supplement its execution, such as template data and env vars. The string can contain go template handles. These are to replace script arguments, as the use of arguments can be complex on certain platforms where the script may be executed.

func NewScript

func NewScript(raw string) *Script

NewScript creates a script based on the raw executable string.

func NewScriptFromFile

func NewScriptFromFile(path string) (*Script, error)

NewScriptFromFile creates a Script from the string extracted from a given file. This can error if the file can not be read.

func NewScriptFromHTTP

func NewScriptFromHTTP(link string) (*Script, error)

NewScriptFromHTTP creates a Script from the string extracted from a given URL. This can error if the contents of the remote resource can not be read.

func (Script) Compile

func (s Script) Compile() (Script, error)

Compile uses the go template engine and the provided data fields to compile the script. These in-turn act a more portable approach than command-line arguments.

func (Script) CompileExec

func (s Script) CompileExec(executor ExecFunc) (Process, error)

CompileExec will "compile" the script using the given data and the golang template system, then calls the given ExecFunc. Returned will be the process that is created as a result of execution. An error is returned if the script fails to execute for any reason.

func (Script) Data

func (s Script) Data() map[string]any

Data returns the map of template data to be used when compiling the script.

func (Script) Env

func (s Script) Env() []string

Env returns the env vars in KEY=VALUE format that will be used when executing the script.

func (Script) Exec

func (s Script) Exec(executor ExecFunc) (Process, error)

Exec will call the given ExecFunc to execute the script. Returned will be the process that is created as a result of execution. An error is returned if the script fails to execute for any reason.

func (Script) MustCompile

func (s Script) MustCompile() Script

MustCompile compiles the script, however will panic if an error occurs.

func (Script) OSCmd

func (s Script) OSCmd(subcommand []string) (*exec.Cmd, error)

OSCmd attempts to convert the script to an os.exec package Cmd. For this, a subcommand must be provided. For example, if the subcommand ["sh", "-c"] was provided, and the compiled script was `echo 'Hello, world!'“, the resulting command would be equivalent to: sh -c "echo 'Hello, world!'". Env vars given to the script are also preserved in the resulting Cmd.

func (Script) Raw

func (s Script) Raw() string

Raw returns the raw executable string as is. If the script contains template handlebars, they will be returned as provided, not compiled.

func (Script) WithEnv

func (s Script) WithEnv(env ...string) Script

WithEnv takes one or more environmental variables in KEY=VALUE format. These will be used when executing the script.

func (Script) WithField

func (s Script) WithField(key string, value any) Script

WithField adds a key/value to the map of template data to be used when compiling the script. If the key already exists, it is overwritten.

func (Script) WithFields

func (s Script) WithFields(fields map[string]any, overwrite bool) Script

WithFields takes a map of fields that is merged with the current script data. If a key already exists in the script data, overwite must be set to true in order to replace it, otherwise that key/value is left untouched.

func (Script) WithOSEnv

func (s Script) WithOSEnv() Script

WithOSEnv appends the environmental variables from the local system to the env var set currently held be the script.