shell

package
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 1, 2024 License: BSD-3-Clause Imports: 22 Imported by: 0

README

Cogent Shell (cosh)

Cogent Shell (cosh) is a shell that combines the best parts of Go and command-based shell languages like bash to provide an integrated shell experience that allows you to easily run terminal commands while using Go for complicated logic. This allows you to write concise, elegant, and readable shell code that runs quickly on all platforms, and transpiles to Go (i.e, can be compiled by go build).

The simple idea is that each line is either Go or shell commands, determined in a fairly intuitive way mostly by the content at the start of the line (formal rules below), and they can be intermixed by wrapping Go within { } and shell code from within backticks (`````). We henceforth refer to shell code as exec code (in reference to the Go & Cogent exec package that we use to execute programs), given the potential ambituity of the entire cosh language being the shell. There are different syntactic formatting rules for these two domains of Go and Exec, within cosh:

  • Go code is processed and formatted as usual (e.g., white space is irrelevant, etc).
  • Exec code is space separated, like normal command-line invocations.

Examples:

for i, f := range strings.Split(`ls -la`, "/n") {   // `ls` executes returns string
    echo {i} {strings.ToLower(f)}           // {} surrounds go within shell
}

Special syntax

Multiple statements per line

  • Multiple statements can be combined on one line, separated by ; as in regular Go and shell languages. Critically, the language determination for the first statement determines the language for the remaining statements; you cannot intermix the two on one line, when using ;

Exec mode

Environment variables

  • set <var> <value> (space delimited as in all exec mode, no equals)

Output redirction

  • Standard output redirect: > and >& (and |, |& if needed)

Control flow

  • Any error stops the script execution, except for statements wrapped in [ ], indicating an "optional" statement, e.g.:
cd some; [mkdir sub]; cd sub

  • & at the end of a statement runs in the background (as in bash) -- otherwise it waits until it completes before it continues.

  • jobs, fg, bg, and kill builtin commands function as in usual bash.

Exec functions (aliases)

Use the command keyword to define new functions for Exec mode execution, which can then be used like any other command, for example:

command list {
	ls -la args...
}

cd data
list *.tsv

The command is transpiled into a Go function that takes args ...string. In the command function body, you can use the args... expression to pass all of the args, or args[1] etc to refer to specific positional indexes, as usual.

The command function name is registered so that the standard shell execution code can run the function, passing the args. You can also call it directly from Go code using the standard parentheses expression.

SSH connections to remote hosts

Any number of active SSH connections can be maintained and used dynamically within a script, including simple ways of copying data among the different hosts (including the local host). The Go mode execution is always on the local host in one running process, and only the shell commands are executed remotely, enabling a unique ability to easily coordinate and distribute processing and data across various hosts.

Each host maintains its own working directory and environment variables, which can be configured and re-used by default whenever using a given host.

  • cossh hostname.org [name] establishes a connection, using given optional name to refer to this connection. If the name is not provided, a sequential number will be used, starting with 1, with 0 referring always to the local host.

  • @name then refers to the given host in all subsequent commands, with @0 referring to the local host where the cosh script is running.

Explicit per-command specification of host
@name cd subdir; ls
Default host
@name // or:
cossh @name

uses the given host for all subsequent commands (unless explicitly specified), until the default is changed. Use cossh @0 to return to localhost.

Redirect input / output among hosts

The output of a remote host command can be sent to a file on the local host:

@name cat hostfile.tsv > @0:localfile.tsv

Note the use of the : colon delimiter after the host name here. TODO: You cannot send output to a remote host file (e.g., > @host:remotefile.tsv) -- maybe with sftp?

The output of any command can also be piped to a remote host as its standard input:

ls *.tsv | @host cat > files.txt
scp to copy files easily

The builtin scp function allows easy copying of files across hosts, using the persistent connections established with cossh instead of creating new connections as in the standard scp command.

scp is always run from the local host, with the remote host filename specified as @name:remotefile

scp @name:hostfile.tsv localfile.tsv

TODO: Importantly, file wildcard globbing works as expected:

scp @name:*.tsv @0:data/

and entire directories can be copied, as in cp -a or cp -r (this behavior is automatic and does not require a flag).

Close connections
cossh close

Will close all active connections and return the default host to @0. All active connections are also automatically closed when the shell terminates.

Other Utilties

** need a replacement for findnm -- very powerful but garbage..

Rules for Go vs. Shell determination

The critical extension from standard Go syntax is for lines that are processed by the Exec functions, used for running arbitrary programs on the user's executable path. Here are the rules (word = IDENT token):

  • Backticks "``" anywhere: Exec. Returns a string.
  • Within Exec, {}: Go
  • Line starts with Go Keyword: Go
  • Line is one word: Exec
  • Line starts with path: Exec
  • Line starts with "string": Exec
  • Line starts with word word: Exec
  • Line starts with word {: Exec
  • Otherwise: Go

TODO:

  • probably need to have wrapper StdIO for every exec so we can flexibly redirect for fg, bg commands.
  • likewise, need to run everything effectively as a bg job with our own explicit Wait, which we can then communicate with to move from fg to bg.

Documentation

Overview

Package shell provides the Cogent Shell (cosh), which combines the best parts of Go and bash to provide an integrated shell experience that allows you to easily run terminal commands while using Go for complicated logic.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddQuotes added in v0.1.4 

func AddQuotes(str string) string

AddQuotes surrounds given string with quotes, also escaping any contained quotes

func EscapeQuotes added in v0.1.4 

func EscapeQuotes(str string) string

EscapeQuotes replaces any " with \"

func ExecWordIsCommand added in v0.1.4 

func ExecWordIsCommand(f string) bool

ExecWordIsCommand returns true if given exec word is a command-like string (excluding any paths)

func ExecWords added in v0.1.4 

func ExecWords(ln string) ([]string, error)

Types

type ReadlineCompleter added in v0.1.4 

type ReadlineCompleter struct {
	Shell *Shell
}

ReadlineCompleter implements github.com/ergochat/readline.AutoCompleter.

func (*ReadlineCompleter) Do added in v0.1.4 

func (rc *ReadlineCompleter) Do(line []rune, pos int) (newLine [][]rune, length int)

type Shell 

type Shell struct {

	// Config is the [exec.Config] used to run commands.
	Config exec.Config

	// StdIOWrappers are IO wrappers sent to the interpreter, so we can
	// control the IO streams used within the interpreter.
	// Call SetWrappers on this with another StdIO object to update settings.
	StdIOWrappers exec.StdIO

	// ssh connection, configuration
	SSH *sshclient.Config

	// collection of ssh clients
	SSHClients map[string]*sshclient.Client

	// SSHActive is the name of the active SSH client
	SSHActive string

	// depth of parens at the end of the current line. if 0, was complete.
	ParenDepth int

	// depth of braces at the end of the current line. if 0, was complete.
	BraceDepth int

	// depth of brackets at the end of the current line. if 0, was complete.
	BrackDepth int

	// stack of transpiled lines, that are accumulated in TranspileCode
	Lines []string

	// stack of runtime errors
	Errors []error

	// Builtins are all the builtin shell commands
	Builtins map[string]func(cmdIO *exec.CmdIO, args ...string) error

	// commands that have been defined, which can be run in Exec mode.
	Commands map[string]func(args ...string)

	// Jobs is a stack of commands running in the background
	// (via Start instead of Run)
	Jobs stack.Stack[*exec.CmdIO]

	// Cancel, while the interpreter is running, can be called
	// to stop the code interpreting.
	// It is connected to the Ctx context, by StartContext()
	// Both can be nil.
	Cancel func()

	// Ctx is the context used for cancelling current shell running
	// a single chunk of code, typically from the interpreter.
	// We are not able to pass the context around so it is set here,
	// in the StartContext function. Clear when done with ClearContext.
	Ctx context.Context
	// contains filtered or unexported fields
}

Shell represents one running shell context.

func NewShell 

func NewShell() *Shell

NewShell returns a new Shell with default options.

func (*Shell) ActiveSSH added in v0.1.4 

func (sh *Shell) ActiveSSH() *sshclient.Client

ActiveSSH returns the active ssh client

func (*Shell) AddCommand added in v0.1.4 

func (sh *Shell) AddCommand(name string, cmd func(args ...string))

AddCommand adds given command to list of available commands

func (*Shell) AddError 

func (sh *Shell) AddError(err error) error

AddError adds the given error to the error stack if it is non-nil, and calls the Cancel function if set, to stop execution. This is the main way that shell errors are handled. It also prints the error.

func (*Shell) AddLine 

func (sh *Shell) AddLine(ln string)

AddLine adds line on the stack

func (*Shell) AddPath added in v0.1.4 

func (sh *Shell) AddPath(cmdIO *exec.CmdIO, args ...string) error

AddPath adds the given path(s) to $PATH.

func (*Shell) CancelExecution added in v0.1.4 

func (sh *Shell) CancelExecution()

CancelExecution calls the Cancel() function if set.

func (*Shell) Cd 

func (sh *Shell) Cd(cmdIO *exec.CmdIO, args ...string) error

Cd changes the current directory.

func (*Shell) Close added in v0.1.4 

func (sh *Shell) Close()

Close closes any resources associated with the shell, including terminating any commands that are not running "nohup" in the background.

func (*Shell) CloseSSH added in v0.1.4 

func (sh *Shell) CloseSSH()

CloseSSH closes all open ssh client connections

func (*Shell) CmdArgs added in v0.1.4 

func (sh *Shell) CmdArgs(errOk bool, sargs []string, i int) []string

CmdArgs processes expressions involving "args" for commands

func (*Shell) CoSSH added in v0.1.4 

func (sh *Shell) CoSSH(cmdIO *exec.CmdIO, args ...string) error

CoSSH manages SSH connections, which are referenced by the @name identifier. It handles the following cases:

  • @name -- switches to using given host for all subsequent commands
  • host [name] -- connects to a server specified in first arg and switches to using it, with optional name instead of default sequential number.
  • close -- closes all open connections, or the specified one

func (*Shell) Code 

func (sh *Shell) Code() string

Code returns the current transpiled lines

func (*Shell) CompleteEdit added in v0.1.4 

func (sh *Shell) CompleteEdit(data any, text string, cursorPos int, completion complete.Completion, seed string) (ed complete.Edit)

CompleteEdit is the complete.EditFunc for the shell.

func (*Shell) CompleteMatch added in v0.1.4 

func (sh *Shell) CompleteMatch(data any, text string, posLine, posChar int) (md complete.Matches)

CompleteMatch is the complete.MatchFunc for the shell.

func (*Shell) DeleteJob added in v0.1.4 

func (sh *Shell) DeleteJob(cmdIO *exec.CmdIO) bool

DeleteJob deletes the given job and returns true if successful

func (*Shell) EndContext added in v0.1.4 

func (sh *Shell) EndContext()

EndContext ends a processing context, clearing the Ctx and Cancel fields.

func (*Shell) Exec 

func (sh *Shell) Exec(errOk, start, output bool, cmd any, args ...any) string

Exec handles command execution for all cases, parameterized by the args. It executes the given command string, waiting for the command to finish, handling the given arguments appropriately. If there is any error, it adds it to the shell, and triggers CancelExecution.

  • errOk = don't call AddError so execution will not stop on error
  • start = calls Start on the command, which then runs asynchronously, with a goroutine forked to Wait for it and close its IO
  • output = return the output of the command as a string (otherwise return is "")

func (*Shell) ExecArgs added in v0.1.4 

func (sh *Shell) ExecArgs(cmdIO *exec.CmdIO, errOk bool, cmd any, args ...any) (*sshclient.Client, string, []string)

ExecArgs processes the args to given exec command, handling all of the input / output redirection and file globbing, homedir expansion, etc.

func (*Shell) Exit added in v0.1.4 

func (sh *Shell) Exit(cmdIO *exec.CmdIO, args ...string) error

Exit exits the shell.

func (*Shell) Fg added in v0.1.4 

func (sh *Shell) Fg(cmdIO *exec.CmdIO, args ...string) error

Fg foregrounds a job by job number

func (*Shell) HandleArgErr added in v0.1.4 

func (sh *Shell) HandleArgErr(errok bool, err error) error

func (*Shell) Host added in v0.1.4 

func (sh *Shell) Host() string

Host returns the name we're running commands on, which is empty if localhost (default).

func (*Shell) HostAndDir added in v0.1.4 

func (sh *Shell) HostAndDir() string

HostAndDir returns the name we're running commands on, which is empty if localhost (default), and the current directory on that host.

func (*Shell) InstallBuiltins 

func (sh *Shell) InstallBuiltins()

InstallBuiltins adds the builtin shell commands to [Shell.Builtins].

func (*Shell) JobIDExpand added in v0.1.4 

func (sh *Shell) JobIDExpand(args []string) int

JobIDExpand expands %n job id values in args with the full PID returns number of PIDs expanded

func (*Shell) JobsCmd added in v0.1.4 

func (sh *Shell) JobsCmd(cmdIO *exec.CmdIO, args ...string) error

JobsCmd is the builtin jobs command

func (*Shell) Kill added in v0.1.4 

func (sh *Shell) Kill(cmdIO *exec.CmdIO, args ...string) error

Kill kills a job by job number or PID. Just expands the job id expressions %n into PIDs and calls system kill.

func (*Shell) OutToFile added in v0.1.4 

func (sh *Shell) OutToFile(cl *sshclient.Client, cmdIO *exec.CmdIO, errOk bool, sargs []string, i int) []string

OutToFile processes the > arg that sends output to a file

func (*Shell) OutToPipe added in v0.1.4 

func (sh *Shell) OutToPipe(cl *sshclient.Client, cmdIO *exec.CmdIO, errOk bool, sargs []string, i int) []string

OutToPipe processes the | arg that sends output to a pipe

func (*Shell) Output 

func (sh *Shell) Output(cmd any, args ...any) string

Output executes the given command string, handling the given arguments appropriately. If there is any error, it adds it to the shell. It returns the stdout as a string and forwards stderr to exec.Config.Stderr appropriately.

func (*Shell) OutputErrOK added in v0.1.4 

func (sh *Shell) OutputErrOK(cmd any, args ...any) string

OutputErrOK executes the given command string, handling the given arguments appropriately. If there is any error, it adds it to the shell. It returns the stdout as a string and forwards stderr to exec.Config.Stderr appropriately.

func (*Shell) ResetLines 

func (sh *Shell) ResetLines()

ResetLines resets the stack of transpiled lines

func (*Shell) Run added in v0.1.4 

func (sh *Shell) Run(cmd any, args ...any)

Run executes the given command string, waiting for the command to finish, handling the given arguments appropriately. If there is any error, it adds it to the shell, and triggers CancelExecution. It forwards output to exec.Config.Stdout and exec.Config.Stderr appropriately.

func (*Shell) RunBuiltinOrCommand added in v0.1.4 

func (sh *Shell) RunBuiltinOrCommand(cmdIO *exec.CmdIO, cmd string, args ...string) bool

RunBuiltinOrCommand runs a builtin or a command

func (*Shell) RunErrOK added in v0.1.4 

func (sh *Shell) RunErrOK(cmd any, args ...any)

RunErrOK executes the given command string, waiting for the command to finish, handling the given arguments appropriately. It does not stop execution if there is an error. If there is any error, it adds it to the shell. It forwards output to exec.Config.Stdout and exec.Config.Stderr appropriately.

func (*Shell) SSHByHost added in v0.1.4 

func (sh *Shell) SSHByHost(host string) (*sshclient.Client, error)

SSHByHost returns the SSH client for given host name, with err if not found

func (*Shell) Scp added in v0.1.4 

func (sh *Shell) Scp(cmdIO *exec.CmdIO, args ...string) error

Scp performs file copy over SSH connection, with the remote filename prefixed with the @name: and the local filename un-prefixed. The order is from -> to, as in standard cp. The remote filename is automatically relative to the current working directory on the remote host.

func (*Shell) Set added in v0.1.4 

func (sh *Shell) Set(cmdIO *exec.CmdIO, args ...string) error

Set sets the given environment variable to the given value.

func (*Shell) Start added in v0.1.4 

func (sh *Shell) Start(cmd any, args ...any)

Start starts the given command string for running in the background, handling the given arguments appropriately. If there is any error, it adds it to the shell. It forwards output to exec.Config.Stdout and exec.Config.Stderr appropriately.

func (*Shell) StartContext added in v0.1.4 

func (sh *Shell) StartContext() context.Context

StartContext starts a processing context, setting the Ctx and Cancel Fields. Call EndContext when current operation finishes.

func (*Shell) Tokens 

func (sh *Shell) Tokens(ln string) Tokens

Tokens converts the string into tokens

func (*Shell) TotalDepth 

func (sh *Shell) TotalDepth() int

TotalDepth returns the sum of any unresolved paren, brace, or bracket depths.

func (*Shell) TranspileCode 

func (sh *Shell) TranspileCode(code string)

TranspileCode processes each line of given code, adding the results to the LineStack

func (*Shell) TranspileConfig added in v0.1.4 

func (sh *Shell) TranspileConfig() error

TranspileConfig transpiles the .cosh startup config file in the user's home directory if it exists.

func (*Shell) TranspileExec 

func (sh *Shell) TranspileExec(ewords []string, output bool) Tokens

TranspileExec returns transpiled tokens assuming Exec code, with the given bools indicating the type of run to execute.

func (*Shell) TranspileExecString 

func (sh *Shell) TranspileExecString(str string, output bool) Tokens

TranspileExecString returns transpiled tokens assuming Exec code, from a backtick-encoded string, with the given bool indicating whether [Output] is needed.

func (*Shell) TranspileFile 

func (sh *Shell) TranspileFile(in string, out string) error

TranspileFile transpiles the given input cosh file to the given output Go file, adding package main and func main declarations.

func (*Shell) TranspileGo 

func (sh *Shell) TranspileGo(toks Tokens) Tokens

TranspileGo returns transpiled tokens assuming Go code. Unpacks any backtick encapsulated shell commands.

func (*Shell) TranspileLine 

func (sh *Shell) TranspileLine(ln string) string

TranspileLine is the main function for parsing a single line of shell input, returning a new transpiled line of code that converts Exec code into corresponding Go function calls.

func (*Shell) TranspileLineTokens 

func (sh *Shell) TranspileLineTokens(ln string) Tokens

TranspileLineTokens returns the tokens for the full line

type Token 

type Token struct {
	// Go token classification
	Tok token.Token

	// Literal string
	Str string

	// position in the original string.
	// this is only set for the original parse,
	// not for transpiled additions.
	Pos token.Pos
}

Token provides full data for one token

func NewToken 

func NewToken(tok token.Token, str ...string) *Token

NewToken returns a new token, for generated tokens without Pos

func (*Token) IsBacktickString 

func (tk *Token) IsBacktickString() bool

IsBacktickString returns true if the given STRING uses backticks

func (*Token) IsBracket 

func (tk *Token) IsBracket() bool

IsBracket returns true if the given token is a bracket delimiter: paren, brace, bracket

func (*Token) IsGo 

func (tk *Token) IsGo() bool

IsGo returns true if the given token is a Go Keyword or Comment

func (*Token) IsOp 

func (tk *Token) IsOp() bool

IsOp returns true if the given token is an operator

func (*Token) IsPathDelim added in v0.1.4 

func (tk *Token) IsPathDelim() bool

func (*Token) IsPathExtraDelim added in v0.1.4 

func (tk *Token) IsPathExtraDelim() bool

func (*Token) IsValidExecIdent added in v0.1.4 

func (tk *Token) IsValidExecIdent() bool

IsValidExecIdent returns true if the given token is a valid component of an Exec mode identifier

func (*Token) IsWord added in v0.1.4 

func (tk *Token) IsWord() bool

IsWord returns true if the token is some kind of word-like entity, including IDENT, STRING, CHAR, or one of the Go keywords. This is for exec filtering.

func (*Token) String 

func (tk *Token) String() string

String returns the string for the token

type Tokens 

type Tokens []*Token

Tokens is a slice of Token

func (*Tokens) Add 

func (tk *Tokens) Add(tok token.Token, str ...string) *Token

Add adds a new token, for generated tokens without Pos

func (*Tokens) AddTokens 

func (tk *Tokens) AddTokens(toks Tokens) *Tokens

AddTokens adds given tokens to our list

func (Tokens) BracketDepths 

func (tk Tokens) BracketDepths() (paren, brace, brack int)

BracketDepths returns the depths for the three bracket delimiters [paren, bracket, brace], based on unmatched right versions.

func (Tokens) Code 

func (tk Tokens) Code() string

Code returns concatenated Str values of the tokens, to generate a surface-valid code string.

func (Tokens) Contains added in v0.1.4 

func (tk Tokens) Contains(toks ...token.Token) bool

Contains returns true if the token string contains any of the given token(s)

func (*Tokens) DeleteLastComma 

func (tk *Tokens) DeleteLastComma()

DeleteLastComma removes any final Comma. easier to generate and delete at the end

func (*Tokens) Insert added in v0.1.4 

func (tk *Tokens) Insert(i int, tok token.Token, str ...string) *Token

Insert inserts a new token at given position

func (Tokens) Last 

func (tk Tokens) Last() *Token

Last returns the final token in the list

func (Tokens) Path added in v0.1.4 

func (tk Tokens) Path(idx0 bool) (string, int)

Path extracts a standard path or URL expression from the current list of tokens (starting at index 0), returning the path string and the number of tokens included in the path. Restricts processing to contiguous elements with no spaces! If it is not a path, returns nil string, 0

func (Tokens) ReplaceIdentAt added in v0.1.4 

func (tk Tokens) ReplaceIdentAt(at int, str string, n int) Tokens

ReplaceIdentAt replaces an identifier spanning n tokens starting at given index, with a single identifier with given string. This is used in Exec mode for dealing with identifiers and paths that are separately-parsed by Go.

func (Tokens) RightMatching 

func (tk Tokens) RightMatching() int

RightMatching returns the position (or -1 if not found) for the right matching [paren, bracket, brace] given the left one that is at the 0 position of the current set of tokens.

func (Tokens) String 

func (tk Tokens) String() string

String is the stringer version which includes the token ID in addition to the string literal

Source Files

View all Source files

Directories

Path Synopsis
cmd
cosh
Command cosh is an interactive cli for running and compiling Cogent Shell (cosh).
 Command cosh is an interactive cli for running and compiling Cogent Shell (cosh).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.