exec

package
v0.35.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 16, 2019 License: Apache-2.0 Imports: 17 Imported by: 0

README

Execution service

The execution service is responsible for opening, managing terminal session, with the ability to send command and extract data. Service keeps tracking the session current directory commands, env variable setting, and will only issue SSH command if there is actual change.

Endly service action integration

Run the following command for exec service operation details:


endly -s=exec

endly -s=exec -a=run
endly -s=exec -a=extract
endly -s=exec -a=open
endly -s=exec -a=close
Service Id Action Description Request Response
exec open open SSH session on the target resource. OpenSessionRequest OpenSessionResponse
exec close close SSH session CloseSessionRequest CloseSessionResponse
exec run execute basic commands RunRequest RunResponse
exec extract execute commands with ability to extract data, define error or success state ExtractRequest RunResponse

RunRequest example

@run.json

{
  "Target": {
    "URL": "ssh://127.0.0.1/",
    "Credentials": "${env.HOME}/.secret/localhost.json"
  },
  "SuperUser":true,
  "Commands":["mkdir /tmp/app1"]
}

@run.yaml

target:
  url:  ssh://127.0.0.1/
  credentials: ${env.HOME}/.secret/localhost.json
commands:
  - whoami
  - ${cmd[0].stdout}:/root/?  mkdir -p /tmp/app
  - ${cmd[0].stdout}:!/root/? mkdir ~/app
  - echo cmd[0].stdout
  - grep /etc/hosts  
extract:
  - key: aliases
    regExpr: (?sm).+(^.+)127.0.0.1
-

ExtractRequest example

{
	"Target": {
	  "URL": "ssh://127.0.0.1/",
	  "Credentials": "${env.HOME}/.secret/localhost.json"
	},
	"SystemPaths": ["/opt/sdk/go/bin"],
	"Commands": [
	  {
		"Command": "go version",
		"Extract": [
		  {
			"RegExpr": "go(\\d\\.\\d)",
			"Key": "Version"
		  }
		]
	  }
	]
}

SSH Session

In order to run any SSH command, service needs to open a session, it uses target.Credentials and secret service to connect the target host.

Opening session is an optional step, run or extract request will open session automatically.

By default session is open in non transient mode, which means once context.Close is called, session will be will be terminated. Otherwise caller is responsible for closing it.

    
        manager := endly.New()
        context := manager.NewContext(toolbox.NewContext())
        target := url.NewResource("ssh://127.0.0.1", "~/.secret/localhost.json")
        defer context.Close() // session closes as part of context.Close
        response, err := manager.Run(context, exec.NewOpenSessionRequest(target, []string{"/usr/local/bin"}, map[string]string{"M2_HOME":"/users/test/.m2/"},false, "/"))
        if err != nil {
            log.Fatal(err)
        }
        openResponse := response.(*exec.OpenSessionResponse)
        sessions :=context.TerminalSessions()
        assert.True(t,sessions.Has(openResponse.SessionID))
        log.Print(openResponse.SessionID)

Run vs Extract:

RunReuest provide a simple way to excute SSH command with conditional execution, it uses util.StdErrors as stdout errors. ExtractRequest has ability to fine tune SSH command execution with extraction data ability. Error matching in ExtractRequest does use any default value.

Run Commands

Command in RunRequest can represents one of the following:

  1. Simple command: i.e echo $HOME
  2. Conditional command: [criteria ?] command i.e. $stdout:/root/? echo 'hello root'",

    manager := endly.New()
    context := manager.NewContext(toolbox.NewContext())
    var target= url.NewResource("ssh://127.0.0.1", "localhost")
    var runRequest = exec.NewRunRequest(target, true, "whoami", "$stdout:/root/? echo 'hello root'")
    var runResponse = &exec.RunResponse{}
    err := endly.Run(context, runRequest, runResponse)

Run Command to Extract Data


    extractRequest := exec.NewExtractRequest(target,
		exec.DefaultOptions(),
		exec.NewExtractCommand(fmt.Sprintf("svn info"), "", nil, nil,
			endly.NewDataExtraction("origin", "^URL:[\\t\\s]+([^\\s]+)", false),
			endly.NewDataExtraction("revision", "Revision:\\s+([^\\s]+)", false)))
    manager := endly.New()
    context := manager.NewContext(toolbox.NewContext())
    var runResponse := &exec.RunResponse{}
    err := endly.Run(context, extractRequest, runResponse)
    if err != nil {
        log.Fatal(err)
    }

Exec SSH Unit Testing

This module provide SSH session recording ability to later replay it during unit testing without actual SSH involvement.

Recroding SSH session

To record actual SSH session use exec.OpenRecorderContext helper method, the last parameters specify location where conversation is recorded, actual dump takes place when context is closed (defer context.Clode()). If you use sudo. any secret or credentials make sure that you rename it to *** before checking in any code so you can use var credentials, err = util.GetDummyCredential()

	manager := endly.New()
	target := url.NewResource("ssh://127.0.0.1", "~/.secret/localhost.json")
	context, err :=  exec.NewSSHRecodingContext(manager, target, "test/session/context")
	if err != nil {
		log.Fatal(err)
	}
	defer context.Close()

Replaying SSH session

In order to replay previously recoded SSH session use exec.GetReplayService helper method to create a test SSHService, use location of stored SSH conversation as parameter, then create context with exec.OpenTestContext

	manager := endly.New()
	var credentials, err = util.GetDummyCredential()
	if err != nil {
		log.Fatal(err)
	}
	target := url.NewResource("ssh://127.0.0.1", credentials)
	context, err := exec.NewSSHReplayContext(manager, target, "test/session/transient")
	if err != nil {
		log.Fatal(err)
	}
	response, err := manager.Run(context, exec.NewOpenSessionRequest(target, []string{"/usr/local/bin"}, map[string]string{"M2_HOME": "/users/test/.m2/"}, false, "/"))
    if err != nil {
        log.Fatal(err)
    }

Documentation

Index

Constants

View Source 
const ServiceID = "exec"

ServiceID represent system executor service id

View Source 
const SudoCredentialKey = "**sudo**"

SudoCredentialKey represent obsucated password sudo credentials key (target.Credentials)

Variables

View Source 
var CommandErrors = []string{util.CommandNotFound, util.NoSuchFileOrDirectory, util.ErrorIsNotRecoverable}

Functions

func GetReplayService 

func GetReplayService(basedir string) (ssh.Service, error)

GetReplayService return replay service

func New 

func New() endly.Service

New creates a new execution service

func NewSSHMultiRecordingContext 

func NewSSHMultiRecordingContext(manager endly.Manager, sessions map[string]*url.Resource) (*endly.Context, error)

NewSSHMultiRecordingContext open multi recorded session

func NewSSHMultiReplayContext 

func NewSSHMultiReplayContext(manager endly.Manager, sessions map[string]*url.Resource) (*endly.Context, error)

OpenMultiSessionTestContext opens test context with multi SSH replay/mocks session

func NewSSHRecodingContext 

func NewSSHRecodingContext(manager endly.Manager, target *url.Resource, sessionDir string) (*endly.Context, error)

NewSSHRecodingContext open recorder context (to capture SSH command)

func NewSSHReplayContext 

func NewSSHReplayContext(manager endly.Manager, target *url.Resource, basedir string) (*endly.Context, error)

NewSSHReplayContext opens test context with SSH commands to replay

func OperatingSystem 

func OperatingSystem(context *endly.Context, sessionName string) *model.OperatingSystem

Os returns operating system for provide session

func TerminalSession 

func TerminalSession(context *endly.Context, target *url.Resource) (*model.Session, error)

TerminalSession returns Session for passed in target resource.

func TerminalSessions 

func TerminalSessions(context *endly.Context) model.Sessions

TerminalSessions returns system sessions

Types

type CloseSessionRequest 

type CloseSessionRequest struct {
	SessionID string
}

CloseSessionRequest closes session

type CloseSessionResponse 

type CloseSessionResponse struct {
	SessionID string
}

CloseSessionResponse closes session response

type Command 

type Command string

Command represents a command expression: [when criteria ?] command

func (Command) String 

func (c Command) String() string

String returns command string

func (Command) WhenAndCommand 

func (c Command) WhenAndCommand() (string, string)

WhenAndCommand extract when criteria and command

type ExtractCommand 

type ExtractCommand struct {
	When    string         `description:"only run this command is criteria is matched i.e $stdout:/password/"`                                           //only run this execution is output from a previous command is matched
	Command string         `required:"true" description:"shell command to be executed"`                                                                  //command to be executed
	Extract model.Extracts `description:"stdout data extraction instruction"`                                                                            //Stdout data extraction instruction
	Errors  []string       `description:"fragments that will terminate execution with error if matched with standard output, in most cases leave empty"` //fragments that will terminate execution with error if matched with standard output
	Success []string       ``                                                                                                                            //if specified absence of all of the these fragment will terminate execution with error.
	/* 126-byte string literal not displayed */
}

Extracts represents an execution instructions

func NewExtractCommand 

func NewExtractCommand(command, when string, success, errors []string, extractions ...*model.Extract) *ExtractCommand

NewExtractCommand creates a new extract command

func (*ExtractCommand) Init 

func (c *ExtractCommand) Init() error

type ExtractRequest 

type ExtractRequest struct {
	Target *url.Resource `required:"true" description:"host where command runs" ` //execution target - destination where to run a command.
	*Options
	Commands []*ExtractCommand `description:"command with data extraction instruction "` //extract command
}

ExtractRequest represents managed command request

func NewExtractRequest 

func NewExtractRequest(target *url.Resource, options *Options, commands ...*ExtractCommand) *ExtractRequest

NewExtractRequest returns a new command request

func NewExtractRequestFromURL 

func NewExtractRequestFromURL(URL string) (*ExtractRequest, error)

NewExtractRequestFromURL creates a new request from URL

func (*ExtractRequest) Clone 

func (r *ExtractRequest) Clone(target *url.Resource) *ExtractRequest

Clones clones requst with supplide target

func (*ExtractRequest) Init 

func (r *ExtractRequest) Init() error

Init initialises request

func (*ExtractRequest) Validate 

func (r *ExtractRequest) Validate() error

Validate validates managed command request

type Log 

type Log struct {
	Stdin  string
	Stdout string
	Error  string
}

Log represents an executed command with Stdin, Stdout or Error

func NewCommandLog 

func NewCommandLog(stdin, stdout string, err error) *Log

NewCommandLog creates a new command log

type OpenSessionRequest 

type OpenSessionRequest struct {
	Target        *url.Resource      //Session is created from target host (servername, port)
	Config        *ssh.SessionConfig //ssh configuration
	SystemPaths   []string           //system path that are applied to the ssh session
	Env           map[string]string
	Transient     bool        //if this flag is true, caller is responsible for closing session, othewise session is closed as context is closed
	Basedir       string      //capture all ssh service command in supplied dir (for unit test only)
	ReplayService ssh.Service //use Ssh ReplayService instead of actual SSH service (for unit test only)
}

OpenSessionRequest represents an open session request.

func NewOpenSessionRequest 

func NewOpenSessionRequest(target *url.Resource, systemPaths []string, env map[string]string, transient bool, basedir string) *OpenSessionRequest

NewOpenSessionRequest creates a new session if transient flag is true, caller is responsible for closing session, otherwise session is closed as context is closed

func (*OpenSessionRequest) Validate 

func (r *OpenSessionRequest) Validate() error

Validate checks if request is valid

type OpenSessionResponse 

type OpenSessionResponse struct {
	SessionID string
}

OpenSessionResponse represents a session id

type Options 

type Options struct {
	SystemPaths []string `description:"path that will be appended to the current SSH execution session the current and future commands"` //path that will be added to the system paths
	Terminators []string ``                                                                                                              //fragment that helps identify that command has been completed - the best is to leave it empty, which is the detected bash prompt
	/* 141-byte string literal not displayed */
	Errors    []string          `description:"fragments that will terminate execution with error if matched with standard output, in most cases leave empty"` //fragments that will terminate execution with error if matched with standard output
	TimeoutMs int               `description:"time after command was issued for waiting for command output if expect fragment were not matched"`              //time after command was issued for waiting for command output if expect fragment were not matched.
	Directory string            `description:"directory where this command should start - if does not exists there is no exception"`                          //directory where command should run
	Env       map[string]string `description:"environment variables to be set before command runs"`                                                           //environment variables to be set before command runs
	SuperUser bool              ``                                                                                                                            ///flag to run it as super user
	/* 156-byte string literal not displayed */
	Secrets secret.Secrets `description:"secrets map see https://github.com/viant/toolbox/tree/master/secret"`
}

Options represents an execution options

func DefaultOptions 

func DefaultOptions() *Options

DefaultOptions creates a default execution options

func NewOptions 

func NewOptions(secrets, env map[string]string, terminators, path []string, superUser bool) *Options

type RunRequest 

type RunRequest struct {
	Target *url.Resource `required:"true" description:"host where command runs" ` //execution target - destination where to run a command.
	*Options
	Commands []Command      `required:"true" description:"command list" `      //list of commands to run
	Extract  model.Extracts `description:"stdout data extraction instruction"` //Stdout data extraction instruction
}

RunRequest represents a simple command

func NewRunRequest 

func NewRunRequest(target *url.Resource, superUser bool, commands ...string) *RunRequest

NewRunRequest creates a new request

func NewRunRequestFromURL 

func NewRunRequestFromURL(URL string) (*RunRequest, error)

NewExtractRequestFromURL creates a new request from URL

func (*RunRequest) AsExtractRequest 

func (r *RunRequest) AsExtractRequest() *ExtractRequest

AsExtractRequest returns ExtractRequest for this requests

func (*RunRequest) Init 

func (r *RunRequest) Init() error

Init initialises request

func (*RunRequest) Validate 

func (r *RunRequest) Validate() error

Validate validates managed command request

type RunResponse 

type RunResponse struct {
	Session string
	Cmd     []*Log
	Output  string
	Data    data.Map
	Error   string
}

RunResponse represents a command response with logged commands.

func NewRunResponse 

func NewRunResponse(session string) *RunResponse

NewRunResponse creates a new RunResponse

func (*RunResponse) Add 

func (i *RunResponse) Add(log *Log)

Add appends provided log into commands slice.

func (*RunResponse) Stdout 

func (i *RunResponse) Stdout(indexes ...int) string

Stdout returns stdout for provided index, or all concatenated otherwise

type StdinEvent 

type StdinEvent struct {
	SessionID string
	Stdin     string
}

StdinEvent represents an execution event start

func NewSdtinEvent 

func NewSdtinEvent(sessionID string, stdin string) *StdinEvent

NewSdtinEvent crates a new execution start event value

func (*StdinEvent) Messages 

func (e *StdinEvent) Messages() []*msg.Message

Messages returns messages

type StdoutEvent 

type StdoutEvent struct {
	SessionID string
	Stdout    string
	Error     string
}

StdoutEvent represents an execution event end

func NewStdoutEvent 

func NewStdoutEvent(sessionID string, stdout string, err error) *StdoutEvent

NewStdoutEvent crates a new execution start event value

func (*StdoutEvent) Messages 

func (e *StdoutEvent) Messages() []*msg.Message

Messages returns messages

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.