executor

Documentation

Overview

Package executor xxx

Index

• Constants
• type BcsExecutorDriver
  • func (driver *BcsExecutorDriver) Abort() (mesos.Status, error)
  • func (driver *BcsExecutorDriver) ExecutorID() string
  • func (driver *BcsExecutorDriver) IsRunning() bool
  • func (driver *BcsExecutorDriver) Join() (mesos.Status, error)
  • func (driver *BcsExecutorDriver) Run() (mesos.Status, error)
  • func (driver *BcsExecutorDriver) SendFrameworkMessage(data string) (mesos.Status, error)
  • func (driver *BcsExecutorDriver) SendStatusUpdate(taskStatus *mesos.TaskStatus) (mesos.Status, error)
  • func (driver *BcsExecutorDriver) Start() (mesos.Status, error)
  • func (driver *BcsExecutorDriver) Status() mesos.Status
  • func (driver *BcsExecutorDriver) Stop() (mesos.Status, error)
• type DriverConfig
• type DriverEnv
  • func (ee *DriverEnv) GetAllEnvs() error
• type Executor
• type ExecutorDriver
  • func NewExecutorDriver(bcsExe Executor) ExecutorDriver

Constants

const (
	// DefaultMetricsTextFile xxx
	DefaultMetricsTextFile = "/data/bcs/export_data"
)

Types

type BcsExecutorDriver

type BcsExecutorDriver struct {
	// contains filtered or unexported fields
}

BcsExecutorDriver BCS implementation for ExecutorDriver

func (*BcsExecutorDriver) Abort

func (driver *BcsExecutorDriver) Abort() (mesos.Status, error)

Abort the driver so that no more callbacks can be made to the executor. The semantics of abort and stop have deliberately been separated so that code can detect an aborted driver (i.e., via the return status of ExecutorDriver.Join, see below), and instantiate and start another driver if desired (from within the same process ... although this functionality is currently not supported for executors).

func (*BcsExecutorDriver) ExecutorID

func (driver *BcsExecutorDriver) ExecutorID() string

ExecutorID get ExecutorID from mesos slave

func (*BcsExecutorDriver) IsRunning

func (driver *BcsExecutorDriver) IsRunning() bool

IsRunning check driver is running

func (*BcsExecutorDriver) Join

func (driver *BcsExecutorDriver) Join() (mesos.Status, error)

Join Waits for the driver to be stopped or aborted, possibly blocking the calling goroutine indefinitely. The return status of this function can be used to determine if the driver was aborted (see package mesos for a description of Status).

func (*BcsExecutorDriver) Run

func (driver *BcsExecutorDriver) Run() (mesos.Status, error)

Run Starts and immediately joins (i.e., blocks on) the driver.

func (*BcsExecutorDriver) SendFrameworkMessage

func (driver *BcsExecutorDriver) SendFrameworkMessage(data string) (mesos.Status, error)

SendFrameworkMessage send a message to the framework scheduler. These messages are best effort; do not expect a framework message to be retransmitted in any reliable fashion.

func (*BcsExecutorDriver) SendStatusUpdate

func (driver *BcsExecutorDriver) SendStatusUpdate(taskStatus *mesos.TaskStatus) (mesos.Status, error)

SendStatusUpdate a status update to the framework scheduler, retrying as necessary until an acknowledgement has been received or the executor is terminated (in which case, a TASK_LOST status update will be sent). See Scheduler.StatusUpdate for more information about status update acknowledgements.

func (*BcsExecutorDriver) Start

func (driver *BcsExecutorDriver) Start() (mesos.Status, error)

Start the executor driver. This needs to be called before any other driver calls are made.

func (*BcsExecutorDriver) Status

func (driver *BcsExecutorDriver) Status() mesos.Status

Status return driver status

func (*BcsExecutorDriver) Stop

func (driver *BcsExecutorDriver) Stop() (mesos.Status, error)

Stop the executor driver. executor will exited

type DriverConfig

type DriverConfig struct {
	Executor Executor // Executor interface
}

DriverConfig hold all custom info for ExecutorDriver

type DriverEnv

type DriverEnv struct {
	MesosSlavePID            string // agent slave pid
	MesosSlaveID             string // agent slave uniq id
	MesosAgentEndpoint       string // agent ip:port endpoint to connect to the agent
	MesosFrameworkID         string // frameworkid from agent
	MesosExecutorID          string // exector id from agent
	SSLEnabled               bool   // true is agent enable https
	MesosSandBox             string // Path to the mapped sandbox inside of the container
	MesosCheckpoint          bool   // If set to true, denotes that framework has checkpointing enabled
	MesosRecoveryTimeout     int    // The total duration that the executor should spend retrying before shutting it self down when it is disconnected from the agent
	MesosSubscriptionBackoff int    // The maximum backoff duration between two retries when disconnected
	MesosShutdownGracePeriod int    // Amount of time the agent would wait for an executor to shut down (e.g., 60secs, 3mins etc.) after sending a SHUTDOWN event
}

DriverEnv The following environment variables are set by the agent that can be

used by the executor upon startup:

MESOS_FRAMEWORK_ID: FrameworkID of the scheduler needed as part of the SUBSCRIBE call. MESOS_EXECUTOR_ID: ExecutorID of the executor needed as part of the SUBSCRIBE call. MESOS_DIRECTORY: Path to the working directory for the executor on the host filesystem(deprecated). MESOS_SANDBOX: Path to the mapped sandbox inside of the container (determined by the

agent flag sandbox_directory) for either mesos container with image or docker container.
For the case of command task without image specified, it is the path to the sandbox
on the host filesystem, which is identical to MESOS_DIRECTORY. MESOS_DIRECTORY
is always the sandbox on the host filesystem.

MESOS_AGENT_ENDPOINT: agent endpoint i.e. ip:port to be used by the executor to connect

to the agent.

MESOS_CHECKPOINT: If set to true, denotes that framework has checkpointing enabled. MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD: Amount of time the agent would wait for an

executor to shut down (e.g., 60secs, 3mins etc.) after sending a SHUTDOWN event.

If MESOS_CHECKPOINT is set i.e. when framework checkpointing is enabled, the following

additional variables are also set that can be used by the executor for retrying
upon a disconnection with the agent:

MESOS_RECOVERY_TIMEOUT: The total duration that the executor should spend retrying

before shutting itself down when it is disconnected from the agent (e.g., 15mins,
5secs etc.). This is configurable at agent startup via the flag --recovery_timeout.

MESOS_SUBSCRIPTION_BACKOFF_MAX: The maximum backoff duration to be used by the executor

between two retries when disconnected (e.g., 250ms, 1mins etc.). This is configurable
at agent startup via the flag --executor_reregistration_timeout.

func (*DriverEnv) GetAllEnvs

func (ee *DriverEnv) GetAllEnvs() error

GetAllEnvs get all info from environment

type Executor

type Executor interface {
	// Registered TODO
	/**
	 * Invoked once the executor driver has been able to successfully
	 * connect with Mesos. In particular, a scheduler can pass some
	 * data to its executors through the FrameworkInfo.ExecutorInfo's
	 * data field.
	 */
	Registered(ExecutorDriver, *mesos.ExecutorInfo, *mesos.FrameworkInfo, *mesos.AgentInfo)

	// Reregistered TODO
	/**
	 * Invoked when the executor re-registers with a restarted slave.
	 */
	Reregistered(ExecutorDriver, *mesos.AgentInfo)

	// Disconnected TODO
	/**
	 * Invoked when the executor becomes "disconnected" from the slave
	 * (e.g., the slave is being restarted due to an upgrade).
	 */
	Disconnected(ExecutorDriver)

	// LaunchTask TODO
	/**
	 * Invoked when a task has been launched on this executor (initiated
	 * via SchedulerDriver.LaunchTasks). Note that this task can be realized
	 * with a goroutine, an external process, or some simple computation, however,
	 * no other callbacks will be invoked on this executor until this
	 * callback has returned.
	 */
	LaunchTask(ExecutorDriver, *mesos.TaskInfo)

	// LaunchTaskGroup TODO
	/**
	 * Invoked when a task has been launched on this executor (initiated
	 * via SchedulerDriver.LaunchTasks). Note that this task can be realized
	 * with a goroutine, an external process, or some simple computation, however,
	 * no other callbacks will be invoked on this executor until this
	 * callback has returned.
	 */
	LaunchTaskGroup(ExecutorDriver, *mesos.TaskGroupInfo)

	// KillTask TODO
	/**
	 * Invoked when a task running within this executor has been killed
	 * (via SchedulerDriver.KillTask). Note that no status update will
	 * be sent on behalf of the executor, the executor is responsible
	 * for creating a new TaskStatus (i.e., with TASK_KILLED) and
	 * invoking ExecutorDriver.SendStatusUpdate.
	 */
	KillTask(ExecutorDriver, *mesos.TaskID)

	// FrameworkMessage TODO
	/**
	 * Invoked when a framework message has arrived for this
	 * executor. These messages are best effort; do not expect a
	 * framework message to be retransmitted in any reliable fashion.
	 */
	FrameworkMessage(ExecutorDriver, string)

	// Shutdown TODO
	/**
	 * Invoked when the executor should terminate all of its currently
	 * running tasks. Note that after Mesos has determined that an
	 * executor has terminated, any tasks that the executor did not send
	 * terminal status updates for (e.g., TASK_KILLED, TASK_FINISHED,
	 * TASK_FAILED, etc) a TASK_LOST status update will be created.
	 */
	Shutdown(ExecutorDriver)

	// Error TODO
	/**
	 * Invoked when a fatal error has occured with the executor and/or
	 * executor driver. The driver will be aborted BEFORE invoking this
	 * callback.
	 */
	Error(ExecutorDriver, string)

	// SetDriver TODO
	/**
	 * driver injection
	 */
	SetDriver(driver ExecutorDriver)
}

Executor callback interface to be implemented by frameworks' executors. Note that only one callback will be invoked at a time, so it is not recommended that you block within a callback because it may cause a deadlock.

Each callback includes an instance to the executor driver that was used to run this executor. The driver will not change for the duration of an executor (i.e., from the point you do ExecutorDriver.Start() to the point that ExecutorDriver.Join() returns). This is intended for convenience so that an executor doesn't need to store a pointer to the driver itself.

type ExecutorDriver

type ExecutorDriver interface {
	// Start TODO
	/**
	 * Starts the executor driver. This needs to be called before any
	 * other driver calls are made.
	 */
	Start() (mesos.Status, error)

	// Stop TODO
	/**
	 * Stops the executor driver.
	 */
	Stop() (mesos.Status, error)

	// Abort TODO
	/**
	 * Aborts the driver so that no more callbacks can be made to the
	 * executor. The semantics of abort and stop have deliberately been
	 * separated so that code can detect an aborted driver (i.e., via
	 * the return status of ExecutorDriver.Join, see below), and
	 * instantiate and start another driver if desired (from within the
	 * same process ... although this functionality is currently not
	 * supported for executors).
	 */
	Abort() (mesos.Status, error)

	// Join TODO
	/**
	 * Waits for the driver to be stopped or aborted, possibly
	 * blocking the calling goroutine indefinitely. The return status of
	 * this function can be used to determine if the driver was aborted
	 * (see package mesos for a description of Status).
	 */
	Join() (mesos.Status, error)

	// Run TODO
	/**
	 * Starts and immediately joins (i.e., blocks on) the driver.
	 */
	Run() (mesos.Status, error)

	// SendStatusUpdate TODO
	/**
	 * Sends a status update to the framework scheduler, retrying as
	 * necessary until an acknowledgement has been received or the
	 * executor is terminated (in which case, a TASK_LOST status update
	 * will be sent). See Scheduler.StatusUpdate for more information
	 * about status update acknowledgements.
	 */
	SendStatusUpdate(*mesos.TaskStatus) (mesos.Status, error)

	// SendFrameworkMessage TODO
	/**
	 * Sends a message to the framework scheduler. These messages are
	 * best effort; do not expect a framework message to be
	 * retransmitted in any reliable fashion.
	 */
	SendFrameworkMessage(string) (mesos.Status, error)

	// Status return ExecutorDriver Status
	Status() mesos.Status

	// ExecutorID get ExecutorID from mesos slave
	ExecutorID() string
}

ExecutorDriver interface for connecting an executor to Mesos. This interface is used both to manage the executor's lifecycle (start it, stop it, or wait for it to finish) and to interact with Mesos (e.g., send status updates, send framework messages, etc.). A driver method is expected to fail-fast and return an error when possible. Other internal errors (or remote error) that occur asynchronously are handled using the the Executor.Error() callback.

func NewExecutorDriver

func NewExecutorDriver(bcsExe Executor) ExecutorDriver

NewExecutorDriver create BcsExecutorDriver with ExecutorConfig