System.Mesos.Executor

Mesos executor interface and executor driver. An executor is responsible for launching tasks in a framework specific way (i.e., creating new threads, new processes, etc). One or more executors from the same framework may run concurrently on the same machine. Note that we use the term "executor" fairly loosely to refer to the code that implements an instance of the ToExecutor type class (see below) as well as the program that is responsible for instantiating a new ExecutorDriver (also below).

In fact, while a Mesos slave is responsible for (forking and) executing the "executor", there is no reason why whatever the slave executed might itself actually execute another program which actually instantiates and runs the SchedulerDriver. The only contract with the slave is that the program that it invokes does not exit until the "executor" has completed. Thus, what the slave executes may be nothing more than a script which actually executes (or forks and waits) the "real" executor.

Implementing an executor

class ToExecutor a where

Callback interface to be implemented by frameworks' executors. Note that only one callback will be invoked at a time, so it is not recommended that you block within a callback because it may cause a deadlock.

Minimal complete definition

Nothing

Methods

registered :: a -> ExecutorDriver -> ExecutorInfo -> FrameworkInfo -> SlaveInfo -> IO ()

Invoked once the executor driver has been able to successfully connect with Mesos.

reRegistered :: a -> ExecutorDriver -> SlaveInfo -> IO ()

Invoked when the executor re-registers with a restarted slave.

disconnected :: a -> ExecutorDriver -> IO ()

Invoked when the executor becomes "disconnected" from the slave (e.g., the slave is being restarted due to an upgrade).

launchTask :: a -> ExecutorDriver -> TaskInfo -> IO ()

Invoked when a task has been launched on this executor (initiated via launchTasks). Note that this task can be realized with a thread, a process, or some simple computation, however, no other callbacks will be invoked on this executor until this callback has returned.

taskKilled :: a -> ExecutorDriver -> TaskID -> IO ()

Invoked when a task running within this executor has been killed (via killTask). Note that no status update will be sent on behalf of the executor, the executor is responsible for creating a new TaskStatus (i.e., with Killed) and invoking sendStatusUpdate.

frameworkMessage :: a -> ExecutorDriver -> ByteString -> IO ()

Invoked when a framework message has arrived for this executor. These messages are best effort; do not expect a framework message to be retransmitted in any reliable fashion.

shutdown :: a -> ExecutorDriver -> IO ()

Invoked when the executor should terminate all of its currently running tasks. Note that after a Mesos has determined that an executor has terminated any tasks that the executor did not send terminal status updates for (e.g., Killed, Finished, Failed, etc.) a Lost status update will be created.

errorMessage

Arguments

:: a

-> ExecutorDriver

-> ByteString

error message

-> IO ()

Invoked when a fatal error has occured with the executor and/or executor driver. The driver will be aborted *before* invoking this callback.

Creating an executor

data ExecutorDriver

A handle that allows an Executor to trigger lifecycle & status update events (e.g. starting & stopping the executor and sending messages to the Scheduler that invoked the executor).

withExecutorDriver :: ToExecutor a => a -> (ExecutorDriver -> IO b) -> IO b

Interacting with Mesos

start :: ExecutorDriver -> IO Status

Starts the executor driver. This needs to be called before any other driver calls are made.

stop :: ExecutorDriver -> IO Status

Stops the ExecutorDriver.

abort :: ExecutorDriver -> IO Status

Aborts the driver so that no more callbacks can be made to the executor. The semantics of abort and stop have deliberately been separated so that code can detect an aborted driver (i.e., via the return status of abort, see below), and instantiate and start another driver if desired (from within the same process ... although this functionality is currently not supported for executors).

await :: ExecutorDriver -> IO Status

Waits for the driver to be stopped or aborted, possibly *blocking* the current thread indefinitely. The return status of this function can be used to determine if the driver was aborted (see mesos.proto for a description of Status).

run :: ExecutorDriver -> IO Status

starts and immediately awaits (i.e., blocks on) the driver.

sendStatusUpdate :: ExecutorDriver -> TaskStatus -> IO Status

Sends a status update to the framework scheduler, retrying as necessary until an acknowledgement has been received or the executor is terminated (in which case, a Lost status update will be sent). See statusUpdate for more information about status update acknowledgements.

sendFrameworkMessage

Arguments

:: ExecutorDriver

-> ByteString

message

-> IO Status

Sends a message to the framework scheduler. These messages are best effort; do not expect a framework message to be retransmitted in any reliable fashion.

Primitive executor management

data Executor

A data structure of the underlying executor & the callbacks that are triggered via the Mesos C++ API.

createExecutor :: ToExecutor a => a -> IO Executor

destroyExecutor :: Executor -> IO ()

withExecutor :: ToExecutor a => a -> (Executor -> IO b) -> IO b

createDriver :: Executor -> IO ExecutorDriver

destroyDriver :: ExecutorDriver -> IO ()