Copyright(c) Ian Duncan 2014
Safe HaskellNone




Mesos scheduler interface and scheduler driver. A scheduler is used to interact with Mesos in order run distributed computations.


Creating a new Scheduler

class ToScheduler a where Source

Callback interface to be implemented by frameworks' schedulers. Note that only one callback will be invoked at a time, so it is not recommended that you block within a callback because it may cause a deadlock.

Minimal complete definition



registered :: a -> SchedulerDriver -> FrameworkID -> MasterInfo -> IO () Source

Invoked when the Scheduler successfully registers with a Mesos master. A unique ID (generated by the master) used for distinguishing this framework from others and MasterInfo with the ip and port of the current master are provided as arguments.

reRegistered :: a -> SchedulerDriver -> MasterInfo -> IO () Source

Invoked when the Scheduler re-registers with a newly elected Mesos master. This is only called when the scheduler has previously been registered. MasterInfo containing the updated information about the elected master is provided as an argument.

disconnected :: a -> SchedulerDriver -> IO () Source

Invoked when the Scheduler becomes "disconnected" from the master (e.g., the master fails and another is taking over).

resourceOffers :: a -> SchedulerDriver -> [Offer] -> IO () Source

Invoked when resources have been offered to this framework. A single offer will only contain resources from a single slave. Resources associated with an offer will not be re-offered to _this_ framework until either (a) this framework has rejected those resources (see launchTasks) or (b) those resources have been rescinded (see offerRescinded). Note that resources may be concurrently offered to more than one framework at a time (depending on the allocator being used). In that case, the first framework to launch tasks using those resources will be able to use them while the other frameworks will have those resources rescinded (or if a framework has already launched tasks with those resources then those tasks will fail with a Lost status and a message saying as much).

offerRescinded :: a -> SchedulerDriver -> OfferID -> IO () Source

Invoked when an offer is no longer valid (e.g., the slave was lost or another framework used resources in the offer). If for whatever reason an offer is never rescinded (e.g., dropped message, failing over framework, etc.), a framwork that attempts to launch tasks using an invalid offer will receive Lost status updates for those tasks (see resourceOffers).

statusUpdate :: a -> SchedulerDriver -> TaskStatus -> IO () Source

Invoked when the status of a task has changed (e.g., a slave is lost and so the task is lost, a task finishes and an executor sends a status update saying so, etc). Note that returning from this callback _acknowledges_ receipt of this status update! If for whatever reason the scheduler aborts during this callback (or the process exits) another status update will be delivered (note, however, that this is currently not true if the slave sending the status update is lost/fails during that time).

frameworkMessage :: a -> SchedulerDriver -> ExecutorID -> SlaveID -> ByteString -> IO () Source

Invoked when an executor sends a message. These messages are best effort; do not expect a framework message to be retransmitted in any reliable fashion.

slaveLost :: a -> SchedulerDriver -> SlaveID -> IO () Source

Invoked when a slave has been determined unreachable (e.g., machine failure, network partition). Most frameworks will need to reschedule any tasks launched on this slave on a new slave.

executorLost :: a -> SchedulerDriver -> ExecutorID -> SlaveID -> Status -> IO () Source

Invoked when an executor has exited/terminated. Note that any tasks running will have Lost status updates automagically generated.

errorMessage :: a -> SchedulerDriver -> ByteString -> IO () Source

Invoked when there is an unrecoverable error in the scheduler or scheduler driver. The driver will be aborted BEFORE invoking this callback.

data SchedulerDriver Source

Type representing the connection from a scheduler to Mesos. This handle is used both to manage the scheduler's lifecycle (start it, stop it, or wait for it to finish) and to interact with Mesos (e.g., launch tasks, kill tasks, etc.).

Managing framework actions with a SchedulerDriver

SchedulerDriver lifecycle management

withSchedulerDriver Source


:: ToScheduler a 
=> a 
-> FrameworkInfo 
-> ByteString

ip:port of the master to connect to. For example: ""

-> Maybe Credential 
-> (SchedulerDriver -> IO b) 
-> IO b 

start :: SchedulerDriver -> IO Status Source

Starts the scheduler driver. This needs to be called before any other driver calls are made.

stop Source


:: SchedulerDriver 
-> Bool

should failover?

-> IO Status 

Stops the scheduler driver. If the failover flag is set to false then it is expected that this framework will never reconnect to Mesos and all of its executors and tasks can be terminated. Otherwise, all executors and tasks will remain running (for some framework specific failover timeout) allowing the scheduler to reconnect (possibly in the same process, or from a different process, for example, on a different machine).

abort :: SchedulerDriver -> IO Status Source

Aborts the driver so that no more callbacks can be made to the scheduler. The semantics of abort and stop have deliberately been separated so that code can detect an aborted driver (i.e., via the return status of await, see below), and instantiate and start another driver if desired (from within the same process). Note that stop is not automatically called inside abort.

await :: SchedulerDriver -> IO Status Source

Waits for the driver to be stopped or aborted, possibly *blocking the current thread indefinitely*. The return status of this function can be used to determine if the driver was aborted (see Status for more information).

run :: SchedulerDriver -> IO Status Source

Starts and immediately awaits (blocks on) the driver.

Managing and interacting with tasks and resources

requestResources :: SchedulerDriver -> [Request] -> IO Status Source

Requests resources from Mesos. Any resources available are asynchronously offered to the framework via the resourceOffers callback.

launchTasks :: SchedulerDriver -> [OfferID] -> [TaskInfo] -> Filters -> IO Status Source

Launches the given set of tasks. Any resources remaining (i.e., not used by the tasks or their executors) will be considered declined. The specified filters are applied on all unused resources (see Filters for more information).

Available resources are aggregated when mutiple offers are provided. Note that all offers must belong to the same slave. Invoking this function with an empty collection of tasks declines offers in their entirety (see declineOffer).

killTask :: SchedulerDriver -> TaskID -> IO Status Source

Kills the specified task. Note that attempting to kill a task is currently not reliable. If, for example, a scheduler fails over while it was attempting to kill a task it will need to retry in the future. Likewise, if unregistered / disconnected, the request will be dropped (these semantics may be changed in the future).

declineOffer :: SchedulerDriver -> OfferID -> Filters -> IO Status Source

Declines an offer in its entirety and applies the specified filters on the resources (see Filters). Note that this can be done at any time, it is not necessary to do this within the resourceOffers callback.

reviveOffers :: SchedulerDriver -> IO Status Source

Removes all filters previously set by the framework (via launchTasks). This enables the framework to receive offers from those filtered slaves.

sendFrameworkMessage :: SchedulerDriver -> ExecutorID -> SlaveID -> ByteString -> IO Status Source

Sends a message from the framework to one of its executors. These messages are best effort; do not expect a framework message to be retransmitted in any reliable fashion.

reconcileTasks :: SchedulerDriver -> [TaskStatus] -> IO Status Source

Reconciliation of tasks causes the master to send status updates for tasks whose status differs from the status sent here.

Low-level lifecycle management