Amazonka.EMR.AddJobFlowSteps

AddJobFlowSteps adds new steps to a running cluster. A maximum of 256 steps are allowed in each job flow.

If your cluster is long-running (such as a Hive data warehouse) or complex, you may require more than 256 steps to process your data. You can bypass the 256-step limitation in various ways, including using SSH to connect to the master node and submitting queries directly to the software running on the master node, such as Hive and Hadoop.

A step specifies the location of a JAR file stored either on the master node of the cluster or in Amazon S3. Each step is performed by the main function of the main class of the JAR file. The main class can be specified either in the manifest of the JAR or by using the MainFunction parameter of the step.

Amazon EMR executes each step in the order listed. For a step to be considered complete, the main function must exit with a zero exit code and all Hadoop jobs started while the step was running must have completed and run successfully.

You can only add steps to a cluster that is in one of the following states: STARTING, BOOTSTRAPPING, RUNNING, or WAITING.

The string values passed into HadoopJarStep object cannot exceed a total of 10240 characters.

Creating a Request

data AddJobFlowSteps

The input argument to the AddJobFlowSteps operation.

See: newAddJobFlowSteps smart constructor.

Constructors

AddJobFlowSteps'

Fields

• executionRoleArn :: Maybe Text

  The Amazon Resource Name (ARN) of the runtime role for a step on the cluster. The runtime role can be a cross-account IAM role. The runtime role ARN is a combination of account ID, role name, and role type using the following format: arn:partition:service:region:account:resource.

  For example, arn:aws:iam::1234567890:role/ReadOnly is a correctly formatted runtime role ARN.

• jobFlowId :: Text

  A string that uniquely identifies the job flow. This identifier is returned by RunJobFlow and can also be obtained from ListClusters.

• steps :: [StepConfig]

  A list of StepConfig to be executed by the job flow.

Instances

ToJSON AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

toJSON :: AddJobFlowSteps -> Value

toEncoding :: AddJobFlowSteps -> Encoding

toJSONList :: [AddJobFlowSteps] -> Value

toEncodingList :: [AddJobFlowSteps] -> Encoding

ToHeaders AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

toHeaders :: AddJobFlowSteps -> [Header]

ToPath AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

toPath :: AddJobFlowSteps -> ByteString

ToQuery AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

toQuery :: AddJobFlowSteps -> QueryString

AWSRequest AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Associated Types

type AWSResponse AddJobFlowSteps

Methods

request :: (Service -> Service) -> AddJobFlowSteps -> Request AddJobFlowSteps

response :: MonadResource m => (ByteStringLazy -> IO ByteStringLazy) -> Service -> Proxy AddJobFlowSteps -> ClientResponse ClientBody -> m (Either Error (ClientResponse (AWSResponse AddJobFlowSteps)))

Generic AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Associated Types

type Rep AddJobFlowSteps :: Type -> Type

Methods

from :: AddJobFlowSteps -> Rep AddJobFlowSteps x

to :: Rep AddJobFlowSteps x -> AddJobFlowSteps

Read AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

readsPrec :: Int -> ReadS AddJobFlowSteps

readList :: ReadS [AddJobFlowSteps]

readPrec :: ReadPrec AddJobFlowSteps

readListPrec :: ReadPrec [AddJobFlowSteps]

Show AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

showsPrec :: Int -> AddJobFlowSteps -> ShowS

show :: AddJobFlowSteps -> String

showList :: [AddJobFlowSteps] -> ShowS

NFData AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

rnf :: AddJobFlowSteps -> ()

Eq AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

(==) :: AddJobFlowSteps -> AddJobFlowSteps -> Bool

(/=) :: AddJobFlowSteps -> AddJobFlowSteps -> Bool

Hashable AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

hashWithSalt :: Int -> AddJobFlowSteps -> Int

hash :: AddJobFlowSteps -> Int

type AWSResponse AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

type AWSResponse AddJobFlowSteps = AddJobFlowStepsResponse

type Rep AddJobFlowSteps

Defined in Amazonka.EMR.AddJobFlowSteps

type Rep AddJobFlowSteps = D1 ('MetaData "AddJobFlowSteps" "Amazonka.EMR.AddJobFlowSteps" "amazonka-emr-2.0-B3cWhP6sdymF3IBUMdPGcd" 'False) (C1 ('MetaCons "AddJobFlowSteps'" 'PrefixI 'True) (S1 ('MetaSel ('Just "executionRoleArn") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Text)) :*: (S1 ('MetaSel ('Just "jobFlowId") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text) :*: S1 ('MetaSel ('Just "steps") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 [StepConfig]))))

newAddJobFlowSteps

Arguments

:: Text

$sel:jobFlowId:AddJobFlowSteps'

-> AddJobFlowSteps

Create a value of AddJobFlowSteps with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

AddJobFlowSteps, addJobFlowSteps_executionRoleArn - The Amazon Resource Name (ARN) of the runtime role for a step on the cluster. The runtime role can be a cross-account IAM role. The runtime role ARN is a combination of account ID, role name, and role type using the following format: arn:partition:service:region:account:resource.

For example, arn:aws:iam::1234567890:role/ReadOnly is a correctly formatted runtime role ARN.

$sel:jobFlowId:AddJobFlowSteps', addJobFlowSteps_jobFlowId - A string that uniquely identifies the job flow. This identifier is returned by RunJobFlow and can also be obtained from ListClusters.

$sel:steps:AddJobFlowSteps', addJobFlowSteps_steps - A list of StepConfig to be executed by the job flow.

Request Lenses

addJobFlowSteps_executionRoleArn :: Lens' AddJobFlowSteps (Maybe Text)

The Amazon Resource Name (ARN) of the runtime role for a step on the cluster. The runtime role can be a cross-account IAM role. The runtime role ARN is a combination of account ID, role name, and role type using the following format: arn:partition:service:region:account:resource.

For example, arn:aws:iam::1234567890:role/ReadOnly is a correctly formatted runtime role ARN.

addJobFlowSteps_jobFlowId :: Lens' AddJobFlowSteps Text

A string that uniquely identifies the job flow. This identifier is returned by RunJobFlow and can also be obtained from ListClusters.

addJobFlowSteps_steps :: Lens' AddJobFlowSteps [StepConfig]

A list of StepConfig to be executed by the job flow.

Destructuring the Response

data AddJobFlowStepsResponse

The output for the AddJobFlowSteps operation.

See: newAddJobFlowStepsResponse smart constructor.

Constructors

AddJobFlowStepsResponse'

Fields

• stepIds :: Maybe [Text]

  The identifiers of the list of steps added to the job flow.

• httpStatus :: Int

  The response's http status code.

Instances

Generic AddJobFlowStepsResponse

Defined in Amazonka.EMR.AddJobFlowSteps

Associated Types

type Rep AddJobFlowStepsResponse :: Type -> Type

Methods

from :: AddJobFlowStepsResponse -> Rep AddJobFlowStepsResponse x

to :: Rep AddJobFlowStepsResponse x -> AddJobFlowStepsResponse

Read AddJobFlowStepsResponse

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

readsPrec :: Int -> ReadS AddJobFlowStepsResponse

readList :: ReadS [AddJobFlowStepsResponse]

readPrec :: ReadPrec AddJobFlowStepsResponse

readListPrec :: ReadPrec [AddJobFlowStepsResponse]

Show AddJobFlowStepsResponse

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

showsPrec :: Int -> AddJobFlowStepsResponse -> ShowS

show :: AddJobFlowStepsResponse -> String

showList :: [AddJobFlowStepsResponse] -> ShowS

NFData AddJobFlowStepsResponse

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

rnf :: AddJobFlowStepsResponse -> ()

Eq AddJobFlowStepsResponse

Defined in Amazonka.EMR.AddJobFlowSteps

Methods

(==) :: AddJobFlowStepsResponse -> AddJobFlowStepsResponse -> Bool

(/=) :: AddJobFlowStepsResponse -> AddJobFlowStepsResponse -> Bool

type Rep AddJobFlowStepsResponse

Defined in Amazonka.EMR.AddJobFlowSteps

type Rep AddJobFlowStepsResponse = D1 ('MetaData "AddJobFlowStepsResponse" "Amazonka.EMR.AddJobFlowSteps" "amazonka-emr-2.0-B3cWhP6sdymF3IBUMdPGcd" 'False) (C1 ('MetaCons "AddJobFlowStepsResponse'" 'PrefixI 'True) (S1 ('MetaSel ('Just "stepIds") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [Text])) :*: S1 ('MetaSel ('Just "httpStatus") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Int)))

newAddJobFlowStepsResponse

Arguments

:: Int

$sel:httpStatus:AddJobFlowStepsResponse'

-> AddJobFlowStepsResponse

Create a value of AddJobFlowStepsResponse with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:stepIds:AddJobFlowStepsResponse', addJobFlowStepsResponse_stepIds - The identifiers of the list of steps added to the job flow.

$sel:httpStatus:AddJobFlowStepsResponse', addJobFlowStepsResponse_httpStatus - The response's http status code.

Response Lenses

addJobFlowStepsResponse_stepIds :: Lens' AddJobFlowStepsResponse (Maybe [Text])

The identifiers of the list of steps added to the job flow.

addJobFlowStepsResponse_httpStatus :: Lens' AddJobFlowStepsResponse Int

The response's http status code.