Amazonka.Glue.Types.SparkSQL

Documentation

data SparkSQL

Specifies a transform where you enter a SQL query using Spark SQL syntax to transform the data. The output is a single DynamicFrame.

See: newSparkSQL smart constructor.

Constructors

SparkSQL'

Fields

• outputSchemas :: Maybe [GlueSchema]

  Specifies the data schema for the SparkSQL transform.

• name :: Text

  The name of the transform node.

• inputs :: NonEmpty Text

  The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

• sqlQuery :: Text

  A SQL query that must use Spark SQL syntax and return a single data set.

• sqlAliases :: [SqlAlias]

  A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

  select * from SqlName

  and that gets data from MyDataSource.

Instances

FromJSON SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

parseJSON :: Value -> Parser SparkSQL

parseJSONList :: Value -> Parser [SparkSQL]

ToJSON SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

toJSON :: SparkSQL -> Value

toEncoding :: SparkSQL -> Encoding

toJSONList :: [SparkSQL] -> Value

toEncodingList :: [SparkSQL] -> Encoding

Generic SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Associated Types

type Rep SparkSQL :: Type -> Type

Methods

from :: SparkSQL -> Rep SparkSQL x

to :: Rep SparkSQL x -> SparkSQL

Read SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

readsPrec :: Int -> ReadS SparkSQL

readList :: ReadS [SparkSQL]

readPrec :: ReadPrec SparkSQL

readListPrec :: ReadPrec [SparkSQL]

Show SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

showsPrec :: Int -> SparkSQL -> ShowS

show :: SparkSQL -> String

showList :: [SparkSQL] -> ShowS

NFData SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

rnf :: SparkSQL -> ()

Eq SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

(==) :: SparkSQL -> SparkSQL -> Bool

(/=) :: SparkSQL -> SparkSQL -> Bool

Hashable SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

Methods

hashWithSalt :: Int -> SparkSQL -> Int

hash :: SparkSQL -> Int

type Rep SparkSQL

Defined in Amazonka.Glue.Types.SparkSQL

type Rep SparkSQL = D1 ('MetaData "SparkSQL" "Amazonka.Glue.Types.SparkSQL" "amazonka-glue-2.0-7miPWwBHdfn8N8SvbpLgE0" 'False) (C1 ('MetaCons "SparkSQL'" 'PrefixI 'True) ((S1 ('MetaSel ('Just "outputSchemas") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe [GlueSchema])) :*: S1 ('MetaSel ('Just "name") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text)) :*: (S1 ('MetaSel ('Just "inputs") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (NonEmpty Text)) :*: (S1 ('MetaSel ('Just "sqlQuery") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 Text) :*: S1 ('MetaSel ('Just "sqlAliases") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 [SqlAlias])))))

newSparkSQL

Arguments

:: Text

$sel:name:SparkSQL'

-> NonEmpty Text

$sel:inputs:SparkSQL'

-> Text

$sel:sqlQuery:SparkSQL'

-> SparkSQL

Create a value of SparkSQL with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:outputSchemas:SparkSQL', sparkSQL_outputSchemas - Specifies the data schema for the SparkSQL transform.

$sel:name:SparkSQL', sparkSQL_name - The name of the transform node.

$sel:inputs:SparkSQL', sparkSQL_inputs - The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

$sel:sqlQuery:SparkSQL', sparkSQL_sqlQuery - A SQL query that must use Spark SQL syntax and return a single data set.

$sel:sqlAliases:SparkSQL', sparkSQL_sqlAliases - A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.

sparkSQL_outputSchemas :: Lens' SparkSQL (Maybe [GlueSchema])

Specifies the data schema for the SparkSQL transform.

sparkSQL_name :: Lens' SparkSQL Text

The name of the transform node.

sparkSQL_inputs :: Lens' SparkSQL (NonEmpty Text)

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

sparkSQL_sqlQuery :: Lens' SparkSQL Text

A SQL query that must use Spark SQL syntax and return a single data set.

sparkSQL_sqlAliases :: Lens' SparkSQL [SqlAlias]

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.