Amazonka.Firehose.Types.ParquetSerDe

Documentation

data ParquetSerDe

A serializer to use for converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet.

See: newParquetSerDe smart constructor.

Constructors

ParquetSerDe'

Fields

• blockSizeBytes :: Maybe Natural

  The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

• compression :: Maybe ParquetCompression

  The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

• enableDictionaryCompression :: Maybe Bool

  Indicates whether to enable dictionary compression.

• maxPaddingBytes :: Maybe Natural

  The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

• pageSizeBytes :: Maybe Natural

  The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

• writerVersion :: Maybe ParquetWriterVersion

  Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

Instances

FromJSON ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

parseJSON :: Value -> Parser ParquetSerDe

parseJSONList :: Value -> Parser [ParquetSerDe]

ToJSON ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

toJSON :: ParquetSerDe -> Value

toEncoding :: ParquetSerDe -> Encoding

toJSONList :: [ParquetSerDe] -> Value

toEncodingList :: [ParquetSerDe] -> Encoding

Generic ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Associated Types

type Rep ParquetSerDe :: Type -> Type

Methods

from :: ParquetSerDe -> Rep ParquetSerDe x

to :: Rep ParquetSerDe x -> ParquetSerDe

Read ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

readsPrec :: Int -> ReadS ParquetSerDe

readList :: ReadS [ParquetSerDe]

readPrec :: ReadPrec ParquetSerDe

readListPrec :: ReadPrec [ParquetSerDe]

Show ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

showsPrec :: Int -> ParquetSerDe -> ShowS

show :: ParquetSerDe -> String

showList :: [ParquetSerDe] -> ShowS

NFData ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

rnf :: ParquetSerDe -> ()

Eq ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

(==) :: ParquetSerDe -> ParquetSerDe -> Bool

(/=) :: ParquetSerDe -> ParquetSerDe -> Bool

Hashable ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

Methods

hashWithSalt :: Int -> ParquetSerDe -> Int

hash :: ParquetSerDe -> Int

type Rep ParquetSerDe

Defined in Amazonka.Firehose.Types.ParquetSerDe

type Rep ParquetSerDe = D1 ('MetaData "ParquetSerDe" "Amazonka.Firehose.Types.ParquetSerDe" "amazonka-kinesis-firehose-2.0-6l1hHmh2UnHBvolxuQFkWB" 'False) (C1 ('MetaCons "ParquetSerDe'" 'PrefixI 'True) ((S1 ('MetaSel ('Just "blockSizeBytes") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :*: (S1 ('MetaSel ('Just "compression") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe ParquetCompression)) :*: S1 ('MetaSel ('Just "enableDictionaryCompression") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Bool)))) :*: (S1 ('MetaSel ('Just "maxPaddingBytes") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :*: (S1 ('MetaSel ('Just "pageSizeBytes") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe Natural)) :*: S1 ('MetaSel ('Just "writerVersion") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedStrict) (Rec0 (Maybe ParquetWriterVersion))))))

newParquetSerDe :: ParquetSerDe

Create a value of ParquetSerDe with all optional fields omitted.

Use generic-lens or optics to modify other optional fields.

The following record fields are available, with the corresponding lenses provided for backwards compatibility:

$sel:blockSizeBytes:ParquetSerDe', parquetSerDe_blockSizeBytes - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

$sel:compression:ParquetSerDe', parquetSerDe_compression - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

$sel:enableDictionaryCompression:ParquetSerDe', parquetSerDe_enableDictionaryCompression - Indicates whether to enable dictionary compression.

$sel:maxPaddingBytes:ParquetSerDe', parquetSerDe_maxPaddingBytes - The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

$sel:pageSizeBytes:ParquetSerDe', parquetSerDe_pageSizeBytes - The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

$sel:writerVersion:ParquetSerDe', parquetSerDe_writerVersion - Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

parquetSerDe_blockSizeBytes :: Lens' ParquetSerDe (Maybe Natural)

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

parquetSerDe_compression :: Lens' ParquetSerDe (Maybe ParquetCompression)

The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

parquetSerDe_enableDictionaryCompression :: Lens' ParquetSerDe (Maybe Bool)

Indicates whether to enable dictionary compression.

parquetSerDe_maxPaddingBytes :: Lens' ParquetSerDe (Maybe Natural)

The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

parquetSerDe_pageSizeBytes :: Lens' ParquetSerDe (Maybe Natural)

The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

parquetSerDe_writerVersion :: Lens' ParquetSerDe (Maybe ParquetWriterVersion)

Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.