amazonka-kinesis-firehose-1.3.7: Amazon Kinesis Firehose SDK.

Copyright(c) 2013-2015 Brendan Hay
LicenseMozilla Public License, v. 2.0.
MaintainerBrendan Hay <brendan.g.hay@gmail.com>
Stabilityauto-generated
Portabilitynon-portable (GHC extensions)
Safe HaskellNone
LanguageHaskell2010

Network.AWS.Firehose

Contents

Description

Amazon Kinesis Firehose API Reference

Amazon Kinesis Firehose is a fully-managed service that delivers real-time streaming data to destinations such as Amazon S3 and Amazon Redshift.

See: AWS API Reference

Synopsis

Service Configuration

firehose :: Service Source

API version '2015-08-04' of the Amazon Kinesis Firehose SDK configuration.

Errors

Error matchers are designed for use with the functions provided by Control.Exception.Lens. This allows catching (and rethrowing) service specific errors returned by Firehose.

InvalidArgumentException

_InvalidArgumentException :: AsError a => Getting (First ServiceError) a ServiceError Source

The specified input parameter has an value that is not valid.

ConcurrentModificationException

_ConcurrentModificationException :: AsError a => Getting (First ServiceError) a ServiceError Source

Another modification has already happened. Fetch VersionId again and use it to update the destination.

ServiceUnavailableException

_ServiceUnavailableException :: AsError a => Getting (First ServiceError) a ServiceError Source

The service is unavailable, back off and retry the operation. If you continue to see the exception, throughput limits for the delivery stream may have been exceeded. For more information about limits and how to request an increase, see Amazon Kinesis Firehose Limits.

ResourceNotFoundException

_ResourceNotFoundException :: AsError a => Getting (First ServiceError) a ServiceError Source

The specified resource could not be found.

LimitExceededException

_LimitExceededException :: AsError a => Getting (First ServiceError) a ServiceError Source

You have already reached the limit for a requested resource.

ResourceInUseException

_ResourceInUseException :: AsError a => Getting (First ServiceError) a ServiceError Source

The resource is already in use and not available for this operation.

Waiters

Waiters poll by repeatedly sending a request until some remote success condition configured by the Wait specification is fulfilled. The Wait specification determines how many attempts should be made, in addition to delay and retry strategies.

Operations

Some AWS operations return results that are incomplete and require subsequent requests in order to obtain the entire result set. The process of sending subsequent requests to continue where a previous request left off is called pagination. For example, the ListObjects operation of Amazon S3 returns up to 1000 objects at a time, and you must send subsequent requests with the appropriate Marker in order to retrieve the next page of results.

Operations that have an AWSPager instance can transparently perform subsequent requests, correctly setting Markers and other request facets to iterate through the entire result set of a truncated API operation. Operations which support this have an additional note in the documentation.

Many operations have the ability to filter results on the server side. See the individual operation parameters for details.

PutRecord

UpdateDestination

PutRecordBatch

CreateDeliveryStream

DescribeDeliveryStream

ListDeliveryStreams

DeleteDeliveryStream

Types

CompressionFormat

DeliveryStreamStatus

NoEncryptionConfig

BufferingHints

bufferingHints :: BufferingHints Source

Creates a value of BufferingHints with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

bhSizeInMBs :: Lens' BufferingHints (Maybe Natural) Source

Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5.

We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.

bhIntervalInSeconds :: Lens' BufferingHints (Maybe Natural) Source

Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.

CopyCommand

copyCommand Source

Creates a value of CopyCommand with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

ccCopyOptions :: Lens' CopyCommand (Maybe Text) Source

Optional parameters to use with the Amazon Redshift COPY command. For more information, see the "Optional Parameters" section of Amazon Redshift COPY command. Some possible examples that would apply to Amazon Kinesis Firehose are as follows.

'delimiter '\t' lzop;' - fields are delimited with "\t" (TAB character) and compressed using lzop.

'delimiter '|' - fields are delimited with "|" (this is the default delimiter).

'delimiter '|' escape' - the delimiter should be escaped.

'fixedwidth 'venueid:3,venuename:25,venuecity:12,venuestate:2,venueseats:6'' - fields are fixed width in the source, with each width specified after every column in the table.

'JSON 's3://mybucket/jsonpaths.txt'' - data is in JSON format, and the path specified is the format of the data.

For more examples, see and Amazon Redshift COPY command exmaples.

ccDataTableColumns :: Lens' CopyCommand (Maybe Text) Source

A comma-separated list of column names.

ccDataTableName :: Lens' CopyCommand Text Source

The name of the target table. The table must already exist in the database.

DeliveryStreamDescription

dsdCreateTimestamp :: Lens' DeliveryStreamDescription (Maybe UTCTime) Source

The date and time that the delivery stream was created.

dsdLastUpdateTimestamp :: Lens' DeliveryStreamDescription (Maybe UTCTime) Source

The date and time that the delivery stream was last updated.

dsdDeliveryStreamARN :: Lens' DeliveryStreamDescription Text Source

The Amazon Resource Name (ARN) of the delivery stream.

dsdVersionId :: Lens' DeliveryStreamDescription Text Source

Used when calling the UpdateDestination operation. Each time the destination is updated for the delivery stream, the VersionId is changed, and the current VersionId is required when updating the destination. This is so that the service knows it is applying the changes to the correct version of the delivery stream.

dsdHasMoreDestinations :: Lens' DeliveryStreamDescription Bool Source

Indicates whether there are more destinations available to list.

DestinationDescription

destinationDescription Source

Creates a value of DestinationDescription with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

EncryptionConfiguration

encryptionConfiguration :: EncryptionConfiguration Source

Creates a value of EncryptionConfiguration with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

ecNoEncryptionConfig :: Lens' EncryptionConfiguration (Maybe NoEncryptionConfig) Source

Specifically override existing encryption information to ensure no encryption is used.

KMSEncryptionConfig

kmsEncryptionConfig Source

Creates a value of KMSEncryptionConfig with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

kecAWSKMSKeyARN :: Lens' KMSEncryptionConfig Text Source

The ARN of the encryption key. Must belong to the same region as the destination Amazon S3 bucket.

PutRecordBatchResponseEntry

data PutRecordBatchResponseEntry Source

Contains the result for an individual record from a PutRecordBatch request. If the record is successfully added to your delivery stream, it receives a record ID. If the record fails to be added to your delivery stream, the result includes an error code and an error message.

See: putRecordBatchResponseEntry smart constructor.

putRecordBatchResponseEntry :: PutRecordBatchResponseEntry Source

Creates a value of PutRecordBatchResponseEntry with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

prbreErrorCode :: Lens' PutRecordBatchResponseEntry (Maybe Text) Source

The error code for an individual record result.

prbreErrorMessage :: Lens' PutRecordBatchResponseEntry (Maybe Text) Source

The error message for an individual record result.

Record

data Record Source

The unit of data in a delivery stream.

See: record smart constructor.

record Source

Arguments

:: ByteString

rData

-> Record 

Creates a value of Record with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

rData :: Lens' Record ByteString Source

The data blob, which is base64-encoded when the blob is serialized. The maximum size of the data blob, before base64-encoding, is 1,000 KB.

Note: This Lens automatically encodes and decodes Base64 data, despite what the AWS documentation might say. The underlying isomorphism will encode to Base64 representation during serialisation, and decode from Base64 representation during deserialisation. This Lens accepts and returns only raw unencoded data.

RedshiftDestinationConfiguration

rdcRoleARN :: Lens' RedshiftDestinationConfiguration Text Source

The ARN of the AWS credentials.

rdcS3Configuration :: Lens' RedshiftDestinationConfiguration S3DestinationConfiguration Source

The S3 configuration for the intermediate location from which Amazon Redshift obtains data. Restrictions are described in the topic for CreateDeliveryStream.

The compression formats SNAPPY or ZIP cannot be specified in S3Configuration because the Amazon Redshift COPY operation that reads from the S3 bucket doesn't support these compression formats.

RedshiftDestinationDescription

rddRoleARN :: Lens' RedshiftDestinationDescription Text Source

The ARN of the AWS credentials.

RedshiftDestinationUpdate

redshiftDestinationUpdate :: RedshiftDestinationUpdate Source

Creates a value of RedshiftDestinationUpdate with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

rduS3Update :: Lens' RedshiftDestinationUpdate (Maybe S3DestinationUpdate) Source

The Amazon S3 destination.

The compression formats SNAPPY or ZIP cannot be specified in S3Update because the Amazon Redshift COPY operation that reads from the S3 bucket doesn't support these compression formats.

rduRoleARN :: Lens' RedshiftDestinationUpdate (Maybe Text) Source

The ARN of the AWS credentials.

S3DestinationConfiguration

s3DestinationConfiguration Source

Creates a value of S3DestinationConfiguration with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

sdcPrefix :: Lens' S3DestinationConfiguration (Maybe Text) Source

The "YYYY/MM/DD/HH" time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket. For more information, see Amazon S3 Object Name Format in the guide-fh-dev.

sdcEncryptionConfiguration :: Lens' S3DestinationConfiguration (Maybe EncryptionConfiguration) Source

The encryption configuration. If no value is specified, the default is no encryption.

sdcCompressionFormat :: Lens' S3DestinationConfiguration (Maybe CompressionFormat) Source

The compression format. If no value is specified, the default is UNCOMPRESSED.

The compression formats SNAPPY or ZIP cannot be specified for Amazon Redshift destinations because they are not supported by the Amazon Redshift COPY operation that reads from the S3 bucket.

sdcBufferingHints :: Lens' S3DestinationConfiguration (Maybe BufferingHints) Source

The buffering option. If no value is specified, BufferingHints object default values are used.

sdcRoleARN :: Lens' S3DestinationConfiguration Text Source

The ARN of the AWS credentials.

S3DestinationDescription

sddPrefix :: Lens' S3DestinationDescription (Maybe Text) Source

The "YYYY/MM/DD/HH" time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket. For more information, see Amazon S3 Object Name Format in the guide-fh-dev.

sddRoleARN :: Lens' S3DestinationDescription Text Source

The ARN of the AWS credentials.

sddBufferingHints :: Lens' S3DestinationDescription BufferingHints Source

The buffering option. If no value is specified, BufferingHints object default values are used.

sddCompressionFormat :: Lens' S3DestinationDescription CompressionFormat Source

The compression format. If no value is specified, the default is NOCOMPRESSION.

sddEncryptionConfiguration :: Lens' S3DestinationDescription EncryptionConfiguration Source

The encryption configuration. If no value is specified, the default is no encryption.

S3DestinationUpdate

s3DestinationUpdate :: S3DestinationUpdate Source

Creates a value of S3DestinationUpdate with the minimum fields required to make a request.

Use one of the following lenses to modify other fields as desired:

sduPrefix :: Lens' S3DestinationUpdate (Maybe Text) Source

The "YYYY/MM/DD/HH" time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket. For more information, see Amazon S3 Object Name Format in the guide-fh-dev.

sduEncryptionConfiguration :: Lens' S3DestinationUpdate (Maybe EncryptionConfiguration) Source

The encryption configuration. If no value is specified, the default is no encryption.

sduCompressionFormat :: Lens' S3DestinationUpdate (Maybe CompressionFormat) Source

The compression format. If no value is specified, the default is NOCOMPRESSION.

The compression formats SNAPPY or ZIP cannot be specified for Amazon Redshift destinations because they are not supported by the Amazon Redshift COPY operation that reads from the S3 bucket.

sduBufferingHints :: Lens' S3DestinationUpdate (Maybe BufferingHints) Source

The buffering option. If no value is specified, BufferingHints object default values are used.

sduBucketARN :: Lens' S3DestinationUpdate (Maybe Text) Source

The ARN of the S3 bucket.

sduRoleARN :: Lens' S3DestinationUpdate (Maybe Text) Source

The ARN of the AWS credentials.