Application for copying Amazon Kinesis data to S3.
git clone https://github.com/bigsnarfdude/ScalaKinesis2S3- Configure your application (start with the template
config.properties) - Configure AWS access credentials:
- Locally, set either
- the environment variables
AWS_ACCESS_KEY_ID=xxxandAWS_SECRET_KEY=yyy - or the JVM options
-Daws.accessKeyId=xxxand-D=aws.secretKey=yyy
- the environment variables
- On Amazon EC2
- No configuration needed if you start the server instance with a proper IAM role (highly recommended!)
- Locally, set either
- Run the app:
sbt run
Checkout the source using amazon-kinesis-connectors module provided by awslabs
sbt package
Licensed under the MIT license.
Uses the Amazon Kinesis Client Library, the Amazon Kinesis Connector Library and the AWS SDK for Java. These are also embedded in the runnable jar. See the AWS licensing terms.
The Kinesis Connector sample code contains code for copying Kinesis data to S3, but is not easily reusable. This application contains the minimal code for the copying process and nothing else.
.