Amazon Kinesis + Integration Tests
It might already be too late to give the solution but I will add what my team has done to replicate AWS resources locally as we use a lot of Kinesis, DynamoDb, S3 and cloudWatch.
We have created wrappers around Localstack -> https://github.com/localstack/localstack that allow us to spin up local instances of the necessary services as docker containers using docker-compose
.
A typical docker-compose.yml
file for us looks like:
version: '2'
services:
localstack:
image: "localstack/localstack"
environment:
- SERVICES=kinesis,dynamodb,cloudwatch
ports:
- "4568"
- "4569"
- "4582"
Then during the setup phase for the integration-tests, our wrapper fires up docker-compose up
and runs the tests against the local infrastructure.
Later during tear-down, the wrapper kills the containers.
I ran into the same issue and the only mock implementation I found so far was a nodejs one: https://github.com/mhart/kinesalite It does the trick - I managed to run my Java Kinesis client against it, just had to set the endpoint on the kinesis.properties:
kinesisEndpoint=http://localhost:4567
The downside is that it is not trivial to use it during build time tests - need to figure a way to start the mock kinesis before the test (using a maven plugin or something), didn't get to it yet..