Why is aws lambda invocation client incorrectly returning ClientExecutionTimeoutException?
Javadocs in aws-sdk-java says:
For functions with a long timeout, your client might be disconnected during synchronous invocation while it waits for a response. Configure your HTTP client, SDK, firewall, proxy, or operating system to allow for long connections with timeout or keep-alive settings.
On the other hand, previously AWS Lambda was limited up to 5 minutes, later this limit was increased up to 15 minutes.
I would check:
- The client sdk version is up to date
- The connection is not closed by your network
- Move to an async invocation via
AWSLambdaAsyncClient.invokeAsync()
for long running invocations.
I've accepted Ezequiel's answer since it was technically a networking / OS issue, but here is a more detailed result:
We had to ensure all relevant clients were configured to keep alive tcp connections. We then had to add these properties to the /etc/sysctl.conf
file on our EC2 residing in a private subnet because the NAT gateway is set to kill idle connections beyond 350s:
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 100
net.ipv4.tcp_keepalive_probes = 6