Using boto to invoke lambda functions how do I do so asynchronously?
The following are a python function that accepts lambda-function-Name to invoke and payload to send to that function.
It invokes the lambda function by boto3 client.
import boto3, json, typing
def invokeLambdaFunction(*, functionName:str=None, payload:typing.Mapping[str, str]=None):
if functionName == None:
raise Exception('ERROR: functionName parameter cannot be NULL')
payloadStr = json.dumps(payload)
payloadBytesArr = bytes(payloadStr, encoding='utf8')
client = boto3.client('lambda')
return client.invoke(
FunctionName=functionName,
InvocationType="RequestResponse",
Payload=payloadBytesArr
)
And usage:
if __name__ == '__main__':
payloadObj = {"something" : "1111111-222222-333333-bba8-1111111"}
response = invokeLambdaFunction(functionName='myLambdaFuncName', payload=payloadObj)
print(f'response:{response}')
An asynchronously executed AWS Lambda function doesn't return the result of execution. If an asynchronous invocation request is successful (i.e. there were no errors due to permissions, etc), AWS Lambda immediately returns the HTTP status code 202 ACCEPTED and bears no further responsibility for communicating any information about the outcome of this asynchronous invocation.
From the documentation of AWS Lambda Invoke action:
Response Syntax
HTTP/1.1 StatusCode X-Amz-Function-Error: FunctionError X-Amz-Log-Result: LogResult Payload
Response Elements
If the action is successful, the service sends back the following HTTP response.
StatusCode
The HTTP status code will be in the 200 range for successful request. For the
RequestResponse
invocation type this status code will be 200. For theEvent
invocation type this status code will be 202. For theDryRun
invocation type the status code will be 204.[...]
The response returns the following as the HTTP body.
Payload
It is the JSON representation of the object returned by the Lambda function. This is present only if the invocation type is
RequestResponse
.
There is a difference between an 'async AWS lambda invocation' and 'async python code'. When you set the InvocationType
to 'Event'
, by definition, it does not ever send back a response.
In your example, invoke()
immediately returns None
, and does not implicitly start up anything in the background to change that value at a later time (thank goodness!). So, when you look at the value of response
15 seconds later, it's still None
.
It seems what you really want is the RequestResponse
invocation type, with asynchronous Python code. You have a bunch of options to choose from, but my favorite is concurrent.futures
. Another is threading
.
Here's an example using concurrent.futures
:
(If you're using Python2 you'll need to pip install futures
)
from concurrent.futures import ThreadPoolExecutor
import json
payload = {...}
with ThreadPoolExecutor(max_workers=5) as executor:
futs = []
for x in xrange(0, 5):
futs.append(
executor.submit(client.invoke,
FunctionName = "loadSpotsAroundPoint",
InvocationType = "RequestResponse",
Payload = bytes(json.dumps(payload))
)
)
results = [ fut.result() for fut in futs ]
print results
Another pattern you might want to look into is to use the Event
invocation type, and have your Lambda function push messages to SNS, which are then consumed by another Lambda function. You can check out a tutorial for SNS-triggered lambda functions here.