Call aws-cli from AWS Lambda

aws-cli is a python package. To make it available on a AWS Lambda function you need to pack it with your function zip file.

1) Start an EC2 instance with 64-bit Amazon Linux;

2) Create a python virtualenv:

mkdir ~/awscli_virtualenv
virtualenv ~/awscli_virtualenv

3) Activate virtualenv:

cd ~/awscli_virtualenv/bin
source activate

4) Install aws-cli and pyyaml:

pip install awscli
python -m easy_install pyyaml

5) Change the first line of the aws python script:

sed -i '1 s/^.*$/\#\!\/usr\/bin\/python/' aws

6) Deactivate virtualenv:

deactivate

7) Make a dir with all the files you need to run aws-cli on lambda:

cd ~
mkdir awscli_lambda
cd awscli_lambda
cp ~/awscli_virtualenv/bin/aws .
cp -r ~/awscli_virtualenv/lib/python2.7/dist-packages .
cp -r ~/awscli_virtualenv/lib64/python2.7/dist-packages .

8) Create a function (python or nodejs) that will call aws-cli:

For example (nodejs):

var Q = require('q');
var path = require('path');
var spawn = require('child-process-promise').spawn;    

exports.handler = function(event, context) {

    var folderpath = '/folder/to/sync';
    var s3uel = 's3://name-of-your-bucket/path/to/folder';

    var libpath = path.join(__dirname, 'lib');
    var env = Object.create(process.env);
    env.LD_LIBRARY_PATH = libpath;

    var command = path.join(__dirname, 'aws');
    var params = ['s3', 'sync', '.', s3url];
    var options = { cwd: folderpath };

    var spawnp = spawn(command, params, options);

    spawnp.childProcess.stdout.on('data', function (data) {
        console.log('[spawn] stdout: ', data.toString());
    });

    spawnp.childProcess.stderr.on('data', function (data) {
        console.log('[spawn] stderr: ', data.toString());
    });

    return spawnp
    .then(function(result) {

        if (result['code'] != 0) throw new Error(["aws s3 sync exited with code", result['code']].join(''));

        return result;

    });

}

Create the index.js file (with the code above or your code) on ~/awscli_lambda/index.js

9) Zip everything (aws-cli files and dependencies and your function):

cd ~
zip -r awscli_lambda.zip awscli_lambda

Now we can use Layers inside Lambda. Bash layer with aws-cli is available at https://github.com/gkrizek/bash-lambda-layer

handler () {
    set -e

    # Event Data is sent as the first parameter
    EVENT_DATA=$1

    # This is the Event Data
    echo $EVENT_DATA

    # Example of command usage
    EVENT_JSON=$(echo $EVENT_DATA | jq .)

    # Example of AWS command that's output will show up in CloudWatch Logs
    aws s3 ls

    # This is the return value because it's being sent to stderr (>&2)
    echo "{\"success\": true}" >&2
}

Not unless you include it (and all of its dependencies) as part of your deployment package. Even then you would have to call it from within python since Lambda doesn't allow you to execute shell commands. Even if you get there, I would not recommend trying to do a sync in a Lambda function since you're limited to a maximum of 5 minutes of execution time. On top of that, the additional spin-up time just isn't worth it in many cases since you're paying for every 100ms chunk.

So you can, but you probably shouldn't.

EDIT: Lambda does allow you to execute shell commands