Which NPM modules are preinstalled in AWS Lambda execution environment?
I couldn't find an official list so I wrote a script to create a list.
I have a number of other libraries installed in my Lambda, those are (excluding built-in nodejs modules which are also available of course):
'aws-sdk', <-- preinstalled by AWS
'awslambda',
'base64-js',
'dynamodb-doc',
'ieee754',
'imagemagick',
'isarray',
'jmespath',
'lodash',
'sax',
'uuid',
'xml2js',
'xmlbuilder'
Code to generate this list:
function flatten(arrayOfArrays) {
return Array.prototype.concat.apply([], arrayOfArrays)
}
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
function extPackageNames(node) {
if (!node.children) return [];
const arrayOfArrays = node.children.map(c => [c.name].concat(extPackageNames(c)))
const result = flatten(arrayOfArrays)
return result
}
exports.handler = async (event) => {
const rpt = require("read-package-tree")
const module = require("module")
const pathArg = process.env.NODE_PATH
const allPaths = pathArg.split(":")
// '/var/task' is this package on lambda
const externalPaths = allPaths.filter(p => p !== "/var/task")
// read all package data
const ps = externalPaths.map((path) => rpt(path).catch(err => err))
const rpts = await Promise.all(ps).catch(err => err)
// extract the information we need
const packagesPerPath = rpts.map(extPackageNames)
const countPerPath = packagesPerPath.map(arr => arr.length)
const packages = flatten(packagesPerPath)
// remove duplicates
const uniquePackages = packages.filter(onlyUnique)
// remove node.js built-in modules
const uniqueCustomPackages = uniquePackages.filter(p => !module.builtinModules.includes(p))
const result = {
node_path: pathArg,
paths: externalPaths.map((e, i) => [e, countPerPath[i]]),
uniqueCustomPackages
}
console.log(result)
const response = {
statusCode: 200,
body: JSON.stringify(result)
};
return response;
};
To run this on lambda you will need to zip it together with a node_modules
folder containing read-package-tree
.
Unfortunately the docs don't seem very specific about it, but you can grab their docker images yourself and bash in to inspect them:
docker run --rm -it --entrypoint /bin/bash amazon/aws-lambda-nodejs:14
The Dockerfile has ENTRYPOINT ["/lambda-entrypoint.sh"]
and by inspecting that file I was able to determine that it runs Node via /var/runtime/bootstrap
.
/var/runtime/bootstrap
adds various directories to NODE_PATH
from which modules can be loaded:
if [ -z "$NODE_PATH" ];
then
nodejs_mods="/opt/nodejs/node_modules"
nodejs14_mods="/opt/nodejs/node14/node_modules"
runtime_mods="/var/runtime/node_modules"
task="/var/runtime:/var/task"
export NODE_PATH="$nodejs14_mods:$nodejs_mods:$runtime_mods:$task"
fi
However /opt/nodejs
doesn't exist in the image and /var/task
is where Lambda puts your custom code.
So /var/runtime/node_modules
is the only directory we need to inspect:
bash-4.2# cd /var/runtime
bash-4.2# npm ls
/var/runtime
└─┬ [email protected]
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous
└── [email protected] extraneous
As you can see, only aws-sdk
is preinstalled. But you could use this process to determine if they add more preinstalled deps in future versions of their Node.js runtime.
I've used the "https" and the "url" package, so those at least is pre-installed. There are quite a few standard node.js modules which need a native layer.
Clearly the AWS modules are in there, for communicating with AWS services. I've used SQS, for example.
Haven't tried "fs" yet, but since it requires a native layer, and is something you might want to do (e.g. persisting stuff to /tmp) I'm assuming it's there.
Somewhere there ought to be a list. But I can't find one. Guess you just have to try, and if the requires fails, then you need to put a module in node_modules, then see if it demands dependencies.
Only the aws-sdk package is preinstalled .
All the rest is loaded from the "node_modules" directory..
You can find information about it here:
https://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html