AWS Lambda is definitely the New Hotness - it allows you to run functions in the cloud with no persistent infrastructure!

Lambda can be described as "able to respond to predefined events by launching AWS resources and executing functions, without the user needing to manage the underlying infrastructure". Lambda is essentially user-created functions that run on demand in the cloud. This has the following benefits:

  • Only pay for what you use - no ongoing costs
  • Hugely scalable

In some projects, we were able to do away with EC2 nodes or containers entirely. For example, we recently built an application with no infrastructure at all - only a web site in S3, and functions executed in Lambda which talked to a Firebase database. This meant that we had absolutely no web nodes at all to run our code:

Application running on S3, Lambda and Firebase

What about workflow?

This is a novel way of doing software; it is certainly quite different to much of what we have done before. We have a few principles that we like to apply to our development practises, and so, one of our first questions was, "how can we incorporate this into our workflow?". This includes considerations like:

  1. Automation - How can we automatically deploy our application?
  2. Speed of development - How can we run it locally so we don't have to keep uploading our code through Amazon's console every time we make a change?
  3. Testing - How can we test our code?

Working with node-lambda

Whilst we can use the AWS console to manage our Lambda functions, we can get a smoother development workflow using node-lambda. This tool allows both pushing of Lambda function and, crucially, means we can run our Lambda functions locally.

Development with node-lambda

To get started, run:

node-lambda setup

You now have two new files - .env and event.json. .env is a file that node-lambda uses to configure your function - what its name is, which role is used to access it, how much memory, and so on. event.json is the data that node-lambda will pass to your function when you invoke it with node-lambda run.

Local testing with node-lambda

Let's assume that your function is in a file index.js and looks like the following:

exports.handler = function(event, context) {
    console.log( "event", event );
    context.done( );
};

Also, let's assume that in event.json you have:

{
    "foo":"bar"
}

You can run it with:

node-lambda run

Because it posts whatever data it finds in event.json to your function, you should see:

event { foo: 'bar' }

To verify it's running truly offline, turn off wifi, unplug your ethernet, and then run node-lambda run again. This proves that it's running locally and not connecting to AWS.

Deployment with node-lambda

To set this up, configure .env so it has all the right values:

AWS_ENVIRONMENT=development
AWS_ACCESS_KEY_ID=YOUR_KEY
AWS_SECRET_ACCESS_KEY=YOUR_SECRET
AWS_ROLE_ARN=your_amazon_role
AWS_FUNCTION_NAME=your_function_name
AWS_MODE=event
AWS_MEMORY_SIZE=128
AWS_TIMEOUT=3
AWS_DESCRIPTION=
AWS_RUNTIME=nodejs

Most of these defaults will work without modification, but the ones you really need to pay attention to are:

  • AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY: access key assigned by AWS. See our post Painless Immutable Infrastructure with Ansible and AWS.
  • AWS_ROLE_ARN: looks something like arn:aws:iam::1234567890123:role/lambda_basic_execution
  • AWS_FUNCTION_NAME: the name of your function. Must be unique. node-lambda will automatically suffix the version number from package.json.

Running the following will publish your function:

node-lambda deploy

Check out .env for the configuration of the function. Essentially, it creates or updates the function {AWS_FUNCTION_NAME}-{version}, where version is taken from your package.json. If your function name is foo and your package.json has version 1.2.3, function foo-1.2.3 will be set up in Lambda. Here's an example:

A couple of versions of the same function

Troubleshooting node-lambda deploy

When telling .env which role to use, make sure you use a role ARN, not the function ARN. I got:

'validation error detected: Value \'arn:aws:lambda:REDACTED:REDACTED:function:REDACTED\' at \'role\' failed to satisfy constraint: Member must satisfy regular expression pattern: arn:aws:iam::\\d{12}:role/?[a-zA-Z_0-9+=,.@\\-_/]+'

I had to enable these Managed Policies for the role lambda_basic_execution:

  • AWSLambdaBasicExecutionRole
  • AWSLambdaRole
  • Modularise your code and unit test it outside of Lambda - your lambda function should hand off to a library that you have built and tested separately using more conventional tools. This way, it's easy to unit test your code in a normal workflow.
  • Test locally with node-lambda run. This verifies that it all integrates well.
  • Use node-lambda deploy to update your function in the cloud.
  • Use the version number is package.json for versioning.
  • Be careful about invoking Lambda directly in browser. Yes, you can do it, but be careful not to expose your client key and secret key.

Wrapping up

These are practises that have worked well for us so far; hopefully this has given you a few ideas to help you to get started with working with AWS Lambda.

Other resources