Terraform Setup for Using AWS Lambda With S3

Build your buckets

I recently came across a scenario where I wanted to make modifications to a bunch of incoming S3 files. The best way to do so was to use event triggers that run an AWS Lambda function every time a new file was uploaded to S3. Although the process of creating S3 buckets, event notifications, and Lambda functions is pretty straightforward while using the console, it can become a bit tricky while using Terraform.

Assuming that you already have a basic Terraform setup, let’s look into how you can first build your buckets.

S3 Buckets:

For variables like the bucket name, you can either store values as local variables in a separate file or you can simple quote them here. Whether you want server-side encryption or not is optional, and usually depends on your use case.

Lambda

To begin with, you need to create a src directory, which will contain the Python code for your Lambda function.

terraform
│   buckets.tf
│   lambda.tf 
│   notifications.tf
│   permissions.tf
│   ...
└───src
│   │   conversion_lambda_python_file.py
│   │   
│   │

The filename you give to your Python file here will later be used as the handler value for Lambda’s Terraform setup. Your Lambda function should have a function called the lambda_handler. It could look something like this:.

Notice how we are reading the event key in the lambda_handler function. Now, our Lambda’s Terraform setup could look somewhat like this: