I wrote about configuring this simple serverless application in a previous blog post. In this part, I will go through how I turned this into Infrastructure as Code (IAC) with Terraform, and why IAC is truly a game-changer when provisioning Cloud resources.
Why Infrastructure as Code?
IAC is not new on this blog. I have previously wrote about using Terraform to provision a new WordPress website. That project involved only 2 AWS services, Lightsail and Route53. You may argue that it is probably simpler to go into the AWS Management Console and provision the website.
However, when projects get more complex, it becomes progressively challenging to always replicate an application after provisioning it manually. Serverless SMS Reminder (Part 1) showed just how many clicks in the management console and fields you will need to fill in. Imagine an even more complex application involving 10 AWS services. Throwing in some human errors, you will be wasting more time troubleshooting than actually provisioning the application.
Infrastructure as Code tools such as CloudFormation and Terraform will allow you to declare the desired end-state configuration in a file or files. The IAC tools can interpret what you have written, call the APIs in the correct sequence and report back to you if the provisioning is successful.
Using IAC allows you to reduce the risk of human errors when configuring manually. IAC also lets you replicate infrastructure provisioning consistently across different environments.
Terraform for Serverless SMS Reminder
All the relevant Terraform Config files and Lambda code can be found in my public Github repository. Feel free to raise issues, fork the project or make a PR.
Terraform loads and reads from all files with the tf extension by default. I typically split the terraform configuration into logically named tf files. In this example, I split it into 4 tf files – cloudwatch.tf, lambda.tf, and sns.tf for each AWS service I’m using, and variables.tf to store all the variables.
By default, Terraform will read tfvars files for variable definitions. I used these files to define the phone numbers I want to send texts to, the SNS message I want to publish, and the AWS region I want to provision this application to.
For the following sections, I will run through snippets of the different tf files in my repository. I had used Terraform AWS provider documentation extensively for this project.
sns.tf
################
# Netflix Topic
################
# create topic
resource "aws_sns_topic" "netflix_topic" {
name = var.topic_1
}
# create sms subscription
resource "aws_sns_topic_subscription" "alex_phone" {
topic_arn = aws_sns_topic.netflix_topic.id
protocol = "sms"
endpoint = var.phone_1
}
resource "aws_sns_topic_subscription" "zk_phone_netflix" {
topic_arn = aws_sns_topic.netflix_topic.id
protocol = "sms"
endpoint = var.phone_5
}The 2 main Terraform resources we will be using are “aws_sns_topic” and “aws_sns_topic_subscription”. These are self-explanatory.
Creating the SNS topic only requires the topic name. To allow this Terraform configuration to be customized easily, I have chosen to use variables where possible. You can define your own unique variables in a terraform.tfvars file.
Instead of using the “sms” protocol for subscribers, you may also choose to use “sqs”, “lambda” or “application”.
# setting this as transactional because promotional and transactional sms costs the same in AP-Southeast-1
resource "aws_sns_sms_preferences" "update_sms_prefs" {
default_sms_type = "Transactional"
}You may also set the default SMS type to be “Transactional” under the “aws_sns_sms_preferences” resource. By default, it will be “Promotional” if undefined. Transactional is usually more costly but guarantees delivery of the message.
lambda.tf
This config file took the longest amount of time and effort to troubleshoot. It made me wonder if using Terraform is the best IAC tool when it comes to configuration Lambda functions.
The prerequisite before this section is to place the zipped lambda code into the root of this folder. The explanation of the code can be found here, and the code repository can be found here.
# IAM role for Lambda to send logs to Cloudwatch and call SNS topic
resource "aws_iam_role" "iam_for_lambda" {
name = "LambdaToSNSandCloudwatch"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}To create an IAM role for this Lambda function, it requires a role policy as above. I realised that omitting this portion caused problems for the function.
resource "aws_iam_policy" "lambda_logging" {
name = "lambda_logging"
path = "/"
description = "IAM policy for logging from a lambda"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*",
"Effect": "Allow"
}
]
}
EOF
}
resource "aws_iam_policy" "sns_publish" {
name = "sns_publish"
path = "/"
description = "IAM policy for publishing to SNS topic from a lambda"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sns:Publish"
],
"Resource": "arn:aws:sns:*:*:*"
}
]
}
EOF
}With the IAM role created, the next step is to create the IAM policies to give Lambda the rights to publish to SNS and Cloudwatch Logs.
resource "aws_iam_role_policy_attachment" "logs_attach" {
role = aws_iam_role.iam_for_lambda.name
policy_arn = aws_iam_policy.lambda_logging.arn
}
resource "aws_iam_role_policy_attachment" "sns_attach" {
role = aws_iam_role.iam_for_lambda.name
policy_arn = aws_iam_policy.sns_publish.arn
}Next, you have to attach the IAM policies you created to the IAM role.
resource "aws_lambda_function" "lambda_to_netflix_topic" {
filename = "sns-lambda.zip"
function_name = "to_netflix_topic"
role = aws_iam_role.iam_for_lambda.arn
handler = "index.handler"
runtime = "nodejs12.x"
environment {
variables = {
SNS_MESSAGE = var.sns_message_1
SNS_TOPIC_ARN = aws_sns_topic.netflix_topic.arn
}
}
}
resource "aws_lambda_permission" "allow_cloudwatch_netflix" {
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda_to_netflix_topic.function_name
principal = "events.amazonaws.com"
}Now, we arrive the Lambda function proper. The “filename” parameter refers to the zipped lambda code in your folder alongside this (.tf) file. The main reason I used environment variables here was because I needed to input the SNS topic ARN that I did not currently have since it was just declared in the (sns.tf) file.
I do not know of any other ways to retrieve the SNS topic ARN for the function code unless I separately provisioned the SNS topic first to extract the ARN. Terraform reads these configuration files and can determine that it has to provision the SNS topic first for the ARN to be available for the Lambda function.
To complete this configuration file, I needed to give the Lambda function the InvokeFunction permission.
cloudwatch.tf
resource "aws_cloudwatch_event_rule" "monthly_netflix" {
schedule_expression = "cron(16 11 1 * ? *)"
name = "Cron-Lambda-SNS-Netflix"
}
resource "aws_cloudwatch_event_target" "sns_netflix" {
rule = aws_cloudwatch_event_rule.monthly_netflix.name
arn = aws_lambda_function.lambda_to_netflix_topic.arn
}The Cloudwatch configuration is rather straightforward. The resource “aws_cloudwatch_event_rule” allows you to define the cron schedule to invoke the Lambda function, and the resource “aws_cloudwatch_event_target” points to the Lambda function you have previously created in (lambda.tf).
Closing Thoughts
Infrastructure as Code tools such as Terraform makes infrastructure provisioning consistent, less error-prone and quick. Personally, I found configuring Lambda functions with Terraform rather challenging, perhaps due to some inaccuracies.
Feel free to reach out to me if you have any suggestions on how I can improve the configuration. Thank you for reading!