Terraform Resource Testing 101

Validate code quality with a Terraform plan using JSON and Pytest

Published in

Better Programming

7 min readFeb 10, 2023

Most people can relate to the experience of “pretesting” as a step in the learning process during formal schooling. In some cases, pretesting might have occurred on the first day of school. Pretesting was when the teacher gave out a test containing a lot of terms or materials you probably knew nothing about but were going to learn in the coming days, weeks, or even months.

I’ve recently learned how to automate the testing of resources from a Terraform JSON plan, which reminds me of pretesting. The resources in the plan haven’t actually been created yet, but it’s a way to validate the code quality to ensure the resources will be created in the way they are intended.

There are many justifiable arguments as to why testing Terraform resources may not be worth the trouble, and I’m not here to argue for or against it. I’m just here to show that it’s possible, easy, and could provide business value. This article will include the code to test Terraform resources with a JSON plan and Pytest. This code will be low-level but should open your mind to the possibilities of testing Terraform resources in this way.

Please note: All images, unless otherwise noted, are by the author.

Required Dependencies

Install Terraform here.
Install Pytest here.

Planning Terraform Resources

Terraform supports many providers, but we will use AWS for this demonstration. For real-world use cases, you would use environment variables to pass along your AWS credentials, which lets Terraform know which account to create the resources in. For simplicity and security purposes, we will create a Terraform plan of AWS resources with mock AWS credentials. The first file you will need is provider.tf, with the following code:

We only need one more file along with the provider.tf file to create the plan for the AWS resources. Terraform's best practice is to separate resources and create similar resources in the same files. This means you would put all EC2 instances in the same file and name it ec2.tf. Then, you would repeat that process for all S3 buckets, and so on.

While that approach is recommended for complex infrastructure, we will put three different resources in the same file for this demonstration's purposes. It doesn’t make a difference for Terraform — Terraform doesn’t require resource separation; it’s just not ideal when creating a real application and needing more reusable code.

The next thing we need is a resources.tf file with the following code:

As you can probably figure out, we will create a plan for an EC2 instance, a security group, and an IAM user via the Terraform code. To put it very plainly, the Terraform terminal execution workflow goes something like this: terraform init, terraform plan, terraform apply.

The names of these commands are pretty straightforward. You initialize Terraform, plan out the resources created, and apply that plan to the desired provider. Our whole goal with creating these Terraform files is to generate the Terraform plan as a JSON file that we can later parse and test. We are going to bypass applying the Terraform for this demonstration.

This demonstration aims to show that the Terraform code can be tested PRIOR to the application, which will help us avoid creating mistakes in our resources. Anything crucial to creating resources can be validated against the Terraform JSON plan. Common mistakes may include the following:

Incorrectly named resources
General typos
Missing tags
Unintended region

Before we create our Terraform plan, let’s create a few directories to house our Terraform plan files. Before continuing, make sure your project is up to date with the following structure:

Current Tree Structure of Files and Directories for Demo

Now we are ready to create our Terraform plan based on the provider.tf and resources.tf files that have already been created. The terminal commands we will implement vary slightly from the typical workflow. Read the comments for an explanation and execute the terminal commands as they are listed below:

## Initialize Terraform
$ terraform init

## Plan Terraform and save file in tests/resources/tf_plan directory
$ terraform plan -out=tests/resources/tf_plan/myplan.plan

## Transform Terraform plan into a JSON file to be used for testing
$ terraform show -json tests/resources/tf_plan/myplan.plan > tests/resources/tf_plan/plan.json

The JSON file is rather long, so if you want to check and see if yours is on target, you can check out mine here.

Pro-tip: The shortcut for formatting JSON in VSCode is fn+shift+option+f!

The part of the file I want to highlight is the beginning, so here is a look at the first 27 lines:

In this demonstration, we will test for our resources’ names. For example, the IAM resource’s name is listed on line 15. We will need to target that key/value pair by drilling down from line 9: planned_values > root_module > resources. This will make more sense as we move on to testing, which we are ready to do now!

Testing Preparation

Our testing needs will remain isolated in the tests directory, which already houses the Terraform plan files created in the last section. From the tests directory, let's create another directory called data_util as a sister of the resources directory. Create a file inside of data_util and name it: data_resolver.py. The purpose of this file is to dynamically take in a JSON file and read its contents for our tests. This file should look like this:

As mentioned in the last section, we will create a test for the expected names of AWS resources created via Terraform. To avoid hardcoding these values into the test, we will utilize a .ini file to house these values and additional values to come.

Please note: From my personal experience, .ini files are very sensitive to tabbing. I recommend you check out this resource about .ini and other configuration files.

In the tests/resources directory, let's create a new file called datatable.ini and insert the expected names of the AWS resources, as shown below:

[RESOURCES]
IAM = my-iam-user
EC2 = my-ec2-instance
SG = my-ec2-sg

Pytest and Terraform JSON Plan

Now we are ready to create our actual test! In the tests directory, let’s create a new directory called terraform and create a file inside this directory called test_tf_plan.py.

We will write one test in this file, but the setup will make it easy to add additional tests as needed. The explanation of setup and code flow will follow the gist below. The file should contain the following:

Three things are happening in this file. Let me walk you through it:

Lines 7–11 (line 12 is included for learning, and it is not necessary for the test). These lines bring the values from the datatable.ini into the test file so they can be validated against the JSON file.
Line 16 utilizes the data_resolver.py file we created for reading JSON files and brings in the JSON Terraform plan file we created during the Terraform section.
Lines 18–28 = THE TEST!!

We start the test on line 18 by integrating Pytest’s parametrize built-in mechanism — learn more here.
Then we drill down into the JSON file on line 20, as I previewed at the end of the Terraform section. Line 21 grabs all values with the “name” key in the JSON file.
Then in lines 24 and 25 (line 26 is also not necessary; it is used for debugging purposes), we check both ways to see if the values from the datatable.ini and the values of “name” in the JSON file match.
The assertion on line 28 then becomes: assert there are no differences between the datatable.ini values and the values of “name” in the JSON file. If there are differences, the test will fail; however, we should expect this test to pass.

To execute the test, simply run the following command:

pytest -sv

Here’s the expected result:

Terminal Screenshot: Execute pytest -sv command, yields 1 passing test — Terminal screenshot: Execute `pytest -sv` command

The test passes because the values of the resource names in the datatable.ini file and the names of the resources created in the resources.tf file are the same. If this were a real-world situation, it would be safe to use terraform apply at this point.

You can verify this test is acting as expected by going into either of those two files and altering a resource name, even by simply deleting one character. After altering a resource name and executing the test command again, the test will fail and print out the difference between the names.

To see the entirety of the demonstration, check out the sample repo I created in GitHub:

GitHub - taylorwagner/terraform-pytest

You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or…

github.com

Pretesting Value and Beyond

It might be critical to your business or application that resource names follow a specific naming convention or that resources have certain tags for billing needs. Adhering to these requirements could potentially hold a great deal of monetary value. Through this article and code, you should see a fairly simple way to automate the testing of Terraform code quality to ensure these requirements are met before creating the resources.

At this point, the pretesting is over. You may be thinking, how can I validate the resources have been created correctly after the terraform apply? I recommend looking into Boto3. To learn more about Boto3 and have the opportunity to mock out Boto3’s methods, check out another article I wrote about Moto and AWS Databases here.