Testing Terraform code has always been a challenge. Several testing suites had been used, but they always required knowledge of a different language and that is anti-pattern for software testing; tests should always be written in the same language as the code that you’re testing.
Existing testing suites:
- terratest from Gruntwork with tests written in Go
- kitchen-terraform – tests written in Ruby
I find it rather disappointing that terratest requires Go because Terraform is written in Go and it would seem to be something that could have been contributed to Terraform so that it was integrated. Much of the value proposition of Terraform is that it is something that can be adopted by people without a programming background, so expecting tests to be written in Go is unsettling, to say the least (and that isn’t considering that it is already an anti-pattern).
History of Testing in Terraform
One of the impediments to testing with Terraform has been the idea that a terraform plan is already a type of testing. Many have considered anything more unnecessary. However, testing requires some rigor and should be a test against what actually happened (and I am also a huge opponent to mock testing… it is basically pointless). The plan does perform testing, but what it is testing is the comparing the current environment to the graph of the desired state. I did this in the days before I used Terraform with PowerShell to modify configuration settings in the Office 365 services.
What I would do is write a series of tests for PowerShell using Pester. Then, when I executed it in a pipeline, I ran the test first and would take the resulting NUnit file and grab all failed tests so that I would only need to change what didn’t pass. I would take that as input into my code and the code had the logic to only modify things to match the desired state (this was not using PowerShell DSC). However, I never really considered that a test. After I executed the changes, I would run the tests again to validate that it had the desired outcome. This is what is missing in the terraform plan as testing concept.
However, Terraform began its march towards testing:
- v0.13.0 – Input variable validation was introduced
- v0.15.0 – the experimental implementation of
terraform testwas released - v1.2.0 – Preconditions and Postconditions were introduced, building on validation
- v1.5.0 – Checks were introduced
- v1.6.0 – the general availability of
terraform testwill be released
Experimental Testing
One thing that always bothered me during the experimental test period was that the formal QA language was not used. After reviewing many features that were experimental and became GA, this seems like a strategy from HashiCorp. There is a clear difference between the syntax and keywords used during the experimental period and the GA period, which can give a clear indication of when some particular Terraform code was written.
In the experimental period, tests would be written in normal HCL in a tests set of subdirectories and the tested code would be imported as a locally sourced module:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| module "main" { | |
| source = "../.." | |
| } |
There was also a built-in test provider that needed to be sourced:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| terraform { | |
| required_providers { | |
| test = { | |
| source = "terraform.io/builtin/test" | |
| } | |
| } | |
| } |
The provider offered a resource called “test_assertions” with some nested blocks for testing, in particular, the “equal” block:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| resource "test_assertions" "defaults" { | |
| component = "defaults" | |
| equal "name" { | |
| description = "The name of the resource is my-resource-name" | |
| got = module.main.name | |
| want = "my-resource-name" | |
| } | |
| } |
The tests would be executed by running terraform test from the root module working directory.
As I mentioned, the language bothered me because QA has many existing standards that cut across languages and suites. Words like “Got” should be “Actual” and “Want” should be “Expected”.
GA Changes
Nearly all of this changes with the GA release that will occur with Terraform v1.6.0. The constant is that terraform test is still the command and it is run in the root module working directory.
Going forward, tests will be written in files in the root module working directory and will have extensions of *.tftest.hcl and *.tftest.json instead of *.tf.
A couple new blocks will be introduced as well:
variables(not to be confused withvariable)run
The variables block will work in much the same way that a locals block is used but it will seed the inputs to variables defined in the code that will be tested similarly to a TFVARS file. Otherwise, think of them as the same.
If in my code I have a variable named name and cidr_block, then I would seed that with:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| variables { | |
| name = "my-resource-name" | |
| cidr_block = ["192.0.2.0/24"] | |
| } |
To write tests, we’ll use the run block. It contains a “command” argument that instructs what type of operation to use for testing, either “plan” or “apply”. Then there are assert nested blocks that implement the same syntax as validation nested blocks for variables and pre/post conditions:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| run "test_resource_name" { | |
| command = apply | |
| assert { | |
| condition = bbbc_resource.mine.name = "my-resource-name" | |
| error_message = "Name did not match expected value" | |
| } | |
| } |
Variable values from the variables block can be overridden by using a variables nested block inside the run block.
Beyond the new blocks, the providers block can be used in our tests to source a specific version of a provider or get it from a specific registry and we can provide specific configuration details about the provider. Maybe we would like to use a specific test account/subscription.
The final feature touches on negative testing. For instance, we might want to supply bad data and the expectation is that the test fails. This can be accomplished with the expected_failures argument for the run block. It accepts a list of identifiers that will fail, but only really supports one identifier to be passed, currently.
Desired Feature
I would really enjoy if verbose testing output was provided with the results of every assertion, similar to PowerShell:
This would give significant details.
Final Thoughts
Terraform test is focused on testing of modules. This is where the priority should exist. If you’re not currently doing any testing, modules should be where testing begins as they are the code that will be re-used. It can also be used as a proxy for quality (but be careful not to overly rely on proxies).
Further, tests are not something that you write at the beginning and call it “done”. Tests should be continually updated as feedback is learned. For instance, a bug was discovered with specific inputs; a test should be added that validates the fix and works as a regression test ongoing to ensure it isn’t reintroduced. Further, tests should spend time on the limits. If you have an input that should be a value from 0-100, have negative tests for -1 and 101, then test exactly 0 and 100, then test something in between.
While the priority on testing is focused on modules, tests can work on all Terraform code. If you have some code that is important, test it.
Dustin Dortch 