Terraform was first released in 2014 and has been a staple in the infrastructure as code (IaC) community. I recently passed the Terraform Authoring and Operations Professional certification and I have been teaching Terraform for a few years and have helped a great many Fortune 500 companies with adoption. This has led me to think about some of the aspects of Terraform that could be improved and what a v2 might look like.
Splatting
Splatting in Terraform is a way to return an attribute from each element in a list of objects. Prior to splatting, a for expression was required to get a list of all of the “id” attributes, for example:
output "subnet_ids" {
value = [for subnet in aws_subnet.subnets : subnet.id]
}
With splatting, this is simplifed:
output "subnet_ids" {
value = aws_subnet.subnets[*].id
}
However, this has very limited use cases. If there are several instances of a resource created with count, splatting can be used, but if for_each is used, splatting cannot be used. The documentation for this has not always been entirely clear, so it has led to frustration for many users.
It would be nice if this could be expanded to work with for_each, as well. This would allow for a more consistent experience and reduce the cognitive load for users. But I think the concept of splatting could be greatly expanded.
Splatting v2
What I really would like is the ability to splat a map of objects into a `for_each`. This behavior exists within PowerShell and offers very advantageous patterns.
In PowerShell, let’s say I want to execute a Cmdlet:
Copy-Item -Path C:\source\* -Destination C:\destination
If I need to iterate over many inputs, I could do the follow:
foreach ($source in $sources) {
Copy-Item -Path $source -Destination C:\destination
}
But let’s say that I conditionally would like to use the -WhatIf parameter:
foreach ($source in $sources) {
if ($source -like "*important*") {
Copy-Item -Path $source -Destination C:\destination -WhatIf
continue
}
Copy-Item -Path $source -Destination C:\destination
}
If the path contains “important”, it will use the -WhatIf parameter. The problem here is that I have two instances of the Copy-Item Cmdlet. If I need ot make a change that would impact each version, I have to make two edits. With splatting, I can have a single instance of the Cmdlet and I can conditionally add the `-WhatIf` parameter by adding to the hashtable:
foreach ($source in $sources) {
$CopyParams = @{
Path = $source
Destination = 'C:\destination'
}
if ($source -like "*important*") {
$CopyParams.Add('WhatIf', $true)
}
Copy-Item @CopyParams
}
In this behavior, splatting will use any of the keys that match parameters in the Cmdlet and ignore any keys that are not available to the Cmdlet. If each instance of $source is already a hashtable, this looks more simplified:
foreach ($CopyParams in $sources) {
if ($CopyParams.source -like "*important*") {
$CopyParams.Add('WhatIf', $true)
}
Copy-Item @CopyParams
}
Something like this would be amazing in Terraform. A map of objects could be passed to the for_each statement in a way that communicates we want to use splatting. We could manipulate that data structure asuch that each object in the map could conditionally have different keys:
locals {
subnets = {
public-1a = {
cidr_block = "10.42.0.0/24"
}
public-1b = {
cidr_block = "10.42.1.0/24"
}
private-1a = {
cidr_block = "10.42.10.0/24"
}
private-1b = {
cidr_block = "10.42.11.0/24"
}
}
subnets_with_attributes = {
for k, v in local.subnets : k => merge(
v,
{
availability_zone = join("", [
data.aws_region.ctx.name,
substr(k, -1, 1)
])
vpc_id = aws_vpc.vpc.id
}
)
}
subnets_auto_map_public = {
for k, v in local.subnets_with_attributes : k => merge(
v,
{
map_public_ip_on_launch = true
}
) if strcontains(k, "public")
}
modified_subnets = merge(
local.subnets_with_attributes,
local.subnets_auto_map_public
)
}
resource "aws_subnet" "subnets" {
for_each = @local.modified_subnets
}
So, a lot of data manipulation was done here. Normally, I might take in a map from a variable input, so the original local.subnets wouldn’t be hard-coded. But the local.subnets_with_attributes is adding the vpc_id and availability_zone to each object in the map. Then, any keys that contain “public” have the map_public_ip_on_launch key added to the object, conditionally. Finally, the modified_subnets map is created by merging the two maps together.
I added a notation of prepending the identifier with “@” similar to PowerShell, but it could use any chosen symbol for the notation that the maintainers would prefer. This would signal that for_each should “splat” all of the arguments into the map by connecting the keys to the arguments when they exist.
Data Manipulation
Some of the data manipulation techniques are cumbersome in Terraform, looking at the previous example. If a portion of a data structure needs to be manipulated, but the entire data structure is still preferred to be a whole, it requires filtering the data structure to access only the values that need manipulation and then combining it with the previous data structure using merge to overwrite the values that were manipulated.
In this particular situation, I could have simply used a ternary operator to always add map_public_ip_on_launch and set the value to true or false based on the condition. However, in situations where a nested data structure is the value of a key, that would create weaker patterns. Having the ability to append an attribute onto an object within a map would be great. Perhaps some sort of unnamed or lambda style of syntax could be used to manipulate the data structure in place:
locals {
subnets += {
for k, v in local.subnets : k => v.append({
map_public_ip_on_launch = true
}) if strcontains(k, "public")
}
}
I am not convinced this is the way it should work as I don’t think my thoughts on this are entirely matured, yet. Others contributing considerations could help shape this idea.
Conclusion
I am sure others have considered what Terraform v2 might look like, including the maintainers at HashiCorp. Hopefully the community is able to provide some guidance around this and shape the future of the product. I think there OpenTofu might also be a good way to “incubate” some ideas, as well. For instance, the idea of encrypting that state file was something many people wanted in Terraform for a long time and the folks involved with OpenTofu made this one of thier first priorities. I think this led HashiCorp to respond with something that may be more compelling with “ephemeral values”. Instead of building in encryption of the state contents to Terraform, eliminating more of the sensitive values from the state file is a better approach, or at least something that can be done in conjunction with encryption. But encryption seems like it should be something handled in the backend and not built directly into the tool. So, others could propose idea and even flesh them out but it would spark the discussion and have many competing ideas to choose from.
Dustin Dortch 