How to troubleshoot 5 common Terraform errors

There's a lot to love about Terraform — Terraform was our most-searched-for topic last year — but nothing is perfect. Errors arise and troubles need troubleshooting. We can help.

In this post, we'll detail some common Terraform errors you may encounter while using Terraform to manage infrastructure — and offer tips on troubleshooting them. Let's go!

Your keys to a better career

Get started with ACG today to transform your career with courses and real hands-on labs in AWS, Microsoft Azure, Google Cloud, and beyond.

When troubleshooting buggy Terraform configurations, you'll more than likely run into an error that falls in one of these four categories:

1. Language errors — Most errors you run into will more than likely fall in this category. These are your syntax errors in your configuration that Terraform prints out the line numbers and a brief explanation of what the error is.
2. State errors — These are also pretty common errors that you are likely to run into when working with Terraform. Most of these errors come from either your state being locked or out of sync. Terraform may destroy or change existing resources if your state is out of sync. Your configuration is always the first place to look when you run into these errors. After you fix any configuration errors, take a look at your state and make sure everything is accounted for there. If there is anything missing from either your configuration or state, you will want to ensure everything is in sync by refreshing, importing, or replacing resources.
3. Core errors — These levels of errors usually mean you have uncovered a bug and may even produce a panic error from the Go language Terraform was written in. These errors usually mean you will be dumping your logs to a file, and attaching it to a templated bug report when you submit an issue on GitHub with the Terraform core development team.
4. Provider errors — Same can be said for the provider errors. These errors revolve around the provider plugins and possible issues with different versions of the plugin. Usually this leads to submitting an issue in GitHub for the development team of that provider plugin.

We're going to focus on five common errors you may run into while using Terraform to manage your infrastructure. These errors fall into our first two categories. We'll take a look at how to fix each one of these errors and go over what to look for to hopefully avoid them in the future. Let’s take a look at our first error.

Grab the Terraform cheat sheet
Check out the top 10 Terraform commands and get a full rundown of all the basic commands you need to get the most out of this intuitive IaC tool in our Terraform cheat sheet.

A Terraform variable interpolation error can occur when trying too append something to an input variable without using the correct interpolation syntax. This error can be caught using the terraform fmt command. This will produce an error similar to our example below:

$ terraform fmt
Error: Invalid character

  on main.tf line 46, in resource "aws_instance" "web_app":
 46:     Name = $var.name-web_app

This character is not used within the language.


Error: Invalid expression

  on main.tf line 46, in resource "aws_instance" "web_app":
  46:     Name = $var.name-web_app

Expected the start of an expression, but found an invalid expression token.

Here you can see that we get two errors. One informing you of an invalid character, and the other informing you of an invalid expression. Both are on the same line.

The fix?

Change this:

   tags = {
      Name = $var.name-web_app
   }
 }

To this:

   tags = {
      Name = "${var.name}-web_app"
   }
 }

This error is common and easy to fix and avoid as long as you remember to use the necessary interpolation syntax when writing your Terraform configuration code.

A cycle error is considered to be an instance of circular logic in your Terraform configuration. When a resource depends on another resource to be created first, but that resource is dependent on the other resource being created, it cause a circle of failed creation in Terraform. Let’s take a look at an example using AWS security groups.

Here is what the error looks like below when you run aterraform validate command:

$ terraform validate

Error: Cycle: aws_security_group.sg_ping, aws_security_group.sg_8080

Not much to go by here in the error other than the resources causing the issue. If we take a look at our configuration below and look for the AWS security groups, we can see what the problem is. You can see that the two security groups that we want to create is looking for the other to be created first. They are both dependent on one another and Terraform can't create either resource because of the dependencies.

 resource "aws_security_group" "sg_ping" {
   name = "Allow Ping"

   ingress {
     from_port       = -1
     to_port         = -1
     protocol        = "icmp"
     security_groups = [aws_security_group.sg_8080.id]
   }
 }

 resource "aws_security_group" "sg_8080" {
   name = "Allow 8080"

   ingress {
     from_port       = 8080
     to_port         = 8080
     protocol        = "tcp"
     security_groups = [aws_security_group.sg_ping.id]
   }
 }

To fix this, we can simply separate our security group resources and our security group rules into separate resources. This way our security groups get created before the rules that need to use dependencies get created after the groups are created like our example below.

resource "aws_security_group" "sg_ping" {
   name = "Allow Ping"
}

resource "aws_security_group" "sg_8080" {
   name = "Allow 8080"
}

resource "aws_security_group_rule" "allow_ping" {
    type = "ingress"
    from_port = -1
    to_port = -1
    protocol = "icmp"
    security_group_id = aws_security_group.sg_ping.id
    source_security_group_id = aws_security_group.sg_8080.id
}

resource "aws_security_group_rule" "allow_8080" {
    type = "ingress"
    from_port = 80
    to_port = 80
    protocol = "tcp"
    security_group_id = aws_security_group.sg_8080.id
    source_security_group_id = aws_security_group.sg_ping.id
}

The Terraform for_each attribute allows you to create a set of similar resources based on the criteria you define.

In our example below, we're creating a set of similar instances, each assigned to a different security group. Terraform cannot parse aws_security_group.*.id in this attribute because the splat expression only interpolates list types, while the for_each attribute is reserved for map types. So we will need to replace the * and add a local value to make this work.

$ terraform validate
Error: Invalid reference

  on main.tf line 44, in resource "aws_instance" "web_app":
  44:   for_each               = aws_security_group.*.id


A reference to a resource type must be followed by at least one attribute
access, specifying the resource name.


Error: Invalid "each" attribute

  on main.tf line 47, in resource "aws_instance" "web_app":
  47:   vpc_security_group_ids = [each.id]


The "each" object does not have an attribute named "id". The supported
attributes are each.key and each.value, the current key and value pair of the
"for_each" attribute set.

First, we'll change our resource block below:

 resource "aws_instance" "web_app" {
   for_each               = aws_security_group.*.id
   ami                    = data.aws_ami.ubuntu.id
   instance_type          = "t2.micro"
   vpc_security_group_ids = [each.id]
   user_data              = <<-EOF
               #!/bin/bash
               echo "Hello, World" > index.html
               nohup busybox httpd -f -p 8080 &
               EOF
   tags = {
     Name = "${var.name}-learn"
   }
 }

To this:

resource "aws_instance" "web_app" {
   for_each               = local.security_groups
   ami                    = data.aws_ami.ubuntu.id
   instance_type          = "t2.micro"
   vpc_security_group_ids = [each.value]
   user_data              = <<-EOF
               #!/bin/bash
               echo "Hello, World" > index.html
               nohup busybox httpd -f -p 8080 &
               EOF
   tags = {
     Name = "${var.name}-learn-${each.key}"
   }
 }
}

Notice above we have changed our for_each value from * to local.security_groups and changed the vpc_security_groups_ids from [each.id] to [each.value]. We also needed to make a quick change to our tag so it also displays each web app that is created by changing ”${var.name}-learn” to ”${var.name}-learn-${each.key}” which will append the number of the VM to the end of the tag name.

Next, we'll create a local block that will convert the list of security groups to a map type like our example below:

locals {
  security_groups = {
    sg_ping   = aws_security_group.sg_ping.id,
    sg_8080   = aws_security_group.sg_8080.id,
  }
}

Now when you run terraform validate, the for_each error should be resolved.

A Terraform output error can occur for a few different reasons, but most commonly it is due to not capturing all the resources you are trying to capture. Take the error below for example.

In this example, since a for expression was used in our main configuration file, our output variables will need to be updated so our outputs can grab all the resources we are trying to display. Let’s take a look.

$ terraform validate

 Error: Missing resource instance key

   on outputs.tf line 3, in output "instance_id":
    3:   value       = aws_instance.web_app.id

 Because aws_instance.web_app has "for_each" set, its attributes must be
 accessed on specific instances.

 For example, to correlate with indices of a referring resource, use:
     aws_instance.web_app[each.key]


 Error: Missing resource instance key

   on outputs.tf line 8, in output "instance_public_ip":
    8:   value       = aws_instance.web_app.public_ip

 Because aws_instance.web_app has "for_each" set, its attributes must be
 accessed on specific instances.

 For example, to correlate with indices of a referring resource, use:
     aws_instance.web_app[each.key]


 Error: Missing resource instance key

   on outputs.tf line 13, in output "instance_name":
   13:   value       = aws_instance.web_app.tags

 Because aws_instance.web_app has "for_each" set, its attributes must be
 accessed on specific instances.

 For example, to correlate with indices of a referring resource, use:
     aws_instance.web_app[each.key]

In our error above, we're trying to output the app id, the public ip, and the tags for our instances. Since we have more than one for each of these, Terraform is throwing an error because our variable is only expecting one of each, and Terraform is confused on which one to grab for our outputs. Below is what our outputs.tf file looks like:

 output "instance_id" {
   description = "ID of the EC2 instance"
    value       = aws_instance.web_app.id
 }

 output "instance_public_ip" {
   description = "Public IP address of the EC2 instance"
    value       = aws_instance.web_app.public_ip
 }

output "instance_name" {
   description = "Tags of the EC2 instance"
   value        = aws_instance.web_app.tags
}

Here you can see that our output variables are only written to grab a single id, a single public ip, and a single tag.

Below is an example of the code we would use to fix this error:

 output "instance_id" {
   description = "ID of the EC2 instance"
    value       = [for instance in aws_instance.web_app: instance.id]
 }

 output "instance_public_ip" {
   description = "Public IP address of the EC2 instance"
    value       = [for instance in aws_instance.web_app: instance.public_ip]
 }

output "instance_name" {
   description = "Tags of the EC2 instance"
   value        = [for instance in aws_instance.web_app: instance.tags.Name]
}

In our above example, we would fix the error by using the for expression to capture all of our instance ids, public tips, and our tags for each instance.

Your Terraform state file stores information on provisioned resources. It will map resources to your configuration and then tracks all associated metadata. If the state is out of sync, Terraform may change your existing resources. After you rule out any configuration errors, it is best to review your state. Ensure your configuration is in sync by refreshing, importing, or replacing resources.

One of the errors you may receive while working with Terraform state is when Terraform is acquiring the state lock. Here is an example of that error below:

Acquiring state lock. This may take a few moments…

  Error: Error acquiring the state lock

  Error message: ConditionalCheckFailedException: The conditional request failed
  Lock Info:
    ID:        c2024f2b-b615-05bf-e516-e49ed2852087
    Path:      <…>
    Operation: OperationTypePlan
    Who:       <…>
    Version:   1.0.2
    Created:   2021-07-28 14:54:32.498842 +0000 UTC
    Info:      


  Terraform acquires a state lock to protect the state from being written
  by multiple users at the same time. Please resolve the issue above and try
  again. For most commands, you can disable locking with the “-lock=false”
  flag, but this is not recommended.

This can occur occasionally and usually means another team member is performing an update to the infrastructure. Sometimes though the lock just gets stuck. To fix this we can try a few things.

First we can run the apply command with the -lock=false flag like the error tells us to. Here is an example of the command below:

terraform force-unlock <ID>

Sometimes this command doesn’t work. You can through a -force flag onto the command to try and force the unlock if the first command does not work like below:

terraform force-unlock -force <ID>

The ID you would use for these commands you can find in the error message.

If both commands don’t unlock the state, you can just wait for a couple of minutes or so, and give it a try again. Finally, try to kill the hanging terraform process. If you are running Terraform on Windows machine, find the process in the Task Manager and kill it. If you are running Terraform on Linux or Mac, run the following commands:

ps aux | grep terraform
kill -9 <process_id>

This should take care of the lock issue if all else fails.

Read: CloudFormation, Terraform, or CDK? A guide to IaC on AWS
Get an overview of the IAC tools available in AWS, and how to choose between them.

So there you have it, gurus! These are just a few common errors you may run into when working with Terraform. Each error is usually easily fixed, you just need to know what you are looking at.

Terraform does a pretty good job at explaining the error and pointing you directly to the problem in your configuration. I find that troubleshooting Terraform is easier than other tools I've used. I'm pretty impressed by how detailed the error messages are, and I really like that it tells you which configuration file has the problem and where to find it in the file.

Hopefully this post will help you easily identify and fix some common errors you'll run into using Terraform. Keep being awesome, gurus — and let’s keep Terraforming!

Looking to learn more about how to troubleshoot Terraform? Check out my Hands-On Troubleshooting with Terraform course.

You can also browse our Terraform training, the top DevOps skills and technologies, and what’s free at ACG this month.

And keep up with all things cloud by following ACG on Twitter, Facebook, subscribing to A Cloud Guru on YouTube, or joining the cloudy conversation in our awesome Discord Community.