Terraform state: What it is, and how to properly manage it

Terraform State. It can be either your best friend or your worst nightmare. Which side of the fence it falls on depends completely on how you manage it. In this blog post, we’ll take a look at Terraform state, why it's awesome, how you can manage your state, and why you need to be careful.

Terraform state is one of the great things about Terraform. It maps the resources in your configuration to their real-world counterparts. This allows you to easily detect drift between your desired state and the actual state of your resources. 

But it doesn’t stop there - Terraform state also improves performance when deploying large configurations, because it allows you to skip the required back-and-forth between the provider’s API and the Terraform executable. Normally Terraform will refresh the state when you run Terraform Plan, by comparing real-world resources with what is currently in state. But you can easily skip this process and save some time, by setting the refresh option to false: terraform plan -refresh=false

State is an essential part of Terraform and you can’t use Terraform without state, but you don’t have to rely on Terraform to manage the state for you. 

• You can inspect resources in your state with the terraform show and terraform state list commands.
• You can import resources into state using the terraform import command.
• You can make terraform forget about real-world resources with the terraform state rm command.
• You can move resources in state with terraform state mv, which is useful when you change the local name in your configuration.

When you run the first terraform plan, Terraform will automatically create a local state file for you, the file is named terraform.tfstate and is placed in the same working directory as your configuration files. The state file isn’t anything special, it’s just a JSON (JavaScript Object Notation) file.

What’s great about the state (you see what I did there?), is that it stores everything, even sensitive data you might forget, like passwords. That’s cool, right?

Yeah, I agree, that’s totally NOT COOL. We all know passwords should never be stored in plain text, especially in the working directory! Also you should never, ever store your state file in a source control repository.

Terraform state should be treated just like any sensitive data. You should encrypt your state file at rest and in transit. Access to the state file should also be tightly controlled, and you should also back up your state. Terraform will automatically back-up the state file for you when you perform certain state-related operations, but it’s important to keep a recent copy as a backup.

One way to help you securely store your and access your state file is to use a backend. Backends allow you to store your state in a remote location, and share the state with your team. There are a range of backends available, including Azure Storage, Amazon S3 and more, but there is another option that handles all of the security for you, and that is Terraform Cloud. 

With Terraform Cloud, you can ensure your state is encrypted at rest and in transit and share the state with your team. Where Terraform Cloud falls short is on backups, you’ll need to make sure you keep a backup of your state in another secure location. The restoration process with a backup is much faster than the process you need to follow to recreate your state (oh yeah, nailed it, again, you’re welcome) from scratch.

You also should consider the size of your state file. You should apply the principle of least privilege, and a state file that is used to manage resources across multiple workload teams can be a security risk, so you should keep your configurations modular, which will lead to smaller and easier to secure state files. Terraform Cloud workspaces can be used to make this easier.

So, what are the most important considerations when managing your Terraform State?

• Always make sure you secure your state file and treat it just like you would any other secret;
• Secure your state at rest and in transit;
• Use the principle of least privilege by separating workloads out and using separate state files for different workload teams and environments; and 
• Always back-up your state files

If you’re keen to get hands-on with Terraform on Azure including how you can manage your state with Terraform Cloud, check out my latest course, Hands-on with Terraform on Azure, or if you’re ready to dive a little deeper, you can check out our complete Terraform library with heaps of lessons and labs to sink your teeth into.