CDK vs Terraform vs CloudFormation: A guide to IaC on AWS

CDK vs Terraform vs CloudFormation - which is best? Learn more about these Infrastructure as Code (IaC) tools on AWS, and find out which one is best for you.

I’ve been working with Amazon Web Services for many years, and while the cloud has changed a lot over time, one thing has remained consistent: Infrastructure as Code (IaC) is a core pillar of a healthy implementation of AWS.

For anything bigger than a toy cloud application, IaC is table stakes. You’d be hard-pressed to find someone managing anything of scale who thinks letting folks point and click in the console is the optimal route.

These days, I actually find it faster to just start with all of my applications or even proof-of-concept with an IaC tool and go from there. Time and time again, I've found it easier to return to projects weeks or months later and quickly be able to understand how things work from a familiar baseline and context. I don’t have to rebuild in my mind exactly what I was thinking from scratch.

The “how” of how to approach IaC is, of course, an AWS engineer's very own version of the old "tabs vs spaces" debate.

So, what IaC tools are available to you in AWS, and how do you choose between them? Read on for our summary and comparison of AWS CloudFormation, AWS Cloud Development Kit, and Terraform.

Your keys to a better career

Get started with ACG today to transform your career with courses and real hands-on labs in AWS, Microsoft Azure, Google Cloud, and beyond.

AWS CloudFormation is the original IaC tool for AWS, released in 2011. I have come to respect, hate, love, and revere its power to describe and manage infrastructure. CloudFormation was originally only offered in JSON, but we were treated to a helping of tabs vs spaces actually mattering with native CFN YAML support in 2016.

CloudFormation is one of the safest ways to build, manage, change, and destroy resources in your infrastructure. It offers robust resource state management, and these days it can tell you what's going to happen before you run your deployment.

Let's take a look at some of the great features that make CloudFormation enjoyable and productive to work with.

One of the powerful features of AWS CloudFormation is macros and transforms, which brings whole new capabilities to add your own opinionated capabilities.

Imagine being able to provide opinionated IAM policy generators or S3 bucket resource macros – whatever you want to do, macros can likely get you there. Take note though. While powerful, you can end up treading dangerous territory, as it becomes easy to effectively build your own Domain-Specific Language (DSL). Instead of CloudFormation managing your resources, you're using CloudFormation as a bad DSL compiler that you'll have to babysit.

For a while, we only had custom resources to provision and manage resources that AWS CloudFormation didn’t natively support. This is now largely superseded by resource providers, which allow you to create private or published providers to bring the management of third party and unsupported resources into your stacks. For example, Datadog, a popular monitoring tool, can be used in your stack to provision and manage monitoring without needing some out-of-band process.

In most of my recent work with AWS CloudFormation, I've defaulted to using the AWS Serverless Application Model, or SAM. SAM is a superset of CloudFormation, with some handy transformations that let you do a bit less typing and wiring up of various resources and permissions. Think of it like a well thought out and “managed” macro. If you are doing anything with AWS Lambda or event-driven computing and looking to level up your YAML wrangling, start with SAM.

AWS Cloud Development Kit (CDK) was released in 2019. Using familiar programming languages and provided libraries in TypeScript, Python, Java, and .NET, developers can write with the same code as the rest of their stack to manage their infrastructure.

CDK, however, is not devoid of AWS CloudFormation. In fact, CDK synthesizes to CloudFormation. You still leverage all the state management and inherent benefits (and downsides) of CloudFormation by adopting CDK.

A quick aside: I do want to highlight that some folks view CloudFormation as the “assembly language” of AWS, largely because of how many tools “compile” down to CloudFormation. I think this is a dangerous comparison. It can lead to the interpretation that, like any high-level language to assembly, you don’t really need to understand how the lower-level instruction set works to effectively leverage the higher-level constructs. In my experience, this is patently untrue in the case of CloudFormation. Even a rudimentary understanding of it leads to better decisions in the higher level usages like CDK.

Ultimately, I would contend that CDK is the most comfortable and natural entry point for developers to start building cloud native applications. 

Let's take a look at some of the main features of AWS CDK.

One of the most powerful features of CDK – that I believe AWS CloudFormation has struggled to natively deliver – is the idea of truly shareable and reusable modules. CDK introduced the concept of constructs. In practice, constructs provide everything from simple wrappings of some specific defaults you would like to re-use across your project all the way to complex multi-resource orchestration and wrapping of resource providers. The distribution method for these constructs then relies on the native.

The other important part of CDK constructs is something neat called jsii. To quote the project; “jsii allows code in any language to naturally interact with JavaScript classes. It is the technology that enables the AWS Cloud Development Kit to deliver polyglot libraries from a single codebase!”. If you write your constructs with TypeScript, it's fairly straightforward to distribute and utilize those constructs across the other core CDK languages - further encouraging sharing of modules.

One of the most elegant ways I can illustrate how nice the CDK experience can be is to show a side-by-side comparison of the usage of Amazon States Language (ASL).

First, what it looks like in AWS CloudFormation Native ASL:

{
  "DeliveryStepFunctionStateMachine": {
    "Type": "AWS::StepFunctions::StateMachine",
    "Properties": {
      "RoleArn": {
        "Fn::GetAtt": ["DeliveryStepFunctionStateMachineRoleC6479370", "Arn"]
      },
      "DefinitionString": {
        "Fn::Join": [
          "",
          [
            "{"StartAt":"MapperTask","States":{"MapperTask":{"Next":"SetStatusTo-pending","Retry":[{"ErrorEquals":["States.ALL"],"MaxAttempts":10}],"Parameters":{"FunctionName":"",
            {
              "Ref": "DeliveryStepFunctionMapper"
            },
            "","Payload.$":"$"},"OutputPath":"$.Payload","Type":"Task","Resource":"arn:",
            {
              "Ref": "AWS::Partition"
            },
            ":states:::lambda:invoke"},"SetStatusTo-pending":{"Next":"retry seconds","Type":"Task","ResultPath":null,"Resource":"arn:",
            {
              "Ref": "AWS::Partition"
            },
            ":states:::dynamodb:updateItem","Parameters":{"Key":{"pk":{"S.$":"$.pk"},"sk":{"S.$":"$.sk"}},"TableName":"",
            {
              "Ref": "PersistenceDDBTable"
            },
            "","ExpressionAttributeNames":{"#status":"status"},"ExpressionAttributeValues":{":status":{"S":"pending"}},"ReturnValues":"ALL_NEW","UpdateExpression":"SET #status = :status"}},"retry seconds":{"Type":"Wait","SecondsPath":"$.retrySeconds","Next":"SetStatusTo-in-progress"},"SetStatusTo-in-progress":{"Next":"DeliverTransactionTask","Type":"Task","ResultPath":null,"Resource":"arn:",
            {
              "Ref": "AWS::Partition"
            },
            ":states:::dynamodb:updateItem","Parameters":{"Key":{"pk":{"S.$":"$.pk"},"sk":{"S.$":"$.sk"}},"TableName":"",
            {
              "Ref": "PersistenceDDBTable"
            },
            "","ExpressionAttributeNames":{"#status":"status"},"ExpressionAttributeValues":{":status":{"S":"in-progress"}},"ReturnValues":"ALL_NEW","UpdateExpression":"SET #status = :status"}},"DeliverTransactionTask":{"Next":"Delivery success?","Retry":[{"ErrorEquals":["States.ALL"],"MaxAttempts":10}],"Parameters":{"FunctionName":"",
            {
              "Ref": "DeliveryStepFunctionDeliverTransaction"
            },
            "","Payload.$":"$"},"OutputPath":"$.Payload","Type":"Task","Resource":"arn:",
            {
              "Ref": "AWS::Partition"
            },
            ":states:::lambda:invoke"},"Delivery success?":{"Type":"Choice","Choices":[{"Variable":"$.status","StringEquals":"complete","Next":"SetStatusTo-complete"},{"Variable":"$.status","StringEquals":"failed","Next":"SetStatusTo-failed"}],"Default":"SetStatusTo-pending"},"SetStatusTo-complete":{"End":true,"Type":"Task","ResultPath":null,"Resource":"arn:",
            {
              "Ref": "AWS::Partition"
            },
            ":states:::dynamodb:updateItem","Parameters":{"Key":{"pk":{"S.$":"$.pk"},"sk":{"S.$":"$.sk"}},"TableName":"",
            {
              "Ref": "PersistenceDDBTable"
            },
            "","ExpressionAttributeNames":{"#status":"status"},"ExpressionAttributeValues":{":status":{"S":"complete"}},"ReturnValues":"ALL_NEW","UpdateExpression":"SET #status = :status"}},"SetStatusTo-failed":{"End":true,"Type":"Task","ResultPath":null,"Resource":"arn:",
            {
              "Ref": "AWS::Partition"
            },
            ":states:::dynamodb:updateItem","Parameters":{"Key":{"pk":{"S.$":"$.pk"},"sk":{"S.$":"$.sk"}},"TableName":"",
            {
              "Ref": "PersistenceDDBTable"
            },
            "","ExpressionAttributeNames":{"#status":"status"},"ExpressionAttributeValues":{":status":{"S":"failed"}},"ReturnValues":"ALL_NEW","UpdateExpression":"SET #status = :status"}}}}"
          ]
        ]
      }
    }
  }
}

Then with AWS CDK (leveraging some existing constructs to handle editing the Amazon DynamoDB records for me).

const STATUS = "$.status"
const RETRY_SECONDS = "$.retrySeconds"
const PENDING = "pending"
const PROGRESS = "in-progress"
const FAILED = "failed"
const COMPLETE = "complete"

const setPending = stepFunction.setStatus(this, props.table, PENDING);
const setProgress = stepFunction.setStatus(this, props.table, PROGRESS);
const setSuccess = stepFunction.setStatus(this, props.table, COMPLETE);
const setFailed = stepFunction.setStatus(this, props.table, FAILED);
const waitForNSeconds = this.waitTask("retry seconds", RETRY_SECONDS);

const definition = this.mapperTask()
  .next(setPending)
  .next(waitForNSeconds)
  .next(setProgress)
  .next(this.deliverTransactionTask())
  .next(
    new sfn.Choice(this, "Delivery success?")
      .when(sfn.Condition.stringEquals(STATUS, COMPLETE), setComplete)
      .when(sfn.Condition.stringEquals(STATUS, FAILED), setFailed)
      .otherwise(setPending)
  );

If you had to read the second code snippet to understand what the first was doing, I’d completely understand. Granted, there is nothing stopping CloudFormation from adopting and supporting a more elegant DSL. In fact, AWS SAM is really an attempt at exactly this, with a focus on the serverless developer experience. 

Given the current community momentum around CDK and growing investment from AWS, I expect to see more and more teams starting with CDK and happily continuing with it as their primary utility for infrastructure management.

Terraform was introduced in 2014 with the goal of being able to orchestrate infrastructure as code. It first targeted AWS, but has grown to be able to manage a large ecosystem of modules. In fact, the capability of multi-provider support is one of the main selling points of the technology.

Terraform introduced its own DSL, called Hashicorp Configuration Language (HCL). On the surface, it feels like a more human-friendly JSON. JSON is also natively supported within Terraform, if you have a masochistic side.

Grab the Terraform cheat sheet
Check out the top 10 Terraform commands and get a full rundown of all the basic commands you need to get the most out of Terraform in our Terraform cheat sheet.

AWS Infrastructure as Code is just fancy state management. The biggest difference between Terraform and AWS CloudFormation is how it actually interacts with the infrastructure itself. With CloudFormation, you can hand it a representation of your goal state and it will perform all the operations on your infrastructure to get you there natively within the platform. Likewise, Terraform takes the representation of your goal state and constructs a plan of API calls directly to your AWS infrastructure to get to that state.

In a perfect world, both approaches work flawlessly. But this is the cloud we're talking about. And everything fails all the time, as Werner Vogels likes to remind us.

Until recently, Terraform was superior in terms of being able to recover from people going outside the process to update resources. It was able to resolve inconsistencies and refresh a correct state of the infrastructure even if someone had manually edited that security group “just to test something”. AWS CloudFormation struggled with these inconsistent states, but the introduction of drift detection attempted to solve some of this headache.

Terraform also offers the more elegant story of importing unmanaged resources, or resources from other stacks. CloudFormation offers this, but only for the subset of resources that support drift detection.

In addition to these benefits, Terraform on AWS is really the one true option for “learn once, utilize most places”. Regardless of your feelings on multicloud or hybrid-cloud, the appeal of training up yourself or your team on a singular technology that can benefit from knowledge transfer across many different possible targets is tempting.

The introduction of CDK for Terraform (CDKTF) effectively allows developers to write CDK that, under the hood, targets Terraform instead of CloudFormation. This is the closest we can get in the cloud world to having our cake and eating it, as you can imagine a CDK application that uses CloudFormation for your AWS nested stack targets and Terraform for external provider stack targets.

So, which tool should you choose? Given the vast amount of choices and business requirements that are out there, it’s irresponsible to levy a one-size-fits-all opinion in a 1600-word article. Rather, I’d approach it with a series of questions to ask yourself when considering your options.

The only truly wrong answer is the one that prevents you from building anything at all.

The IaC space is growing, and everyone has their own opinion and how things should work. I’d argue competition is healthy and in some cases has forced the providers themselves to step up their game. Here are some other tools available in the IaC space.

Trek10 is an AWS Premier Consulting Partner focusing on cloud-native and serverless applications.