I spend a lot of time building infrastructure on AWS. A Lot.

Like many others in the industry I use software tools like CloudFormation and Terraform to do my work. In some cases, I use them both! While I wouldn’t recommend using two tools to manage a single workload, I would absolutely recommend using CloudFormation to initialize your environments.

For example, at work, when we onboard new AWS accounts we initialize the account with some basic access roles and policies from a CloudFormation template because it’s simple to drop-in. There’s no manually provisioning any IAM, setting trust relationships, or policies, or even worse, instructing a customer to do those things, we just distribute a known template using a “Stack Launcher” and customers go click a button. When the stack is available we can then follow on with Terraform modules/templates to deploy the remainder of services/configurations needed for the account.

Terraform on the other hand is a tad more complex. It needs some setup beforehand to do it properly. Those who have operated Terraform for their environments are likely familiar with the concepts of Backends and State Locking. You are also likely familiar with the chicken or the egg scenario you run into when using Terraform in a team environment in which infrastructure is needed for Terraform to safely manage infrastructure…If you’re wanting to use S3 to store state and DynamoDB for storing lock data, what then builds the bucket and table? What about complex state stores like Consul which runs on a cluster of servers?

While there are some clever solutions to the problem (such as in the chicken/egg link above) I prefer a simpler solution.

Perform a one-time, manual initialization of your Terraform Backend using CloudFormation and reference the outputs in your Terraform configuration.

Here’s the CloudFormation template I use to manage Terraform Backend for my home projects (including the infrastructure serving this very site):

AWSTemplateFormatVersion: 2010-09-09
Description: LZAHQ Terraform backend - versioned, encrypted state storage and locking table
    Type: AWS::S3::Bucket
      AccessControl: Private
        - ServerSideEncryptionByDefault:
            SSEAlgorithm: AES256
      BucketName: !Ref AWS::StackName
        Status: Enabled
    Type: AWS::S3::BucketPolicy
      Bucket: !Ref TerraformStateBucket
        - Action: s3:*
          Effect: Allow
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/lance
              - ${BucketArn}/*
              - {BucketArn: !GetAtt TerraformStateBucket.Arn}
    Type: AWS::DynamoDB::Table
        - AttributeName: LockID
          AttributeType: S
        - AttributeName: LockID
          KeyType: HASH
        ReadCapacityUnits: 5
        WriteCapacityUnits: 5
      TableName: !Ref AWS::StackName
    Description: Bucket used to store Terraform remote state file
    Value: !Ref TerraformStateBucket
    Description: DynamoDB table used for Terraform state locking functionality
    Value: !Ref TerraformStateTable

I’ve deployed it just with my IAM user credentials since this is a personal account. In a work environment you may want to setup your own stack launcher so that when you inevitably must log into a new account using the root user or some assumed role (such as when dealing with accounts made from Organizations) you can drop in your Terraform backend stack along with a management IAM role so that you can start deploying the remainder of infrastructure using Terraform.

λ [ ~ ] $ export CF_TEMPLATE=/tmp/tf-backend.yml
λ [ ~ ] $ export STACK_NAME=lzahq-terraform-backend
λ [ ~ ] $ vim $CF_TEMPLATE # paste the contents of the template
λ [ ~ ] $ aws cloudformation deploy \
    --stack-name $STACK_NAME \
    --template-file $CF_TEMPLATE
λ [ ~ ] $ aws cloudformation update-termination-protection \
    --stack-name $STACK_NAME \

Once the stack is deployed, plug the corresponding values into your Terraform configuration:

terraform {
  backend "s3" {
    region = "us-west-2"
    bucket = "lzahq-terraform-backend"
    key = "lzahq.tfstate"
    dynamodb_table = "lzahq-terraform-backend"

Boom. You’ve got your Terraform backend ready for use in your project and it’s safely kept isolated from the rest of your templates. You likely won’t need to touch this for many years.