Skip to content

Instantly share code, notes, and snippets.

@nathanielks

nathanielks/README.md

Last active March 15, 2025 03:17
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save nathanielks/9282d59ab065f8ee0e373ca7199df085 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save nathanielks/9282d59ab065f8ee0e373ca7199df085 to your computer and use it in GitHub Desktop.
Download ZIP
These scripts are for use with https://github.com/humanmade/cf-to-tf.
Raw
README.md

CloudFormation -> Terraform Import Helpers

I wrote these scripts to help faciliate batch import of our CloudFormation templates into Terraform Configuration.

region-batch.sh

This script is a light wrapper around batch.sh. It simply iterates over the list of regions provided, exports expected variables, creates required directories, and calls batch.sh.

BASE_DIR=/path/to/dir EXCLUDE_REGEX="(some-stack-production|some-stack-staging)" ./region-batch.sh us-east-1 us-west-1

batch.sh

Retrieves all CloudFormation stacks for a REGION and processing any whose names don't match the provided EXCLUDE_REGEX. Will create the WORKING_DIR if it doesn't exist. Uses cf-to-tf to generate terraform config files and state. Also downloads the CloudFormation Stack template from AWS and uses that as the template_body. Runs terraform init in each directory so it's ready to rock 🤘.

REGION=eu-central-1 WORKING_DIR=/path/to/dir/${REGION} EXCLUDE_REGEX="(some-stack-production|some-stack-staging)" ./batch.sh

import-stack.sh

REGION=ap-southeast-1 STACK_NAME=some-stack-production STACK_DIR=/path/to/dir/${REGION}/${STACK_NAME} ./import-stack.sh

find-misconfigured.sh

Iterates over terraform projects in BASE_DIR and runs terraform plan in each directory. If the output doesn't contain No changes., then there's something wrong with the project and requires attention.

BASE_DIR=/path/to/dir ./find-misconfigured.sh

stack-compare.sh

Scans BASE_DIR for files/stack.json and compares them to any files in TEMPLATES_DIR, echoing out matching files.

BASE_DIR=/path/to/dir TEMPLATES_DIR=/path/to/dir-with-cloudformation-templates ./stack-compare.sh
Raw
batch.sh
#! /usr/bin/env bash
# Usage: REGION=eu-central-1 WORKING_DIR=/path/to/dir/${REGION} EXCLUDE_REGEX="(some-stack-production|some-stack-staging)" ./batch.sh
set -euo pipefail
if ! command -v aws >/dev/null 2>&1; then
>&2 echo "aws is required to use this script. \`brew install awscli\` or visit http://docs.aws.amazon.com/cli/latest/userguide/installing.html for installation instructions."
exit 1
fi
REGION="${REGION:-${AWS_REGION}}"
JSON="$(aws cloudformation describe-stacks --query 'Stacks[].StackName' --region "$REGION")"
EXCLUDE_REGEX="${EXCLUDE_REGEX:-}"
ORIG_DIR="$(pwd)"
WORKING_DIR="${WORKING_DIR:-$(pwd)}"
CONFIG="
provider \"aws\" {
region = \"${REGION}\"
# Make sure AWS_PROFILE is set in your environment. This will be used to
# authenticate with the AWS API
}
"
echo "Creating config.tf with AWS provider information"
echo "$CONFIG" | dd status=none of="${WORKING_DIR}/config.tf"
for STACK_NAME in $(echo "$JSON" | jq -rc '.[]'); do
if [[ -n ${EXCLUDE_REGEX} ]]; then
if [[ $STACK_NAME =~ $EXCLUDE_REGEX ]]; then
echo "Skipping $STACK_NAME, matches exclude regex."
continue
fi
fi
export REGION
export STACK_DIR="${WORKING_DIR}/${STACK_NAME}"
export STACK_NAME
./import-stack.sh
done
cd "${ORIG_DIR}" || exit
Raw
find-misconfigured.sh
#! /usr/bin/env bash
# Usage: BASE_DIR=/path/to/dir ./find-misconfigured.sh
set -euo pipefail
DIRS="$(find "${BASE_DIR}" -maxdepth 2 -mindepth 2 -type d)"
for DIR in $DIRS; do
cd "${DIR}"
terraform plan | grep "No changes." > /dev/null || echo "${DIR}"
done
Raw
import-stack.sh
#! /usr/bin/env bash
# Usage: REGION=ap-southeast-1 STACK_NAME=some-stack-name STACK_DIR=/path/to/dir/${REGION}/${STACK_NAME} ./import-stack.sh
set -euo pipefail
if ! command -v aws >/dev/null 2>&1; then
>&2 echo "aws is required to use this script. \`brew install awscli\` or visit http://docs.aws.amazon.com/cli/latest/userguide/installing.html for installation instructions."
exit 1
fi
if ! command -v terraform >/dev/null 2>&1; then
>&2 echo "terraform is required to use this script. \`brew install terraform\` or visit https://www.terraform.io/downloads.html for installation instructions."
exit 1
fi
if ! command -v cf-to-tf >/dev/null 2>&1; then
>&2 echo "cf-to-tf is required to use this script. Visit https://github.com/humanmade/cf-to-tf for installation instructions."
exit 1
fi
if ! command -v jq >/dev/null 2>&1; then
>&2 echo "jq is required to use this script. \`brew install jq\` or visit https://stedolan.github.io/jq/download/ for installation instructions."
exit 1
fi
if ! command -v json2hcl >/dev/null 2>&1; then
>&2 echo "json2hcl is required to use this script. https://github.com/kvz/json2hcl#install for installation instructions."
exit 1
fi
STACK_JSON="$(aws cloudformation describe-stacks --stack-name "$STACK_NAME" --region "$REGION")"
FILES_DIR="${STACK_DIR}/files"
echo "Processing ${STACK_NAME}"
echo "Setting AWS_REGION to ${REGION}"
export AWS_REGION="${REGION}"
echo "Creating ${STACK_DIR}"
mkdir -p "${STACK_DIR}"
cd "${STACK_DIR}" || echo "error changing to ${STACK_DIR}"
echo "Symlinking config.tf"
ln -sf ../config.tf config.tf
echo "Generating config for ${STACK_NAME}"
echo "$STACK_JSON" | cf-to-tf -s - config \
| jq '.resource.aws_cloudformation_stack.main=(.resource.aws_cloudformation_stack.main + { template_body: "${file(var.template_path)}"})' \
| jq --arg template_path "files/stack.json" '. = (. + { variable: { template_path: { default: $template_path} } })' \
| json2hcl | cf-to-tf clean-hcl | terraform fmt - | dd status=none of=main.tf
echo "Generating state for ${STACK_NAME}"
echo "$STACK_JSON" | cf-to-tf -s - state | jq '.' | dd status=none of=terraform.tfstate
echo "Creating ${FILES_DIR} to store Stack Templates in"
mkdir -p "${FILES_DIR}"
echo "Downloading Stack Template for ${STACK_NAME}"
cf-to-tf -s "${STACK_NAME}" template | jq '.' | dd status=none of="${FILES_DIR}/stack.json"
# TODO can we compare the stack to current application-2.0.json?
echo "Running \`terraform init\`"
terraform init
Raw
region-batch.sh
#! /usr/bin/env bash
# Usage: BASE_DIR=/path/to/hm-salt/terraform EXCLUDE_REGEX="(some-stack-production|some-stack-staging)" ./region-batch.sh us-east-1 us-west-1
set -euo pipefail
for REGION in "$@"; do
export REGION
export WORKING_DIR="${BASE_DIR}/${REGION}"
echo "Creating ${WORKING_DIR}"
mkdir -p "${WORKING_DIR}"
echo "Creating Terraform configuration for ${REGION}"
./batch.sh
done
Raw
stack-compare.sh
#! /usr/bin/env bash
# Usage: BASE_DIR=/path/to/dir TEMPLATES_DIR=/path/to/dir-with-cloudformation-templates ./stack-compare.sh
set -euo pipefail
# Required:
# BASE_DIR
# TEMPLATES_DIR
STACK_FILES="$(find "${BASE_DIR}" -name 'stack.json')"
TEMPLATE_FILES="$(find "${TEMPLATES_DIR}" -name '*.json')"
for STACK_FILE in $STACK_FILES; do
for TEMPLATE_FILE in ${TEMPLATE_FILES}; do
if ! diff -B <(jq '.' < "$STACK_FILE") <(jq '.' < "$TEMPLATE_FILE") &>/dev/null; then
continue;
fi
echo "$STACK_FILE matches $TEMPLATE_FILE"
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment