Photo by Maksym Kaharlytskyi on Unsplash
Importing CloudFormation templates into a CDK application
And understanding CDK proxy objects
Table of contents
- Introduction
- Why import a CloudFormation template into a CDK application?
- We start with a CloudFormation template
- Import resources to CDK
- Proxy objects
- Modify resources imported using CfnInclude
- Modify resources using lookup methods
- Use convenience methods exposed by proxy objects
- Deploy a stack with imported resources
- Conclusion
Introduction
In this brief blog post, we will look into how we can integrate an existing AWS CloudFormation template into an AWS CDK (Cloud Development Kit) application, and some of the use cases where we might want to use this approach. We will also talk about proxy objects and how they work when importing resources.
Before we start
👋🏼 Hi, I'd love to connect with like-minded professionals and talk about all things AWS. Reach out to me on LinkedIn and feel free to share your thoughts on this blog post, too.
Why import a CloudFormation template into a CDK application?
At first glance, importing a CloudFormation template into a CDK application might sound odd, but there are several scenarios where it makes perfect sense. Two of the most common use cases I came across are:
Deploying a template that someone from within or outside of our organization shared with us, as a part of our CDK application. For example, an SaaS provider sharing resources required to integrate with their platform, or an internal operations team sharing a template that deploys a VPC with on-premise connection.
Migrating from CloudFormation to CDK without the need to rewrite all of our existing infrastructure.
It allows us to deploy our stack(s) as a unified application, regardless of whether resources are defined in CloudFormation or CDK, using a single command (cdk deploy
), instead of managing and deploying stacks using two deployment mechanisms.
We start with a CloudFormation template
Imagine we have deployed a CloudFormation stack to create resources for our business application. It’s a simple static website hosted with an S3 bucket and CloudFront distribution.
AWSTemplateFormatVersion: "2010-09-09"
Parameters:
Environment:
Type: String
Resources:
WebsiteBucket:
Type: AWS::S3::Bucket
Properties:
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
CloudFrontDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Origins:
- DomainName: !GetAtt WebsiteBucket.DomainName
Id: S3Origin
S3OriginConfig: {}
OriginAccessControlId: !Ref OriginAccessControl
Enabled: true
DefaultRootObject: index.html
# ... left out for simplicity
OriginAccessControl:
Type: AWS::CloudFront::OriginAccessControl
Properties:
# ...
WebsiteBucketPolicy:
Type: AWS::S3:BucketPolicy
Properties:
# ...
After the initial POC, the team decided to switch to CDK to take advantage of its capabilities.
“So what do we do with the existing CloudFormation templates?”
Instead of dedicating time to rewriting our existing infrastructure as code, we want to keep delivering new features for our users. We also want to keep our deployment pipeline simple and deploy all our stacks and resources using a unified approach.
Import resources to CDK
The cloudformation-include.CfnInclude
module allows us to import resources from a CloudFormation template, and use them in our CDK application as if they were originally defined using L1 constructs. The module essentially adds an AWS CDK API wrapper to the resources defined in the template.
Let’s take a look at how it works. First, we implement a CDK Stack for our static website and then import the CloudFormation template.
from aws_cdk import Stack
from aws_cdk import aws_cloudfront as cloudfront
from aws_cdk import aws_s3 as s3
from aws_cdk import cloudformation_include as cfn_include
class StaticWebsiteStack(Stack):
def __init__(self, scope, construct_id, **kwargs):
super().__init__(scope, construct_id, **kwargs)
self._bucket: s3.IBucket = None
self._distribution: cloudfront.IDistribution = None
template = cfn_include.CfnInclude(
scope=self,
id="StaticWebsite",
# Point to the location where we store our CloudFormation template
template_file="./libs/website.template.yaml",
# To keep logical IDs of the resources, the `preserve_logical_ids`
# flag must be set to `True` (default value). If set to `False`,
# CDK will generate unique logical IDs and try to redeploy them.
preserve_logical_ids=True,
# Here we can provide parameters that will be passed to the
# CloudFormation template.
parameters={
"Environment": "Dev",
},
)
# Get the bucket using its logical ID and `get_resource` method
cfn_bucket = template.get_resource(logical_id="WebsiteBucket")
# Transform the L1 construct (CfnBucket) to IBucket (proxy object)
self._bucket = s3.Bucket.from_bucket_name(
scope=self,
id="WebsiteBucket",
bucket_name=cfn_bucket.ref,
)
# Get the distribution using its logical ID and `get_resource` method
cfn_distribution = template.get_resource(logical_id="CloudFrontDistribution")
# Transform the L1 construct (CfnDistribution) to IDistribution (proxy object)
self._distribution = cloudfront.Distribution.from_distribution_attributes(
scope=self,
id="CloudFrontDistribution",
distribution_id=cfn_distribution.ref,
domain_name=cfn_distribution.get_att("DomainName").to_string(),
)
@property
def bucket_arn(self):
return self._bucket.bucket_arn
@property
def distribution_domain_name(self):
return self._distribution.distribution_domain_name
We have referenced the S3 bucket and CloudFront distribution using get_resource
method and transformed them to L2 constructs which grant us access to some of the convenience methods such as S3 bucket permission grants. Straightforward.
But there’s an important aspect of importing resources I want to go through.
Proxy objects
If you look carefully at the previous code example, you will notice that the static methods from_bucket_name
and from_distribution_attributes
don’t return instances of type s3.Bucket
and cloudfront.Distribution
. Instead, they return instances of s3.IBucket
and cloudfront.IDistribution
(notice the “I”) respectively. They are what AWS calls proxy objects.
L2 constructs created from L1 constructs are proxy objects that refer to the L1 resource, similar to those created from resource names, ARNs, or lookups. Modifications to these constructs do not affect the final synthesized AWS CloudFormation template (since you have the L1 resource, however, you can modify that instead).
Reference: docs.aws.amazon.com/cdk/v2/guide/cfn_layer...
There are a couple of points I want to touch on because they might be confusing. I will talk specifically about S3 buckets but the same applies to other resource types too.
Both
s3.Bucket
ands3.IBucket
are considered L2 constructsYou can transform
s3.CfnBucket
(L1) intos3.IBucket
→ creating a proxy objectBut, you can not transform
s3.CfnBucket
intos3.Bucket
Modify resources imported using CfnInclude
The resources imported using the CfnInclude
module will become a part of (will be managed by) our CDK app and can be accessed using L1 constructs. Because we have access to these L1 constructs, we can modify their attributes.
template = cfn_include.CfnInclude(
scope=self,
id="StaticWebsite",
template_file="./libs/website.template.yaml",
preserve_logical_ids=True,
parameters={
"Environment": "Dev",
},
)
cfn_bucket: s3.CfnBucket = template.get_resource(logical_id="WebsiteBucket")
# ✅ This will work perfectly fine because we're using an L1 construct
cfn_bucket.bucket_name = "updated-through-l1-construct"
But we can't modify the S3 bucket through its proxy object. In the following code example, we transform the s3.CfnBucket
into the s3.IBucket
construct (proxy object) and try to modify the bucket’s name. This code will throw an error.
template = cfn_include.CfnInclude(
scope=self,
id="StaticWebsite",
template_file="./libs/website.template.yaml",
preserve_logical_ids=True,
parameters={
"Environment": "Dev",
},
)
cfn_bucket: s3.CfnBucket = template.get_resource(logical_id="WebsiteBucket")
self._bucket: s3.IBucket = s3.Bucket.from_cfn_bucket(cfn_bucket)
# ❌ AttributeError: property 'bucket_name' of '_BucketBaseProxy' object has
# no setter
self._bucket.bucket_name = "updated-through-proxy-object"
To sum up, if we import resources from a CloudFormation template using the CfnInclude
module, we have access to L1 constructs which we can use to modify the underlying resources. However, the underlying resources can’t be modified through their proxy (L2) objects.
Modify resources using lookup methods
Contrary to resources imported using CfnInclude
module, resources imported using lookup methods will not become a part of our CDK app—CDK won’t manage them. Therefore, any changes made to the proxy objects will not affect the existing resources. These proxy objects also don’t hold a reference to an L1 construct.
imported_bucket: s3.IBucket = s3.Bucket.from_bucket_name(
self,
"S3BucketFromMyAwsAccount",
bucket_name="cdk-hnb659fds-assets-111122223333-eu-central-1"
)
# ❌ AttributeError: property 'bucket_name' of '_BucketBaseProxy' object has
# no setter
imported_bucket.bucket_name = "updated-through-proxy-object"
# This will throw an error too because `imported_bucket` is a proxy object
# without a reference to an L1 construct (no default_child).
imported_cfn_bucket = imported_bucket.node.default_child
# ❌ AttributeError: 'NoneType' object has no attribute 'bucket_name'
imported_cfn_bucket.bucket_name = "updated-through-l1-construct"
So, to reference an already existing resource from our AWS account, we can use a static lookup method (assuming the resource supports it) to create a proxy object. This proxy object, however, can’t be used to modify the underlying resource because CDK doesn’t manage it. If we want to bring an existing resource into our application, we can use cdk import
instead.
Use convenience methods exposed by proxy objects
Although we can’t use proxy objects to modify the underlying resource, we can use their convenience (L2) methods such as S3 permission grants.
In the next screenshot, you can see some of the L2 methods the S3 proxy object exposed for creating an EventBridge rule that would get triggered on various CloudTrail data events.
Deploy a stack with imported resources
We can now reference the StaticWebsiteStack
stack in our app.py
file and deploy the template as a part of our CDK application.
⚠️ If we already have an active stack deployed from the CloudFormation template, we don’t want CDK to create a new stack. Instead, we want CDK to take over the existing stack and manage its resources. We can do that by explicitly setting the stack_name
attribute to the name of the existing CloudFormation stack.
from aws_cdk import App
from libs.static_website import StaticWebsiteStack
app = App()
static_website_stack = StaticWebsiteStack(
app,
"StaticWebsiteStack",
# Setting the `stack_name` attribute to the name of the existing
# CloudFormation stack.
stack_name="StaticWebsiteBucketManual",
)
When we run cdk diff
, CDK will see that there is already an active stack and compare the synthesized template against the deployed template. The first time we try to deploy our stack, CDK will create the BootstrapVersion
parameter (that’s expected) and our output should look similarly to this:
We can now execute cdk deploy
and CDK will take care of importing the resources from the template and deploying the changes against the existing stack. That’s it, easy!
Conclusion
We discussed how the CfnInclude
module allows you to import, manage, and deploy CloudFormation templates using CDK. We mentioned some of the most common use cases for using this approach. And last but not least, we talked about proxy objects and their limitations.
✋🏼 Before you go
Thank you for reading all the way here. If you enjoyed this blog post, please feel free to share it or share feedback with me. You can reach out and connect with me on my LinkedIn http://linkedin.com/in/imduchy.