Infrastructure as Code (IaC) has revolutionized the way that infrastructure is provisioned. In short, IaC is defining your cloud infrastructure (Amazon VPC, subnet, Amazon EC2 instances, security groups, etc.) in a template file or in actual code.

Initially, you could only define the infrastructure in a template using JSON or YAML and then create a stack using AWS CloudFormation. Now, there is another option – the Cloud Development Kit (CDK) – that allows you to write code in common programming languages such as JavaScript and Python to define your cloud infrastructure. Under the hood, the CDK converts the code to an AWS CloudFormation template and then creates a stack from that. No matter which route you choose, IaC provides many benefits such as automation, repeatability, compliance-ready design, and the ability to leverage source control.

Automation

By defining your infrastructure as code with a servicelike AWS CloudFormation you can easily build your entire infrastructure with the click of a button. Before cloud computing platforms, like AWS, the infrastructure team would need to manually spin up each server, configure their settings and services, and install any needed software and packages. This was a manual, time-consuming process with a high risk of human error. By using AWS CloudFormation and its associated helper scripts such as cfn-init and cfn-signal, you can install and configure software packages as the infrastructure is provisioned ensuring everything is built in the correct order.

AWS provides the Metadata section in AWS CloudFormation to define information that can be used to customize the setup of an instance. The AWS::CloudFormation::Init: section under Metadata helps us declare information that we need to help install and configure our instances. For example, we can automate the installation and configuration of a LAMP stack onto our Amazon EC2 instance. As seen below, we declare two configSets: Install and Configure. Under the Install configSet, we declare the packages that we want to install and the package manager we want to use to install them (yum in this case).

Further down in the Amazon EC2 resource definition, the UserData section is where we can define commands to run automatically on startup of an instance. In this case, we update the AWS CloudFormation bootstrap package and then run the cfn-init command, which looks at the AWS::CloudFormation::Init section where we defined the packages that we want to install. It passes in the name of the AWS CloudFormation stack, the name of the resource, the configSets that we want to run and the region as command line parameters.

After the cfn-init command, there is another AWS CloudFormation helper script command called cfn-signal. This command is receiving the output (success or failure) from the cfn-init command and signals to the CreationPolicy if the installation was successful. The timeout in the CreationPolicy section means that AWS CloudFormation will wait for five minutes for a success signal. If it doesn’t receive a signal in that time period, the AWS CloudFormation will stop the stack creation and mark it as “failed to create.”

Repeatability

Once you have defined your infrastructure in an AWS CloudFormation template, you can repeatably create environments anytime. Here at Innovative Solutions, we have a standard networking templates that can be used for any new projects. This removes human error involved with manually provisioning your infrastructure with each new project.

Compliance-ready

By default, an AWS CloudFormation stack allows update actions on all the underlying resources. To solve this, we can define a stack policy that will ensure that the resources in the AWS CloudFormation stack cannot be updated. There are also other tools such as Drift Detection to ensure no one is changing the underlying infrastructure. Ad hoc manual changes to the stack should never be permitted because this could result in a non-compliant environment. Especially for a production environment, all changes should be run through the AWS CloudFormation template via a stack update.

Source control

Another great part of having your infrastructure defined as code is you can check it into source control just as you would with any code. This allows you and your team to be able to see the history of templates and the various changes that happen over time. Also, this allows your team to collaborate on the development of templates.

Organizing and managing templates between teams

When starting out with AWS CloudFormation you will probably put all your resources in one template. However, as your infrastructure gets more complex, this will become unmanageable. For example, a company has three teams working on a given application: a network team, an application development team, and a security team. Each team will have multiple resources that they need to provision for the application. Let’s say the network team needs to make a change to the VPC resource they have defined in the AWS CloudFormation template. If the teams are sharing one template, this could cause confusion and unnecessary overlap. To solve this issue, the best practice is to create three separate templates, one for each of the teams. This way each team can manage their own template without needing to check with and coordinate with the other teams before making changes to their resources.

Certainly, there will be resources that will need to be shared and referenced between the three templates. To solve this, we can use cross-stack references, which allow resources to be exported from one template and imported into another. For example, if the security team needs to reference the VPC defined in the network stack, it can do so by importing the VPC resource (if the network stack exported that VPC resource).

In the network stack, we need to export the ProdVPC resource:

In the application stack, we import the VPC Id for use in defining the target group of our Elastic Load Balancer.

Another best practice is to use nested stacks to re-use templates that are commonly used. Let’s say you have a network stack that is used in all the applications you create. Instead of defining the same network stack in each application template, you can make the network stack its own template, host it in Amazon S3, and then when you require another stack, you can define a resource type AWS::CloudFormation::Stack and then point to the location of the network template.

Here is an example of what this looks like in a template:

We are defining a CloudFormation stack that is referencing a template file that is stored in S3.

How does Innovative Solutions leverage IaC?

Innovative Solutions has been leveraging AWS CloudFormation for years because of the many benefits it provides for our organization. We have developed many different templates for networking, security, and others, that have been incrementally improved through the years. Having mature AWS CloudFormation templates at our disposal makes it easy to build infrastructure quickly and reliably. This allows us to save time and focus on the actual workload.

Our templates are stored in source control so they can be easily updated as services evolve. All past versions are tracked and can be easily updated for collaboration. For each project, we are able leverage our AWS CloudFormation templates to easily deploy multiple identical stacks for each of our environments (dev, staging, production).

Cloud Development Kit

The Cloud Development Kit (CDK) is another excellent way to define your AWS infrastructure as code. In fact, CDK abstracts away a lot of complexity of the AWS CloudFormation template. It allows you to provision AWS resources in popular programming languages such as C#, Java, JavaScript, and TypeScript, instead of creating a separate template file written in JSON or YAML. Using the CDK also allows you to use programming logic (if statements/for loops) that developers are comfortable with to help provision infrastructure resources. Writing ten lines of code using the CDK can produce hundreds of lines of an AWS CloudFormation template.

When you run your CDK app, an AWS CloudFormation template is synthesized (created). This doesn’t create any resources. The cdk deploy command actually creates the stack and the underlying resources.

Below is a sample Python CDK application that creates an SQS queue and an SNS topic. The queue is added as a subscription to the SNS topic so that it will receive messages when they are pushed to the SNS topic.

As seen above, this is simple, easy to understand code to write in Python. These lines of code create a CloudFormation template that is 150 lines long! The CDK provides an amazing level of abstraction that organizations can adopt quickly, if they haven’t already.

IaC has forever changed how we create virtual infrastructure. Once an organization learns how to leverage IaC, they will never go back to manually creating virtual servers and configuring all the settings and services associated with them. Not only is doing all this manual work extremely tedious, it also poses a high risk of human error because of the manual steps involved. With the development of the CDK, there is less of a barrier to entry leveraging IaC at your organization. You can find numerous sample templates online on the AWS website e. There is some up-front work involved with IaC, but once you are up and running you will appreciate the multitude of benefits that come with it.

 

Do you still have questions about Infrastructure as Code (IaC) ?

Feel free to contact us, we’d love the opportunity to further discuss anything you have read.

Contact us for more information