How to Grow Your Terraform Projects
Terraform is an open-source infrastructure that manages cloud services, making it easier for thousands of people to manage the infrastructure of their programs. Its consistent concept makes it easy for even the layperson to understand, making it so popular in DevOps circles.
Terraform uses declarative files to write Infrastructure as Code, a much simpler way to work than extensive coding. Users can also see the changes in real-time to ensure everything follows the code.
However, creating it on a larger scale is difficult if you don’t know what you’re doing. Increasing code complexity and Terraform dependency become much more challenging to sustain. Ensuring you can rely on your code in the most crucial times becomes even more critical when significantly scaling your platform.
So, how can you grow and scale your Terraform project?
Working Locally
Using Terraform on the same system that runs the original developmental code makes transitioning much easier. Using Terraform on the same device as the original coding device is even better. Install Terraform on the local computer and gain access to the target provider.
Terraform’s features allow for multiple people to access coding in multiple locations. So, while the primary computer holds the initial coding, others can make real-time adjustments to coding. Terraform has tools to make immediate adjustments to multiple networks, quickly integrating them into the system.
Pros
For developers, access to a single computer means that everyone who has access can jump into work immediately—catching up on adjustments and problems associated with centralized coding.
The integrated adjustments made to Terraform make it ideal for companies to hire developers worldwide using remote state and state locking. If you’re using an alternative version control system, Terraform provides tools to increase integration speed as well.
Cons
Unfortunately, working locally means that everything is manual, which takes a lot of time. If the developer or team is unavailable when networks are down, no one can make changes.
Working with multiple people at once can also cause problems. For example, if someone executes a change to the network while someone else works on it, the network can jam, requiring more time to fix it than it took to implement the changes. Large teams are more prone to these problems, which can cause huge issues for large corporations.
Working with multiple people could also lead to security risks, and all workers need to access a Terraform state. Even when developers pull information from a secured location, Terraform still stores the code in plain text. Anyone with access to the software could immediately pull information from these resources.
Terraform requires manual testing. Excess time designing and implementing tests makes them tedious and takes up network time. You’ll often need outside software to run tests, such as Spacelift test modules.
Implementing Homegrown Automation
Adding Terraform workflow into your in-house CI/CD process, which you already have implemented, is also possible. In this approach, you may keep an eye on everything as the system logs all of the CI/CD pipeline changes. You can examine the data at any time and in real-time.
Pros
It’s not a bad choice if you are worried about security. You will maintain control over everything, including the security setup.
Cons
The most serious disadvantage and concern with this technique is the amount of time and work required to establish a complete pipeline. To create an efficient workflow, you need to think about processes like unit tests, periodical drift detection, or compliance checks.
It’d be an added burden on the team to keep it all in-house, with all of the configurations. Having a built-in automation solution appears appealing initially, but you’ll soon run into several issues.
Using Open-Source
Atlantis is most often associated with Terraform as an automation and collaboration layer. It’s an open-source tool that allows programmers to efficiently manage technology tasks and operations. It relies entirely on pull requests and doesn’t allow mapping projects within the infrastructure.
Terraform, even with Atlantis, has always struggled with handling interdependencies with projects, which is why Terragrunt, essentially a wrapper for Terraform, has become more popular.
While some open-source tools allow for multiple workspaces, most are meant for tests. Small infrastructures use open-source tools, but using Terraform at scale requires a more sophisticated system. Many entrants into the system can cause security risks and can compromise the infrastructure.
Pros
Many open-source tools are free to download. Most of the paid options include trial periods to help you test which tools are most beneficial to your Terraform project.
Open-source tools are open to adaptation so that developers can make changes based on their needs. Open-source users can connect with fellow developers to suggest solutions to problems, which makes the turnaround for improvements extremely quick.
Recruiters often look for top talent from open-source websites, particularly those that make frequent comments on improvements. Recruiters will hire both freelancers and in-house developers to improve current developments.
Cons
Free software takes time to learn, and hiring someone to adapt the free software to a business can cost more money than companies initially thought. While paid software is more expensive, it can save projects thousands in the long run.
If anything goes wrong with your open-source tool, developers have to make their own adjustments. Most open-source tools are online creations and have little to no structure, so the company must find a solution themself.
Licensing is tricky with online open-source tools, so developers take the risk of lawsuits. What’s worse, some open-source tools are free to access by anyone, which significantly reduces security.
Using Specialized Management Solutions
Most of the management solutions with specialized management platforms already include tools for collaboration and security.
Terraform Cloud
Terraform Cloud is far safer than Terraform itself. Its features let developers set organizational standards before creating resources within Terraform Cloud.
Terraform Cloud has an artifact repository for modules that allow developers to roll out previous versions of coding in case updates are insufficient. Every module has its own repository, so backdated coding prevents the loss of information, so module management is much easier in this format.
Terraform Cloud also includes the remote execution backend, giving developers the chance to test their software to ensure it’s actually working. Developers can use Terraform Cloud on their personal computers, upload coding, and test it immediately.
Unfortunately, any code testing while implementing continuous integration requires direct input into the coding infrastructure. Maintaining continuous integration is difficult for a large group of developers, as frequent tests mean temporary setbacks for other coders.
Possibly the biggest Terraform Cloud shortcoming is the lack of periodical drift detection runs. Changes within new software that affect the repository require additional testing to ensure the codebase is up to date. Terraform Cloud can run testing, but it requires a daily manual check.
Spacelift
Spacelift is a flexible IaC management platform. Terraform Cloud and Spacelift have the same functionality in both deployment execution and compliance.
Similar to HashiCorp’s implementation of compliance as code, Sentinel, Spacelift uses Open Policy Agent. That would be beneficial for companies that previously used OPA in their workflow (i.e., for Kubernetes).
Users sometimes need access to a Terraform state directly, and in the case of Terraform Cloud, it would require access to the provider. Spacelift lets you run any command within a codebase with tasks. It processes requests based on previously set permissions, giving users the ability to import, remove, or move resources directly.
Spacelift outpaces Terraform Cloud in its periodic drift detection runs. Developers can schedule drift detection runs, ultimately automating the process of fault detection.
Conclusion
Terraform’s open-source infrastructure provides a great codebase for any project. However, its easy-to-use approach becomes much more convoluted at scale. There are multiple ways to scale your Terraform workflows. Implementing in-house solutions is difficult and could cost more than purchasing additional open-source software. But using management platforms is, by far, the most comprehensive way to grow projects in Terraform.
About the author:
Mariusz Michalowski
Mariusz is a Community Manager at Spacelift, a flexible management platform for Infrastructure as Code. He is passionate about automation, DevOps, and open-source solutions. In his free time, he enjoys car detailing, swimming, and nonfiction books.
