Terraform Architecture
Originally
ADR-0025-TERRAFORM-ARCHITECTURE (v11) · Source on Confluence ↗Terraform Architecture
Summary
In order to minimize the number of touch points for engineering, modules should be consolidated as much as possible in alignment with product and product engineering.
Decision
The highest-order abstraction for Terraform modules should exist at the project level. All resources necessary for the operation of that project should be contained within the project’s infrastructure repository.
To conform with best practices, each environment the project will require should exist within separate submodules. The root module should define all resources that will be repeated between environments. Environment specific configuration should be defined in the root module and passed to variables in the submodules.
Submodules should primarily be reponsible for state, with as little environment-specific configuration as necessary.
State should separated per-environment, as well as at the root level.
Example structure:
project/
main.tf
repositories.tf
prod/
main.tf
gke.tf
service_accounts.tf
cloudsql.tf
doppler.tf
dev/
...Context
The number of resources necessary to manage all of the infrastructure concerns related to a product is growing rapidly. This has the side effect of increasing complexity in managing resources and has a profound effect on the context needed to understand the impact of changes.
To reduce the number of touch points, the context required to track changes between resources, and minimize the complexity in managing infrastructure, the Terraform root module that product developers interact with should be the basis of any and all changes needed for a given product.
Consequences
By consolidating all infrastructure for a product into 1 repo, it can become harder to manage and mitigate organizational risk. Due diligence will be necessary in ensuring basic and useful guardrails exist, and that best practices documentation is built, maintained, and followed.
Alternatives
No Change from Current
The current terraform architectural model is that each resource that might be needed is a module, and each module has a high level aggregate module that defines all the implementations of the low level module.
This does a great job of separating concerns, managing state, and allows for independent development of modules without interconnectivity. However, the drawbacks in maintaining distributed modules, race conditions on updating the modules, and high level context necessary make it burdensome to maintain and interact with.