Macro repository with DOCs, IaC, APP, and much more for Terraform TDD PoC `:wq`
Report Bug
·
Request Feature
"In software development,
TDDis the process of writing tests and then developing functionality to pass the tests. WithTDD, only necessary functionality is implemented, cleaner code can be developed, and tests can be used as documentation for how something should work. But to do this, you need to know how to applyTDDto infrastructure and see if you realize the same benefits." Rosemary Wang
Write tests first, then write code (feature) that passes the tests!
In other words:
- Write the test
- Make the code fails the test (RED)
- Make the code passes the test (GREEN)
- Repeat steps 2 and 3 until the code is stable (REFACTOR)
- Code is build for defined features (less code, less bugs)
- Early bug notification
- Better Designed, cleaner, and more extensible code
- Confidence to Refactor
- Good for teamwork
- Good for code review
- Good for code sharing
- Good for code versioning
- Code is "easier" to read and understand
- Code is "easier" to maintain and evolve
- Code is "easier" to test and debug (testable)
- Code is "easier" to reuse (modular)
The real key here is CONFIDENCE! = MINIMAL RISK
"The test pyramid is a way of thinking about how different kinds of automated tests should be used to create a balanced portfolio. Its essential point is that you should have many more low-level UnitTests than high level BroadStackTests running through a GUI." Martin Fowler
The following diagrams show how the Test Pyramid works on a regular SDLC.
- Lack of knowledge about testing. No clear way to test
- Tooling
- Slow CI servers, flows, feedback loops, and other bottlenecks
- Lack of automation
- Lack of confidence in the testing process (flaky, intermittent, etc.)
- Bad architecture/ Code is hard to test
- Maintenance and evolution
- We are humans, We Make Assumptions
While self-built IDPs have been around in elite teams for around 5 years, they’re now going mainstream in 2021, embraced by huge enterprises like Spotify, Airbnb, and Zalando.
TLDR; Internal Developer Platforms (IDPs) are configured by Ops teams and used by developers. Ops teams specify what resources start up with what environment or at what request. They also set base-line templates for application configurations and govern permissions. This helps them to automate recurring tasks such as spinning up environments and resources and makes their setup easier to maintain by enforcing standards. Developer teams gain autonomy by changing configurations, deploying, spinning up fully provisioned environments, and rollback. IDPs can be built or bought.
"Policy-based control for cloud native environments. Stop using a different policy language, policy model, and policy API for every product and service you use. Use OPA for a unified toolset and framework for policy across the cloud native stack. Whether for one service or for all your services, use OPA to decouple policy from the service's code so you can release, analyze, and review policies (which security and compliance teams love) without sacrificing availability or performance." Cloud Native Computing Foundation (CNCF)
You can't "test" an entire end-to-end architecture. Instead, break your IaC into smaller pieces (modules) and test them individually. Yevgeniy Brikman
"Static Code Analysis commonly refers to the running of Static Code Analysis tools that attempt to highlight possible vulnerabilities within ‘static’ (non-running) source code by using techniques such as Taint Analysis and Data Flow Analysis." OWASP
- Style Guide: Check a set of conventions (sometimes arbitrary) about how to write code for that project. It is much easier to understand a large codebase when all the code in it is in a consistent style. Some tools:
- ✅
terraform fmt - ✅
terraform-docs - ✅
editorconfig-checker - ⚙️
terraform-graph-beautifier - ⚙️
terraform-visual
- Compiler/Parser/Interpreter: Statically check the code for syntactic and structural issues. Some tools:
- ✅
terraform validate
- Linters: Statically validate your code to catch common errors. Linters provide crucial information to speed up debugging and save time in your development. Warn about depreciated syntax and unused declarations, enforce best practices and naming conventions. Some tools:
- ✅
tflint - ✅
tfsec - ✅
terrascan - ✅
conftest - ☑️
sentinel - ⚙️
tfupdate - ⚙️
commit-lint - ⚙️
secretlint - ⚙️
misspell - ⚙️
super-linter - ⚙️
megalinter
- Dry run: Partially execute the code and validate the
plan, but don't actually deploy. It looks for misconfiguration that may lead to security or compliance problems. Changes made by Terraform may alter the status of resources hosted by a cloud provider. Some tools:
- ✅
checkov - ✅
terraform-compliance - ✅
conftest - ☑️
sentinel - ⚙️
infracost - ⚙️
regula - ⚙️
kics
*BONUS ✅ pre-commit-terraform: Setup pre-commit git hooks to take care of Terraform configurations (auto-format, validate, update docs).
Test a single "unit" works in isolation. Break your infra code into small modules and unit test those. There's no pure unit testing for IaC. Yevgeniy Brikman
- Analyze the code and find the most important things to test. Focus on: Logic, Standards, Architectural Conformance, NO Execution, NO dependencies.
- Write a test for each of those things.
- Run the test and see if it passes.
- Repeat steps 2 and 3 until the code is stable.
- Refactor the code to make it easier to test.
Some tools:
- ✅
conftest - ☑️
sentinel - ⚙️
clarity
"In simple words, it is a testing of integration points in a distributed setup following a common agreement." Martin Fowler
"We know that the Terraform code works (the Hashi guys are doing a great job). What we don't know is if we put it incorrectly... or if we put it right." Rosemary Wang
- Analyze the code and find the most important things to test. Focus on: Check Interactions Between 2 Modules, Input & Outputs, Compare the expected STATE, NO Execution, NO dependencies.
- Write a test for each of those things.
- Run the
terraform planto reflect the desired state. - Run the test on the
terraforn plan outputand see if it passes. - Repeat steps 2 and 3 until the code is stable.
- Refactor the code to make it easier to test.
Some tools:
- ✅
conftest - ☑️
sentinel
"Integration tests determine if independently developed units of software work correctly when they are connected to each other. The point of integration testing, as the name suggests, is to test whether many separately developed modules work together as expected." Martin Fowler
"You can test units in isolation from the outside world. But 99% of infrastructure code is about talking to the outside world. So you can only test infra code by deploying to the real environment." Yevgeniy Brikman
"When using Terraform, you can test the behavior of your infrastructure by deploying it to a real environment. using Iac tools Like Terraform, testing de deployment is redundant. Is Covered by Hashi guys. focus in your own business interactions (Functional, Policy & Security, Use Cases)." Rosemary Wang
- Analyze the code and find the most important things to test. Focus on: Confirms Interactions Between 2+ Modules, Execution, Dependencies, Resources, Isolate sections of system.
- Write the module examples of those things.
- Write a test for each of those examples.
- Run the
terraform applyto deploy real infrastructure. - Run the test directly on the Cloud Provider to validate it works.
- Run the
terraform destroyto undeploy the infrastructure.
Some tools:
- ✅
terratest - ⚙️
kitchen-terraform
*BONUS ⚙️ aws-nuke: Use a sandbox or playground account and remove all resources from an AWS account nightly.
You will need the following things properly installed on your computer.
Contributions are what make community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please create a pull request.
Don't forget to give the project a star! Thanks again!
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m '✨ #101: Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
- Andrés Amado - @acactown - acactown@gmail.com
Project Link: https://github.com/acactown/terraform-tdd
I've included a few of my favorite sites to kick things off!