In this post, we will outline our usual strategy to get an entire team to quickly fill testing holes – like writing infrastructure tests. Why? Because we find ourselves using this approach frequently at clients. Some context: Qxf2 engineers work with early stage products that neglected writing tests in favour of shipping quickly. This means, we almost always enter situations where there are many holes to fill when it comes to testing.

The teams we enter always seem to be playing “catch up” when it comes to writing tests. We like keeping the number of testers low but expand the footprint of testing within teams. So, how do we do it? By involving the whole team. But involving an entire team in writing tests is easier said than done. There is some technique and nuance. In this post, we will cover our typical strategy.

1. Background

Since we cannot reveal what we do at clients, let us take an internal example at Qxf2 to explain the process. At Qxf2, we didn’t have any tests in place for the things we had set up on AWS. Sometimes, we’d make changes to our infrastructure, update operating systems, upgrade Python/node versions, or update security patches, but we couldn’t be certain if everything would still run smoothly on AWS. This is the sort of testing hole that is very normal in early stage companies.

2. Identify high value test scenarios

Since you are going to be asking for time from the entire team, you need to prepare yourself. First, identify high value tests and make a list. In our case, we determined the most important aspects of our infrastructure that require good amount of testing. These included security configurations, network setups, resource provisioning, and compliance standards. Identifying test scenarios involves pinpointing the critical parts of your infrastructure that need testing.

3. Select the Right Tool: Chef InSpec

The second preparatory task is to choose a tool (or tech stack) for your tests. For our choice of tool, we’ve drawn upon insights from our previous two blogs on Chef InSpec. These blogs provide compelling reasons for selecting Chef InSpec as our preferred tool for IaC testing.
a) Auditing OS level resources with Chef InSpec
b) Auditing AWS cloud resources with Chef InSpec

4. Figure out how the team will collaborate on tests

The third preparatory task is to figure out how the team will collaborate. This step usually involves a tool like Jira to track the work and a version control system to collect the code being written. In our case, since most of our colleagues had context, we simply created a GitHub repository. The below providing the high level folder structure to store our IaC tests for various applications.



The folder structure and the initial common code in the repository matters. This involves simulating the entire exercise in your imagination and then figuring out how to make the test writing process simple and efficient. You will notice that in our case, the directories were structured so people could independently tackle different tests. That way, even if one or two folks were slow, they did not hold up the entire group.

Note: At Qxf2, we used a tool called “Chef InSpec” to write tests for our applications on AWS. Each application had its own folder structure(above depicted), which is also known as a profile name in Chef InSpec terms. I had already explained about profiles in my previous two blogs.

5. Allocate sprint time

Now, that you are ready, speak to your product owner. Explain how you could spend a long time writing the tests in between your testing OR they could allot just 2 hours per sprint per person and get the backlog done much quicker.

6. Using the allotted sprint time

In our case, we used the 2 hours with the team as follows:

6.a. Example Walkthrough (1 Hour):
I spent one hour showing the team practical examples of how to write IaC test cases. I even setup few example tests in the GitHub repository that I had made. At the end of the session, I distributed the different tests I had identified in Step 2 to different team members. Each person got about 2 tests. Folks had to somehow find time to implement those tests in the next sprint. Since you are working with developers, there is not much else you need to do. They can read code and probably have better standards that you – so that part is hopefully easy!

6.b. Implementation (3 Hours):
This is when the team begins implementing IaC tests for various internal Qxf2 apps hosted on AWS. I kept myself open for any questions/doubts and would follow up during standups to see if my colleagues had any trouble. This is an important step. Engage the developers and make sure you unblock them quickly.

7. Review PRs quickly

If your development team has taken the time to write tests, then the onus is on you to respect their time and deal with their PRs quickly. In my case, after writing the tests, team members submitted individual Pull Requests (PRs). These PRs underwent a code review process. Usually, you will have only two decisions to make here:

7.a. Merge after Code Review
If the PR looks good, merge. But do not stop there. Thank the developer in a public channel. Visibility and positive feedback about tests is a good way to integrate testing within your team’s daily routine.

7.b. Feedback and Code Corrections
If the code is not good enough or the tests written were not good enough, please bring that up to the developer. Developers are not always thinking about the tests from the same perspective as a tester. So make sure you put across your point of view. Again, the feedback has to be quick. One of the worst mistakes you can do is to let the PR sit unreviewed for days together.

8. Integrate with CI/CD pipeline

I am repeating myself but making tests visible and live is a good way to foster interest within the team. So make sure you integrate these tests with your CI/CD pipelines. At Qxf2, one of my colleague took the initiative to create a GitHub Actions workflow specifically tailored for running Chef InSpec tests. This means that whenever there’s a change in your infrastructure code, GitHub Actions will automatically trigger the workflow to run the associated tests. Along with it he has also written blog to address “Troubleshooting Common Issues when using Chef InSpec with GitHub Actions“.

Conclusion

This post might seem straightforward. But each step requires multiple skills! You need to be able to take technical decisions, influence developers, negotiate time with the product owners, react quickly to questions and blockers, etc. When implemented right though, the pay off is huge. Your entire team is empowered to continue writing tests and equally importantly, maintaining them!

Hire Qxf2

Qxf2 likes working with small teams and testing early stage products. We integrate well with team and enable developers to test better. We are among a small set of companies that like to work with teams, help them implement the right testing strategy and then leave. If you are looking for short-term testing leadership on your project, please reach out.

---