Improving team ownership of testing

2. Implementing your plan to get everyone in the team to test

Once you have completed the preparation phase, it is time for planning and executing the implementation phase with your team. This phase involves putting your prepared strategies into action and ensuring that everyone is on board with the plan. Here are some critical components to focus on during the implementation process

Fig 3. Create and implement a plan for stronger testing practices across your engineering team

1. Identify what to test: Work with your team to identify which tests you will write. Do not fall into the trap of simply starting with the latest tickets or areas you plan to refactor next. Your testing time is limited and should be used where it has the largest impact. We recommend sitting with your team to answer questions like:
   • which areas routinely have client facing issues?
   • what areas are hard to test on a developer machine?
   • where do you expect the most incremental changes to happen?
   • any routinely brittle areas?
   A common mistake: Usually when a development team answers these questions, they try to solve the problem by changing code. Resist that instinct. Instead, invest your time to putting tests around these critical areas.
2. Set realistic testing goals: This is a broad topic, but we will give you a brief overview of our perspective. First, here are some testing goals and approaches that tend to fail: focusing solely on the number of tests, overemphasizing one type of test (e.g.: unit tests), and writing tests only for the newest tickets. To simplify greatly, we believe most companies progress through these maturity levels in terms of test coverage.

   Fig 4. Evolution of test coverage at companies

   For early stage products, we recommend that you aim for 'multi-layer sparse' test coverage level. A 'multi-layer' defense means adding very few UI tests, some API tests and some regular exploratory testing to your existing unit tests. But it also means a whole lot more. There are other things for your developers to work on - like creating synthetic data for repeatable tests, virtualizing services to enable developers to test on their own machines, data quality checks, infrastructure tests, contract tests to decouple teams, alerting and monitoring, etc.
3. Choose a testing stack: Modern testing stacks can be hairy and hard to select. To keep this section short, we will say that you minimally need to consider if the testing stack is being chosen for a team or multiple teams. Stay away from shiny new objects. When a tool is chosen, figure out if it was chosen because it was easy to get started with or because it has legs. Do not build frameworks before writing enough tests.

   Fig 5. Typical toolbelt that Qxf2 QA engineers work with (as of 2023)

   Well, we have so much experience on this topic that we could write a book filled with counter-intuitive insights. If you do need help at this stage, just write to [email protected] and we will be happy to share our expertise.
4. Coaching your team on testing: Work on increasing your team's testing IQ. Unsurprisingly, developers rely heavily on writing code and automating tests. But they often spend time writing tests that do not matter. While that is better than no tests, you are not getting the most out of the time developers spent on testing. We find it effective to have continuing discussions around thinking critically about testing. We have a starter list of such topics that we customize to suit the teams we work with. Reach out to us if you need ideas on what topics to discuss.

3. Risks when you want your entire team to own quality

All strategies come with risks. We have a large list of risks. To keep things simple, we have listed the top 5 risks you need to be aware of when pursuing team ownership of your testing.

Fig 6. Risks when you try to make testing a team responsibility

1. Conflicts among developers: Conflicts are a natural part when introducing new responsibilities like testing. Your approach will depend on your leadership style, but it is crucial to be transparent and honest about the challenges and reasons behind the changes, as this builds trust. Encourage collaboration by fostering an environment where team members feel comfortable sharing their thoughts on testing and improving testability. This can prevent conflicts from escalating and turn them into opportunities for team growth.
2. Unpleasant surprises: Unpleasant surprises can occur during and after testing. Quality of testing is inherently hard to verify, and a false sense of security can develop as more tests are added. Be wary if there are no negative updates, as this can be a red flag. On the other hand, team members vocalizing problems is a positive sign, indicating they are engaged and proactive in addressing issues.
3. Excuses: Some team members may know how to avoid testing responsibilities or shift them onto junior team members. Be prepared for this and hold people accountable. Establish clear expectations and consistently enforce them to ensure everyone participates in the testing process.
4. Bystander effect: The bystander effect occurs when everyone is responsible, leading to no single person taking accountability and leadership. To address this, acknowledge and reward proactive behavior, rotate responsibilities to ensure everyone is actively involved. This can prevent tasks from slipping through the cracks and ensure accountability across the board.
5. Team composition: Team composition can pose challenges. Junior members might need guidance as some skills take time to learn. High attrition can disrupt team dynamics, making it hard to integrate testing when team members change frequently. In high-growth teams, it is essential to modify onboarding processes to include testing responsibilities from the outset. These changes in team structure and onboarding can help overcome challenges and smoothly integrate testing into the workflow.

4. Other testing options for early-stage products and teams

When things go bad, we see many companies try to tackle the problem by adding surface level testing. This is usually in the form of hiring a relatively inexperienced tester. The problem is much deeper and involves getting the team back on track to owning testing and quality. Qxf2 usually goes through a more thorough but quick 5-layer analysis to arrive at a suitable testing strategy. A lot of our work in such situation involves setting up critical defenses, improving testability, selecting the right technical stack for your team and then enabling your developers to write tests that matter. You could consult with us or hire a senior tester and give them the goal of getting your team to own the testing.

Note: Unlike many software testing providers, Qxf2 focuses solely on testing early-stage products. We do one thing and we do it well - that is provide testing services for startups. If you are a startup or working on an early-stage product that is gaining traction, we have several customized QA service offerings that you might not be aware of. The offerings are tailored to be front-loaded, fractional and involve options that are not usually available in the market. We encourage you to visit our homepage to learn more about our expertise in startup QA and how we can help you.

Closing thoughts

This article was written by Avinash Shetty and Arunkumar Muralidharan. Our understanding of getting teams to own testing has accumulated over the years. One article cannot capture our ongoing journey of learning and improving.

Fig 7. Testimonials from Qxf2 clients showing our evolution in testing approach

We have had to leave out a LOT of what we know just to keep this article to a reasonable length. So if you want us to elaborate on anything in this article or want to share your own experience on the topic, please reach out to Avinash ([email protected]) or Arun ([email protected]).