Startups struggle to hire good QA. We think it is because they interview QA engineers poorly. Common mistakes include asking knowledge-based questions, using unclear/overloaded language, asking candidates to test things without giving them any context whatsoever (e.g.: how would you test a chair?). We think there is a better way.
We have interviewed hundreds of QA engineers and successfully setup QA teams at several startups. We have also made plenty of mistakes along the way. Here are some tips on how to interview QA engineers effectively.
1. Make the candidate test a highly limited application
Good testing relies on context to give it focus. A generic app (Gmail/Google News) is too large to evaluate whether the QA is good at testing or not. Instead, make the candidate test a highly limited application. Think of this exercise as Fizz-Buzz for QA engineers. If you don't have one, use this simple factorial calculator that we developed. You can repurpose it to ask more technical questions too (e.g.: develop a script to for this application).
Note: We recommend this test even if you are hiring QA automation engineers.
2. Rather than ask knowledge-based questions ...
... consider giving them 30-minutes to learn and apply a new topic. A QA engineer's knowledge is patchy and somewhat like a hunter's knowledge of a forest - extremely good in some parts and totally non-existent in others. So, it is not always fair to ask QA engineers knowledge-based questions. Instead, try evaluating their ability to learn and apply something new. For example, we show/demo Google Analytics to our candidates and then ask them situational questions like "I'd like to deploy our application to production on a Monday morning. Are you ok with that?" (Hint: If Google Analytics says your traffic is pretty high at that time, it is probably not a good time to deploy)
3. Ask about their previous experience
Most people do this naturally. We have some extra ideas to dig deeper into a tester's experience:
4. For senior candidates, don't give too much context
Well, don't give too much context right away. See if they are able to ask good, clarifying questions. You will appreciate this habit once they are hired, especially if you often receive poorly scoped work. One hack to implement this suggestion is to have a non-engineering role interview the QA.
5. If the candidate seems tense ...
We have observed this often - QA engineers do not interview well. So, if you find the candidate is tense, spend a couple of minutes asking simpler questions that have no binary right/wrong answers to calm them down. For example,
a. What do you think makes a good QA?
b. What was the most interesting bug you ever caught?
6. Suggested structure of the QA interview
| Round | Suggested time | Notes | |
|---|---|---|---|
| 1. | Background | 30 minutes | - Learn about the candidate's background - See point 3 above for ideas |
| 2. | Testing skills | 45 minutes | - Learn how the candidate approaches testing - See point 1 above for ideas |
| 3. | Learning skills | 45 minutes | - Is the QA hesitant to learn new things? - See point 2 above for ideas |
| 4. | With non-engineers | (2 or 3) 15 minute slots | - Typically with product owners, Ux and support - Just let it happen. Engineering needn't control this |
| 5. | Culture fit | 15-30 minutes | - Typically with the VP of engineering - Most startups already do this well |
These are some things that we think will help you interview QA engineers better. This article was written by Avinash Shetty and Arunkumar Muralidharan. If you have questions or comments, email Avinash ([email protected]) or Arun ([email protected]).
© Qxf2 Services 2013 -
