An introduction to R&D at Qxf2

As a techie, you cannot escape the buzz around IoT, elastic cloud computing, data analytics, machine learning, AI and the like. Most of the deep testing we would like to do in these areas lack easily accessible knowledge sources and definitely lack good tooling. We are sure that larger companies have faced many of these challenges and have arrived at a number of solutions. But unfortunately, we haven’t found solutions available in the public domain. So we have decided to devote R&D cycles to tackle some of these problems. I am sharing our plan online in the hope that smarter minds than us can provide feedback on the path we have chosen to travel.


The risk and benefits of R&D

We know what you are thinking …

Spending money on open-ended R&D could seem risky for a small company like ours. But we are OK with investing on R&D because:

  1. it gives us a chance to do creative work (employee morale ftw!)
  2. it helps us get familiar with the latest technologies (… even if we produce nothing worthwhile)
  3. we may end up attracting clients who are already working in these areas
  4. it could help us hire engineers with similar ideas
  5. we could learn about existing solutions. We bet bigger companies have already taken serious shots (or solved) at a bunch of these problems
  6. we have already established a culture of R&D (look at this blog!) and directly benefited from it
  7. we are in a decent position financially and can take this risk

Strategy

It is hard to come up with strategy when so much of what we are trying to do is new to us. Currently, we have identified two approaches:

  1. Figure out where we can help. There are many skills somewhat tangential to the primary development skills in each of these emerging areas. For example, scraping, cleaning and maintaining data could be a valuable tangential skill to people working on Machine Learning algorithms. This could be a good entry point for the motivated tester.
  2. Look beyond the engineering solution. To be useful as a tester in emerging areas, we think the following thought experiment will help.
    1. Spot a technical problem
    2. Assume the problem is going to be solved by someone else
    3. Imagine how we will test the solution

Obviously, just doing this much will not be enough … BUT we think this will give us a good starting point.

Note: We needed to make organizational changes to support this kind of work. We’ll follow up with posts about how we are experimenting with our hiring, on-boarding, training and team structures.


R&D has already helped us

We have had at least one person dedicated to R&D since mid-2014. R&D has worked well for us. It has helped us:

  1. land new clients
  2. look smart and prepared when we go to a new client
  3. allowed us to train new hires quickly
  4. helped retain technically inclined testers who do not get similar opportunities elsewhere
  5. expand our skill set beyond that of a typical tester
  6. gain confidence that we can solve problems and pick up new things quickly

What we have learned so far

This cartoon is more autobiographical that you suspect

Over the years, we have learned a few things (mostly via trial and error) about running an R&D unit within a services company:

  1. Prioritize interesting over useful: We seem to get a lot more ideas by doing something that interests us rather than have a concrete goal. We stumbled upon a nice talk called Discovery without objectives that reinforced this belief in us. If you disagree with this point, we encourage you to watch the video and think about the arguments made in it.
  2.  

  3. Have roadmaps: Don’t bother being accurate or concrete (see examples in our next posts) and do not hold people accountable to achieving all items on the roadmap. Instead, use the roadmap just to get your colleagues started. Once people get started, they generate their own ideas and next steps.
  4.  

  5. Split work into small pieces: Make an attempt to break work into really small chunks. Ideally, each chunk of work should be done within one week. When a new hire is introduced to the concept of R&D, no matter how smart, productive and independent they are, they want to be able to show visible progress. They want to know what they are doing is useful in the short term. Giving them small victories is key to building their momentum.
  6.  

  7. Share the work: Making people share their work (e.g.: write blog posts, use public GitHub repos) forces them to articulate their work better. The shared output will serve as good reference material for your training program. Sharing work openly also results in Internet strangers wanting to collaborate and help you. Your employees feel good when strangers around the world benefit from their work and recognize them for it.
  8.  

  9. Take time to come up with good examples: We imposed this constraint to get over the habit of people doing things only at the surface level. We wanted to avoid producing tutorials and examples that end up being too simple to be useful. Coming up with a detailed (and interesting) example that helps the learner feel like they have made substantial progress is key. We make our employees come up with examples related to their hobbies. For example, if our colleague Shiva (an Arsenal fan) works on data scraping, he would pick an example of scraping data from the English Premier League.
  10.  

  11. Tolerate failure: Be patient. Let your employees know it is ok to fail on R&D. Most work that is produced will likely generate plenty of dead ends. No single person will have all the answers. People will get frustrated when they get blocked. We need to manage that and intervene and reinforce the belief that failing on R&D is natural.
  12.  

  13. It is difficult to onboard people: While our R&D sounds cool, our experience is that most employees struggle to adapt to the idea of doing something open ended. It takes an experienced person about 6-8 months(!) before they feel comfortable with R&D. It probably takes longer before they realize its value to the company and to their own resume! So provide a support system for new employees.

  14. Further reading

    Interested in what we are up to? Take a look at some R&D roadmaps we have created:

    1. Qxf2’s DevOps roadmap
    2. Qxf2’s Hardware and Robotics roadmap
    3. Qxf2’s Data analytics, Machine Learning and AI roadmap

    While we have split our R&D efforts into three logical pieces, we would like to stress that we understand that they are all inter-related. The testing we do will combine all these fields. We have chosen this split up just make it easier on the engineer. But when we work on a real world problem, we ignore these artificial lines and do whatever good testing demands.


    Related posts

    1. Where is Qxf2 headed?
    2. The need for change (at Qxf2)
    3. An introduction to R&D at Qxf2
      1. Qxf2’s DevOps roadmap
      2. Qxf2’s Hardware and Robotics roadmap
      3. Qxf2’s Data analytics, Machine Learning and AI roadmap
    4. An introduction to training at Qxf2
    5. An introduction to hiring and onboarding at Qxf2
    6. Experimenting with team structures at Qxf2

    Arunkumar Muralidharan

    I want to find out what conditions produce remarkable software. A few years ago, I chose to work as the first professional tester at a startup. I successfully won credibility for testers and established a world-class team. I have lead the testing for early versions of multiple products. Today, I run Qxf2 Services. Qxf2 provides software testing services for startups. If you are interested in what Qxf2 offers or simply want to talk about testing, you can contact me at: mak@qxf2.com. I like testing, math, chess and dogs.

2 Comments

  1. Helge said:

    Very interesting reading material. How do you decide the next step? What % of your annual budget do you spend on R&D?

    Sorry if this is rude. I love what you are doing.

    October 11, 2018
    Reply
    • Thanks for the question. I am genuinely surprised people are reading this article 🙂

      How do you decide the next step?

      I don’t really know how to do this optimally or even well! In fact, we struggle mightily here. Our highest ideal is to base it on what is interesting to us at that time (see the video ‘Discovery without objectives’ linked in the post). But we often fall short of this ideal because we have shared many things that require development/maintenance and we are also working hard on cool projects at our clients.

      As nice and interesting as this article may sound, Qxf2 is far from ideal. I have just listed what we have tried and what we want to try. But it is worth repeating and re-repeating that we are not experts and we don’t know if what we are doing is going to work out or not.

      What % of your annual budget do you spend on R&D?

      I cannot answer this directly because we are so small and that leads to too much variance. How much we dedicate to R&D majorly depends on how much client work we have in a year. Over the years, it has been significantly more than the ~15% that big companies invest in R&D. The way I manage money is that I take a salary that is sufficient for my living needs plus some to invest in my retirement funds. The rest of the profit is simply ear-marked to hire new employees and support R&D.

      October 12, 2018
      Reply

Leave a Reply

Your email address will not be published.