In this post, we’ll briefly outline how our DevOps roadmap looks. We’ll also give you a feel for how work proceeded before we had a roadmap and how it has proceeded after we came up with one.
Why use a roadmap in the first place?
The return on investment on R&D is not obvious. Our R&D work, in the short term, seems to go waste and is largely ignored. On the other hand, employees like to know that they are making progress. They want some validation that the work they do matters. These two conflicting forces make it hard for us to help employees get started on R&D work. When they start, our employees want to know what exactly they will be accomplishing and why will it matter. We caved in and presented people with an inaccurate plan (a high-level roadmap) rather than no plan at all. And surprise, surprise! People had a much easier time getting started with R&D. Once they get started, they ignored the roadmap and generated their own next steps.
Why should a tester even look at this?
Because this is going to be a standard part of your work sometime in the next few years. To learn more, read The need for change (at Qxf2).
The DevOps roadmap
At a high level (blue nodes), here is what we listed:
1. Containers: Shipping an environment along with an application makes so much sense. We think the idea of containers has legs and so we want to explore them.
2. Configuration management tools: We’ll soon have to model systems and be able to orchestrate approximate versions of our production systems. So knowing configuration tools would be a good start.
3. IaaS (Infrastructure as a Service): These are the popular IaaS providers as of today.
4. CI tools: We need to know how to tie our containers and modeled environments into CI tools.
5. Monitoring: The data collected from monitoring tools give additional insights into the health of the application. So we thought we would explore some common monitoring tools.
6. Log parsers: When you have dozens of independent applications glued together, you are going to have a lot of logs and a lot of information is going to be buried in these logs. Figuring out how to effectively filter useful data from multiple log file is going to be a powerful and useful skill to possess.
7. Modelling an application: It’s going to be hard to get a QA environment. We’ll need to be able to mimic/abstract (and not directly use) some components. We need to figure out good tools to mimic/mock/abstract various components of an application.
8. Random tools: We have listed some tools that we have heard of we would like to explore. These are tools we have come across when reading blog posts and would like to try out.
Where we started
We love Python. And we have a decent amount of experience dealing with systems. So we started exploring SaltStack. Progress was remarkably hard. We figured t0 do better if we had something concrete to get going.
Baby step 1 – Running Selenium tests using SaltStack
We decided to do a project that involved one master and one minion. We just wanted the master to command the minion to run a simple Selenium test. We managed to do that quite easily. Our first version was with SaltStack and VMs using Vagrant boxes. That was relatively easy to set up and took us less than a couple of days.
Baby step 2 – Poor man’s BrowserStack with AWS
We built on our previous baby step to extend our work on DevOps. We tried out different IaaS (Infrastructure as a Service) providers including AWS and Google Compute. We liked AWS. We decided to try a highly primitive clone of BrowserStack. We managed to use Selenium Grid, AWS and SaltStack to set up our own cloud-based GUI testing infrastructure.
Baby step 3 – Poor man’s BrowserStack with Docker
Now that we had a firm footing, we expanded our work to include containers. We tried Docker since that seems to be the most popular container technology. During this work, we managed to create and publish a Docker image for Selenium, Python, and ChromeDriver.
Work done since the roadmap
Based on what we picked up during our baby steps, we felt comfortable coming up with a roadmap. A funny thing happened after we came up with a roadmap – we started spotting opportunities to use our knowledge at clients. One of the clients started using BitBucket pipelines and Docker for their continuous integration. We have learned quite a lot from them and managed to integrate our Python-based API test suite with the BitBucket pipelines. Another of our clients (in the IT service management space) had a really complex application that spoke to several different applications and acted as a translator. We ended up making their testing infrastructure generic and available on developer machines by using Docker containers to map some of the devices they were monitoring and translating. We felt comfortable enough about Docker that we integrated Docker with our Python Automation Framework for Selenium and Appium.
Key takeaway: The roadmap helped us get started. We continue to land at places we would not have imagined. But that is ok. Discovering next steps is what happens naturally over the course of work.
Developing a feel for the work
No one person at Qxf2 can be expected to know everything we are trying to cover. So it is important to develop a ‘feel’ for the progress. We feel much better about where we are because it seems to have a number of characteristics that overlap with some of our previously successful R&D work:
a) The work feels like a multi-purpose move. Using IaaS providers and being hands-on with Docker helped us get familiar with DevOps and will help us a model and deploy QA environments. However, it also strengthens our web application and mobile testing framework too!
b) There are ‘familiar’ elements. We have one foot firmly planted in an area we know well. So we aren’t leaping too far out of our comfort zone. The person doing the work has enough support within Qxf2 and on the Internet.
c) The engineer’s resume gets better. Our engineer, if he/she wishes to quit Qxf2, still has a very strong and relevant skill on their resume.
Worth repeating
If you are planning on trying something similar with your team, we’d like to reiterate two things that have helped us:
a) keep the weekly tasks small and specific
b) don’t get too attached to deadlines
Just focus on what is being done and what you can learn from it.
Related posts
- Where is Qxf2 headed?
- The need for change (at Qxf2)
- An introduction to R&D at Qxf2
- An introduction to training at Qxf2
- An introduction to hiring and onboarding at Qxf2
- Experimenting with team structures at Qxf2
I want to find out what conditions produce remarkable software. A few years ago, I chose to work as the first professional tester at a startup. I successfully won credibility for testers and established a world-class team. I have lead the testing for early versions of multiple products. Today, I run Qxf2 Services. Qxf2 provides software testing services for startups. If you are interested in what Qxf2 offers or simply want to talk about testing, you can contact me at: [email protected]. I like testing, math, chess and dogs.
