Rocketrip Test Strategy
This report is one of the several practice exercises Qxf2 testers undertake. We wrote this entire report without having any access to the application. We spoke to exactly zero employees at Rocketrip. All our knowledge about Rocketrip comes only from public sources of information available on the Internet. So, yes! Good testers can learn a lot about a product without having access to it.
I. Travellers in Rocketrip's journey
We thought about the problem Rocketrip is solving. We identified four key partners/stakeholders and imagined what they likely value in the software.
Show more details
-
Companies
- Did they save money - benchmark budget should be reasonable
- Good dashboards to notice savings 'at a glance'
- Motivation to save should not affect employee performance
-
Employees
- Should save money - benchmark budget should be reasonable
- Good cost saving tips
- Good integration for bill submission
- Choices of gift card vendors
- Ease of use - get a budget, see calendar events, redeem points, etc
-
HR folk
- Ease of uploading and updating company policies, employee details
- Ease integrating Rocketrip with current expense and travel systems
- Good customer support
-
Travel and gift card vendors
- Customer acquisition
II. Groundwork for testing Rocketrip
Keeping the above stakeholders and their wants in mind, we imagined how the software would look. Some of the sources we used to learn more about Rocketrip were:
GSummit 2014 talk,
case studies,
job postings,
website,
press releases,
interviews and
blog posts. We came up with the below list of components, features and testing recommendations.
Show more details
-
Benchmark budget calculation:
- Why? The secret sauce for Rocketrip.
- What? We expect the algorithm to be based on type of booking, company policy, employee details and real time market data.
- How to test? Given the importance of the benchmark budget to the product, we recommend multiple layers of testing here. We would likely test the algorithm, third party integrations and the UI separate from one another. We think we would end up with the following: unit tests, integration tests with vendor APIs, monitoring and alerting for vendor APIs, API tests, ability to mock real time data for repeatable tests and performance tests.
-
Reporting and Dashboards
- Why? Drives home why people should use Rocketrip
- What? The reports probably cover savings related analytics, saver scenarios, booking analytics and travel expense comparisons.
- How to test? We recommend testing for accuracy and visual appeal. Accuracy tests will likely be performed by a combination of API tests and DB checks. For UI tests, we would write Selenium tests to cover combinations of browsers, access levels (administrator, employer, employee) and screens/graphs.
-
Get baseline budget form
- Why? Key interface for an employee to use Rocketrip.
- What? A form for employees to get baseline budget for their travel plans.
- How to test? We recommend thorough form-testing. The presence of date fields (hello time zones!), categories of travel, multiple text inputs and links to book through select vendors require automation. We would most probably need to write Selenium tests to ensure good coverage across the various actions a user can perform on the form. This includes checking for good feedback/validation messages on the UI. We would also likely write API tests. This piece may need performance testing depending on how many users you have and the architecture of calculating the baseline budget.
-
Manage expense reports
- Why? Employee needs to submit expense form to earn rewards
- What? We expect a form that integrates with some 3rd party expense management tools.
- How to test? This seems to be (without looking at the product) one of the non-scaleable aspects of Rocketrip. For this feature, we would recommend that you test only till 'good enough' and reduce engineering effort by architecting the 3rd party integration in a way that it can be deployed independently. Every time a new vendor is integrated, we advocate thoroughly testing the first time around. Then add some good monitoring for the API you use to connect to third party. We expect you will not modify this part of the code too often once the integration is done. We would also strongly recommend building something to independently deploy each integration. This will help reduce the regression testing load of the QA team.
-
Manage rewards
- Why? Adds to the 'stickiness' of the product. Provides an additional revenue stream for Rocketrip through vendor deals.
- What? We expect a catalog of different vendors.
- How to test? We need to ensure that the user experience to select vendors is good for the employees. Depending on the balance of power in the relation, Rocketrip may think about how easy the vendor makes it for the employee to redeem points. Testing this piece will need some co-ordination with vendors to set up test accounts. We would perform a combination of UI and API tests.
-
User management (admin)
- Why? Great products deserve users!
- What? Need a way to set up employees and roles in the product. We envision that the software allows certain roles to add/edit/delete employee details, define their roles and figure out their budget allowance based on company policy.
- How to test? We recommend workflow tests. Routine CRUD operations are easy to test, but the challenge is performing these actions in different sequences and confirming that the permissioning and reporting looks correct. We would request the ability to impersonate users on the test environment. Additionally we likely need UI automation, unit tests for access control, API tests to enhance our UI coverage and additional security testing too. Mistakes in user management propagate through the product and manifest themselves everywhere. So it makes sense to build heavy testing for this feature.
-
Expense management (admin)
- Why? Admins need a way to manage submitted expense reports.
- What? Review bills submitted, approve expenses and manually over-ride mistakes.
- How to test? We recommend workflow tests. We recommend more UI testing and some basic API testing for key actions like approve, delete and over-ride.
-
Travel policy management (admin)
- Why? Companies need to give Rocketrip their travel policy details.
- What? A few screens to setup and update the company's travel policy.
- How to test? We do not have enough knowledge of travel policies to imagine this. We figure if we had a chance to work with you, we could quickly look at your existing clients and figure out how to test this piece.
|
|
While onboarding a new client we expect updating the travel policy management and the integration with third party expense management systems to be places where engineering effort does not scale
|
|
|
We expect that your screens for admin functions to have received less love and care from your team. It probably would have made sense for a startup to spend its limited time polishing the user facing screens and take on debt of manually managing the admin functionality.
|
III. Testing tools
We are conversant with a variety of automation tools. We are making an educated guess that you use Python and possibly Django. Reason: We used builtwith and noticed you use Django CSRF - indicating the place users login to your website. Our hunch was strengthened when we saw this job posting for a software developer. We are listing several tutorials we have published on our blog. Please take this as evidence of our comfort with Python and automation.
 |
|
1. Unit testing:
Python Mock,
JUnit,
TestNG,
Jasmine,
Android
|
|
2. API testing:
API basics,
API tools,
Python Mechanize,
Runscope
|
|
3. Selenium:
Batman and page objects,
Page object model,
Art of writing xpaths
|
|
4. Mobile automation:
Appium 1,
Appium 2,
Selendroid,
Identifying elements
|
|
5. Performance:
JMeter,
Gatling
|
|
6. Infrastructure:
Jenkins 1,
Jenkins + Python,
Sauce Labs,
BrowserStack
|
|
IV. Learnings from our past missions
Phew! That was a lot of technical detail to answer your ad. But if you are looking for your first tester, it is likely that you are facing issues that have manifested as people problems too. Qxf2 Services is built for startups. We have had the experience of being the first tester hired at a startup. We can help you. But irrespective of whether you end up hiring us or not, here are a few suggestions for hiring your first tester.
Show more details
-
Hire for attitude. Its likely that your first tester needs to be like this guy until the testing team has won credibility within your organization.
-
Look for testers that think in ranges. At a startup it is likely that approaches to testing shift dramatically and quickly based on external factors. Its useful if testers understand the dynamic context and adjust their testing.
-
Balance product scalability and organizational scalability. It is good to have testers who balance both product scalability (think unit tests and automated checks for regression) as well as organizational scalability (think behavior, values, interacting with people, verbalizing complexity).
Some good indicators of whether your first tester is succeeding are:
-
Customer support feel less surprised and more sure-footed about deploys.
-
Has built credibility for the testing team. People trust your testing team when they sign off on a deploy.
V. About us
 |
Avinash Shetty: I have 8+ years of experience in software testing. As a student of the context-driven approach to testing, I feel there is a lot to learn out there which keeps me excited. My work has helped me gain experience in different areas of testing like CRM, Web, Mobile, Database testing, tools like Selenium and Appium using Java and Python. Beside testing I love watching and playing sports.
LinkedIn Blog Twitter |
 |
Vrushali Mahalley: I am a Computer Science graduate from BITS-Pilani and have worked at Sun Microsystems (now Oracle) for 7+ yrs.I was part of the testing and sustaining team for Portal Server and Identity Management products.Currently working at Qxf2 Services ,with a fresh perspective on testing and enjoying the work here after a break of 6 yrs from software industry.I love travelling and listening to music.
LinkedIn Blog |
 |
Qxf2 Services provides software testing services for startups. We love the craft of testing. Our testing experts help you deliver better software. We can setup your test team, introduce automation, evaluate your current testing, interview candidates for you and so much more! Contact: [email protected]
LinkedIn Blog Twitter |
