Startups may not always need full time professional testers. Crowdsourced testing is a good way to pull in short-term, tactical and highly directed testing help. Usability testing of a consumer application is an example of a good use case for a crowdsourced testing. While most people are aware that crowdsourced testing exists, the process of using these providers is somewhat opaque. How to list your tests? What does the output look like? What happens when you make changes to your application? Can you use them for regression testing? This post shines light on the process of working with a crowdsourced testing provider. We take you through the entire journey - from registration to repeating your tests after your latest usability modifications.
At a very high level, you need to decide what you want from the crowdsourcing provider, pick a crowdsourcing provider, formulate your questions, upload your test steps and then review the results.
Start by asking yourself what problem you want the crowdsourced testing provider to solve. Are you looking for feedback on the usability of your product? Are you looking for functional testing? Are you looking for accessibility testing? Are you looking to smoke out bugs? Do you already have a suite of tests with detailed steps that you want to hand off for a short while?
Research online to figure out the best crowdsourcing platform for your use case. Alternatively, use our handy guide to compare and pick a crowdsourcing provider.
Register with the crowdsourcing provider. Once you register and login, you will have the option of creating tests. Tests should be detailed and clear to any user who is accessing your application for the first time. Remember to not use any internal jargon or naming conventions when detailing the tests. Each test is associated with a list of questions you can ask the tester. Questions can range from "what stood out to you on this page?" to "was the sign up button too big?". Tests are most effective when you design one test per page of the application you want tested.
PRO TIP 1: Many providers offer a free trial if you just ask - especially if you are seriously shopping around. So ask!
PRO TIP 2: Plan at least 2-3 days in advance - this helps you iron out any kinks with test setup
Decide on the number of testers you want, the kind of platform coverage you desire and the browsers you want the testers to use when performing your tests.
PRO TIP: Start small. Try 2 testers on 2 platforms and 2 browsers.
Almost all crowdsourcing providers allow you to preview the test you have setup. Spend some time carefully reviewing the tests you have added. This is your last chance to tweak the tests before the tests go live!
At the click of a button, your test can be launched. You now need to wait for the results to roll in. You will receive email alerts. How long you need to wait depends on your test volume and the provider you use. Your wait could be as short as an hour or as long as one week.
Your results can come in different formats: short videos, screen captures, written answers to your questions. You need to view what each tester did and consolidate their results. There will be some overlap depending on the kind of tests you ran. We find this to be the most time consuming part of using a crowdsourced testing provider. Some providers allow you to pay for the option of getting consolidated results - but we have not experimented with that option yet.
Once you make changes to your application, you can simply clone your existing tests, make any tweak you want and then repeat this process.
And that is how you go about using a crowdsourced testing provider.
© Qxf2 Services 2013 -
