Modern testing for modern stacks

This page lists Qxf2 Services's experience with different crowdsourced testing providers. The data is accurate as of June 2015.

Crowdsourced testing providers compared

UserTesting Loop11 UsabilityHub TryMyUI Userlytics 99tests Pay4Bugs
Type of Testing Usability, Functional Usability, Accessibility Usability Usability Usability Usability, Functional Functional
Things that can be tested Website, Wireframes, Static Images, Mobile app Website, Wireframes, Static Images, Mobile app Website, Concept mapping, Wireframes, Static Images Website, Wireframes, Static Images, Mobile app Website, Wireframes, Static Images, Mobile app, desktop applications, Videos, presentations Website, Mobile app Website, Mobile app
Tasks to evaluate Customized Task, Impression(Five second) Task Customized Task Impression(Five second) Test, Navigation Test, Click Test,Preference Test Impression(15 seconds) Test, Customized Task Impression Test, Customized Task, Navigation Test Customized Task Customized Task
Final output
  • Video file
  • Answers
  • Report
  • Answers
  • Metrics
  • Answers
  • Video
  • Answers
  • Video
  • Answers
  • Bug list
  • Bug list
Result timeline 1 hour - 24 hrs(Time Zone) 24 hours 20-30 hours 1 hour - 24 hrs(Time Zone) 24 hours 1 week starts within hours
Price link link link link link link link
Set up Ease Y Y Y Y Y Y Y
Clone tests? Y Y Not specified Not specified Not specified Not specified Not specified
Questions to evaluate Verbal response, Multiple choice, Rating scale, Open Ended Multiple choice, Rating scale, Open Ended Customized, Template Customized, Template Customized, Template Customized Task Customized Task
Browsers Selection possible Not specified Not specified Multiple Multiple Multiple Multiple
Platforms Web applications, Mobile Apps Web applications Web applications Multiple Multiple Multiple Multiple
Product Stage Prototype,Live Prototype,Live Prototype,Live Prototype,Live Prototype,Live Prototype,Live Prototype,Live
Video Duration(minutes) 15 NA NA 15-20 15 NA NA
Select tester profiles Y Y Y Y Y Y Y
Support turnaround (hrs) 24 hours 24 hours 24 hours 24 hours 24 hours 24 hours 24 hours
Company Size Medium Small Small ? ? Small Small
Company age (years) 7 <5 <5? <5? <5 <5 <5

If you want to learn more about our experience with any specific provider or want us to clarify anything in the table, feel free to write to us.

paper cut