Problem: Different test runs require different configuration on Sauce Labs. In this post, we will show you how to make your automated test runs to use specific configuration parameters of Sauce Labs.
TWEAKING SAUCE LABS CONFIGURATION
Some nightly automated selenium checks that we run at Qxf2 Services need to be run only on specific browsers. Further some of our nightly tests were running a lot slower on Sauce labs. We decided to tweak a few configuration parameters on Sauce Labs to improve performance. We also added some useful reporting parameters like build and tag to help testers identify different runs easily. This post will walk you through the steps you need to do to be able to map your automated test runs to use specific configuration parameters of Sauce Labs. For this example, we are going to show you how to set the following parameters:
1. browser
2. browser version
3. platform
4. name: name of the test run
5. build: version of your application the test was running against
6. tag: tag your test cases to different groups
7. record-video
8. video-upload-on-pass
9. record-screenshots
The configuration parameters will be listed in a file and read via the Python module ConfigParser.
OVERVIEW
1. Create a configuration file
2. Accept command line options for configuration file and test run name
3. Use ConfigParser to read the configuration
4. Run the tests
5. Check the result on Sauce Labs
STEP 1: Create a configuration file
We can create a configuration file that uses the test run name to identify sections and list the desired configuration under it. For this example we created a file called test_configuration.ini. Add the contents below to your configuration file and tweak the parameter values as you like. In this example we are showing you two test runs (Nightly Run and Acceptance Run) using different configurations.
[Nightly Run] browser = ff version = 28 platform = Windows 8 name = 20-May-2013 nightly run build = Build-17.23 tag = capa_20-May-2013 record-video = false video-upload-on-pass = false record-screenshots = false [Acceptance Run] browser = ie version = 8 platform = Windows 7 name = Acceptance test run build = Build-17.23 tag = capa_20-May-2013 record-video = false video-upload-on-pass = false record-screenshots = false |
NOTE: We have chosen to disable video recording, screenshot, video upload on pass to speed up our test runs.
STEP 2: Accept command line options for configuration file and test run name
In our previous post, we showed you an example of accepting command line arguments in our test scripts. In this post we will modify the command line parameters to accept the configuration file and test run name. The code snippet to accept the configuration file and command line is given below.
#---START OF SCRIPT if __name__ == '__main__': #Lets accept some command line options from the user #We have chosen to use the Python module optparse usage = "usage: %prog -c -t \nE.g.1: %prog -c C:\\tests\\conf\\test_configuration.ini -t \"Nightly Run\"\n---" parser = OptionParser(usage=usage) parser.add_option("-c","--config",dest="config_file",help="The full path of the configuration file") parser.add_option("-t","--test_run",dest="test_run",help="The name of the test run") (options,args) = parser.parse_args() |
STEP 3: Use ConfigParser to read the configuration
Qxf2 Services loves, and I mean *loves*, Python. We have chosen to use the module ConfigParser to read our configuration file. Python newbies can use either this link (pmotw) or this link (Python wiki) to get started with ConfigParser. On our end, we are modifying our Selenium2OnSauce class to look like the snippet below:
class Selenium2OnSauce(unittest.TestCase): "Example class written to run Selenium tests on Sauce Labs" def __init__(self,config_file,test_run): "Constructor: Accepts the configuration file and the test run name as parameters" self.config_file = config_file self.test_run = test_run def setUp(self): "Setup for this test involves spinning up the right virtual machine on Sauce Labs" parser = SafeConfigParser() #THE MAGIC STARTS HERE parser.read(self.config_file) if parser.get(self.test_run, 'browser').lower()=='chrome': desired_capabilities = webdriver.DesiredCapabilities.CHROME if parser.get(self.test_run, 'browser').lower()in ['firefox','ff']: desired_capabilities = webdriver.DesiredCapabilities.FIREFOX if parser.get(self.test_run, 'browser').lower()in ['internet explorer','ie']: desired_capabilities = webdriver.DesiredCapabilities.INTERNETEXPLORER desired_capabilities['version'] = parser.get(self.test_run, 'version') desired_capabilities['platform'] = parser.get(self.test_run, 'platform') desired_capabilities['name'] = parser.get(self.test_run, 'name') desired_capabilities['build'] = parser.get(self.test_run, 'build') desired_capabilities['tags'] = parser.get(self.test_run, 'tag') desired_capabilities['record-video'] = parser.get(self.test_run, 'record-video') desired_capabilities['video-upload-on-pass'] = parser.get(self.test_run, 'video-upload-on-pass') desired_capabilities['record-screenshots'] = parser.get(self.test_run, 'record-screenshots') self.driver = webdriver.Remote( desired_capabilities=desired_capabilities, command_executor="http://$USERNAME:[email protected]:80/wd/hub" ) self.driver.implicitly_wait(30) |
STEP 4. Run the tests
Bringing everything together, your script should look something like the code below
import logging, unittest, sys from optparse import OptionParser from selenium import webdriver from selenium.webdriver.common.keys import Keys from ConfigParser import SafeConfigParser class Selenium2OnSauce(unittest.TestCase): "Example class written to run Selenium tests on Sauce Labs" def __init__(self,config_file,test_run): "Constructor: Accepts the configuration file and the test run name as parameters" self.config_file = config_file self.test_run = test_run def setUp(self): "Setup for this test involves spinning up the right virtual machine on Sauce Labs" parser = SafeConfigParser() parser.read(self.config_file) if parser.get(self.test_run, 'browser').lower()=='chrome': desired_capabilities = webdriver.DesiredCapabilities.CHROME if parser.get(self.test_run, 'browser').lower()in ['firefox','ff']: desired_capabilities = webdriver.DesiredCapabilities.FIREFOX if parser.get(self.test_run, 'browser').lower()in ['internet explorer','ie']: desired_capabilities = webdriver.DesiredCapabilities.INTERNETEXPLORER desired_capabilities['version'] = parser.get(self.test_run, 'version') desired_capabilities['platform'] = parser.get(self.test_run, 'platform') desired_capabilities['name'] = parser.get(self.test_run, 'name') desired_capabilities['build'] = parser.get(self.test_run, 'build') desired_capabilities['tags'] = parser.get(self.test_run, 'tag') desired_capabilities['record-video'] = parser.get(self.test_run, 'record-video') desired_capabilities['video-upload-on-pass'] = parser.get(self.test_run, 'video-upload-on-pass') desired_capabilities['record-screenshots'] = parser.get(self.test_run, 'record-screenshots') self.driver = webdriver.Remote( desired_capabilities=desired_capabilities, command_executor="http://$USERNAME:[email protected]:80/wd/hub" ) self.driver.implicitly_wait(30) def test_search_in_python_org(self): "An example test: Visit python.org and search for BeautifulSoup" #Go to the URL self.driver.get("http://www.python.org") #Assert that the title is correct self.assertIn("Python", self.driver.title) #Identify the xpath and send the string you want elem = self.driver.find_element_by_xpath("//input[@id='id-search-field']") print "About to search for the string BeautifulSoup on python.org" elem.send_keys("BeautifulSoup") elem.send_keys(Keys.RETURN) print "Search for the string BeautifulSoup on python.org was successful" def tearDown(self): "Finish the test run" self.driver.quit() #---START OF SCRIPT if __name__ == '__main__': #Lets accept some command line options from the user #We have chosen to use the Python module optparse usage = "usage: %prog -c -t \nE.g.1: %prog -c C:\\tests\\conf\\test_configuration.ini -t \"Nightly Run\"\n---" parser = OptionParser(usage=usage) parser.add_option("-c","--config",dest="config_file",help="The full path of the configuration file") parser.add_option("-t","--test_run",dest="test_run",help="The name of the test run") (options,args) = parser.parse_args() #Create a test obj with our parameters test_obj = Selenium2OnSauce(config_file=options.config_file,test_run=options.test_run) #We are explicitly calling setup snd tearDown because what we are showing is technically not a unit test test_obj.setUp() #Run the test itself. NOTE: This is NOT a unit test test_obj.test_search_in_python_org() #We are explicitly calling setup snd tearDown because what we are showing is technically not a unit test test_obj.tearDown() |
You can run the test script the normal way you do. We run it via the command prompt. Here is a screenshot from our machine.
STEP 5: Check the result on Sauce Labs
You can see the results on your web account. Login to your Sauce Labs account and you should see a result table like the screen shot below.
Voila! You can run your automated Selenium test runs with different configurations.
I am a dedicated quality assurance professional with a true passion for ensuring product quality and driving efficient testing processes. Throughout my career, I have gained extensive expertise in various testing domains, showcasing my versatility in testing diverse applications such as CRM, Web, Mobile, Database, and Machine Learning-based applications. What sets me apart is my ability to develop robust test scripts, ensure comprehensive test coverage, and efficiently report defects. With experience in managing teams and leading testing-related activities, I foster collaboration and drive efficiency within projects. Proficient in tools like Selenium, Appium, Mechanize, Requests, Postman, Runscope, Gatling, Locust, Jenkins, CircleCI, Docker, and Grafana, I stay up-to-date with the latest advancements in the field to deliver exceptional software products. Outside of work, I find joy and inspiration in sports, maintaining a balanced lifestyle.
Really useful! I like this blog very much!
Here i have a question:
From this page https://saucelabs.com/account I can always see “Results” as “Finished”.
Then I failed my unit test case deliberately, but still get “Finished”. Is there any good way I can see pass or fail in the sauce page obviously?