Problem: Different test runs require different configuration on Sauce Labs. In this post, we will show you how to make your automated test runs to use specific configuration parameters of Sauce Labs.
TWEAKING SAUCE LABS CONFIGURATION
Some nightly automated selenium checks that we run at Qxf2 Services need to be run only on specific browsers. Further some of our nightly tests were running a lot slower on Sauce labs. We decided to tweak a few configuration parameters on Sauce Labs to improve performance. We also added some useful reporting parameters like build and tag to help testers identify different runs easily. This post will walk you through the steps you need to do to be able to map your automated test runs to use specific configuration parameters of Sauce Labs. For this example, we are going to show you how to set the following parameters:
1. browser
2. browser version
3. platform
4. name: name of the test run
5. build: version of your application the test was running against
6. tag: tag your test cases to different groups
7. record-video
8. video-upload-on-pass
9. record-screenshots
The configuration parameters will be listed in a file and read via the Python module ConfigParser.
OVERVIEW
1. Create a configuration file
2. Accept command line options for configuration file and test run name
3. Use ConfigParser to read the configuration
4. Run the tests
5. Check the result on Sauce Labs
STEP 1: Create a configuration file
We can create a configuration file that uses the test run name to identify sections and list the desired configuration under it. For this example we created a file called test_configuration.ini. Add the contents below to your configuration file and tweak the parameter values as you like. In this example we are showing you two test runs (Nightly Run and Acceptance Run) using different configurations.
[Nightly Run] browser = ff version = 28 platform = Windows 8 name = 20-May-2013 nightly run build = Build-17.23 tag = capa_20-May-2013 record-video = false video-upload-on-pass = false record-screenshots = false [Acceptance Run] browser = ie version = 8 platform = Windows 7 name = Acceptance test run build = Build-17.23 tag = capa_20-May-2013 record-video = false video-upload-on-pass = false record-screenshots = false |
NOTE: We have chosen to disable video recording, screenshot, video upload on pass to speed up our test runs.
STEP 2: Accept command line options for configuration file and test run name
In our previous post, we showed you an example of accepting command line arguments in our test scripts. In this post we will modify the command line parameters to accept the configuration file and test run name. The code snippet to accept the configuration file and command line is given below.
#---START OF SCRIPT if __name__ == '__main__': #Lets accept some command line options from the user #We have chosen to use the Python module optparse usage = "usage: %prog -c -t \nE.g.1: %prog -c C:\\tests\\conf\\test_configuration.ini -t \"Nightly Run\"\n---" parser = OptionParser(usage=usage) parser.add_option("-c","--config",dest="config_file",help="The full path of the configuration file") parser.add_option("-t","--test_run",dest="test_run",help="The name of the test run") (options,args) = parser.parse_args() |
STEP 3: Use ConfigParser to read the configuration
Qxf2 Services loves, and I mean *loves*, Python. We have chosen to use the module ConfigParser to read our configuration file. Python newbies can use either this link (pmotw) or this link (Python wiki) to get started with ConfigParser. On our end, we are modifying our Selenium2OnSauce class to look like the snippet below:
class Selenium2OnSauce(unittest.TestCase): "Example class written to run Selenium tests on Sauce Labs" def __init__(self,config_file,test_run): "Constructor: Accepts the configuration file and the test run name as parameters" self.config_file = config_file self.test_run = test_run def setUp(self): "Setup for this test involves spinning up the right virtual machine on Sauce Labs" parser = SafeConfigParser() #THE MAGIC STARTS HERE parser.read(self.config_file) if parser.get(self.test_run, 'browser').lower()=='chrome': desired_capabilities = webdriver.DesiredCapabilities.CHROME if parser.get(self.test_run, 'browser').lower()in ['firefox','ff']: desired_capabilities = webdriver.DesiredCapabilities.FIREFOX if parser.get(self.test_run, 'browser').lower()in ['internet explorer','ie']: desired_capabilities = webdriver.DesiredCapabilities.INTERNETEXPLORER desired_capabilities['version'] = parser.get(self.test_run, 'version') desired_capabilities['platform'] = parser.get(self.test_run, 'platform') desired_capabilities['name'] = parser.get(self.test_run, 'name') desired_capabilities['build'] = parser.get(self.test_run, 'build') desired_capabilities['tags'] = parser.get(self.test_run, 'tag') desired_capabilities['record-video'] = parser.get(self.test_run, 'record-video') desired_capabilities['video-upload-on-pass'] = parser.get(self.test_run, 'video-upload-on-pass') desired_capabilities['record-screenshots'] = parser.get(self.test_run, 'record-screenshots') self.driver = webdriver.Remote( desired_capabilities=desired_capabilities, command_executor="http://$USERNAME:[email protected]:80/wd/hub" ) self.driver.implicitly_wait(30) |
STEP 4. Run the tests
Bringing everything together, your script should look something like the code below
import logging, unittest, sys from optparse import OptionParser from selenium import webdriver from selenium.webdriver.common.keys import Keys from ConfigParser import SafeConfigParser class Selenium2OnSauce(unittest.TestCase): "Example class written to run Selenium tests on Sauce Labs" def __init__(self,config_file,test_run): "Constructor: Accepts the configuration file and the test run name as parameters" self.config_file = config_file self.test_run = test_run def setUp(self): "Setup for this test involves spinning up the right virtual machine on Sauce Labs" parser = SafeConfigParser() parser.read(self.config_file) if parser.get(self.test_run, 'browser').lower()=='chrome': desired_capabilities = webdriver.DesiredCapabilities.CHROME if parser.get(self.test_run, 'browser').lower()in ['firefox','ff']: desired_capabilities = webdriver.DesiredCapabilities.FIREFOX if parser.get(self.test_run, 'browser').lower()in ['internet explorer','ie']: desired_capabilities = webdriver.DesiredCapabilities.INTERNETEXPLORER desired_capabilities['version'] = parser.get(self.test_run, 'version') desired_capabilities['platform'] = parser.get(self.test_run, 'platform') desired_capabilities['name'] = parser.get(self.test_run, 'name') desired_capabilities['build'] = parser.get(self.test_run, 'build') desired_capabilities['tags'] = parser.get(self.test_run, 'tag') desired_capabilities['record-video'] = parser.get(self.test_run, 'record-video') desired_capabilities['video-upload-on-pass'] = parser.get(self.test_run, 'video-upload-on-pass') desired_capabilities['record-screenshots'] = parser.get(self.test_run, 'record-screenshots') self.driver = webdriver.Remote( desired_capabilities=desired_capabilities, command_executor="http://$USERNAME:[email protected]:80/wd/hub" ) self.driver.implicitly_wait(30) def test_search_in_python_org(self): "An example test: Visit python.org and search for BeautifulSoup" #Go to the URL self.driver.get("http://www.python.org") #Assert that the title is correct self.assertIn("Python", self.driver.title) #Identify the xpath and send the string you want elem = self.driver.find_element_by_xpath("//input[@id='id-search-field']") print "About to search for the string BeautifulSoup on python.org" elem.send_keys("BeautifulSoup") elem.send_keys(Keys.RETURN) print "Search for the string BeautifulSoup on python.org was successful" def tearDown(self): "Finish the test run" self.driver.quit() #---START OF SCRIPT if __name__ == '__main__': #Lets accept some command line options from the user #We have chosen to use the Python module optparse usage = "usage: %prog -c -t \nE.g.1: %prog -c C:\\tests\\conf\\test_configuration.ini -t \"Nightly Run\"\n---" parser = OptionParser(usage=usage) parser.add_option("-c","--config",dest="config_file",help="The full path of the configuration file") parser.add_option("-t","--test_run",dest="test_run",help="The name of the test run") (options,args) = parser.parse_args() #Create a test obj with our parameters test_obj = Selenium2OnSauce(config_file=options.config_file,test_run=options.test_run) #We are explicitly calling setup snd tearDown because what we are showing is technically not a unit test test_obj.setUp() #Run the test itself. NOTE: This is NOT a unit test test_obj.test_search_in_python_org() #We are explicitly calling setup snd tearDown because what we are showing is technically not a unit test test_obj.tearDown() |
You can run the test script the normal way you do. We run it via the command prompt. Here is a screenshot from our machine.
STEP 5: Check the result on Sauce Labs
You can see the results on your web account. Login to your Sauce Labs account and you should see a result table like the screen shot below.
Voila! You can run your automated Selenium test runs with different configurations.
I am a software tester with over 14 years of experience in software testing. Currently, I am working at Qxf2 Services Bangalore. As a student of the context-driven approach to software testing, I feel there is a lot to learn out there which keeps me very excited. My work has helped me gain good experience in different areas of testing like CRM, Web, Mobile and Database testing. I have good knowledge of building test scripts using Automation tools like Selenium and Appium using Java and Python. Besides testing, I am a “Sports Fanatic” and love watching and playing sports.
Really useful! I like this blog very much!
Here i have a question:
From this page https://saucelabs.com/account I can always see “Results” as “Finished”.
Then I failed my unit test case deliberately, but still get “Finished”. Is there any good way I can see pass or fail in the sauce page obviously?