Running Selenium automation using Sauce Labs: Part 3

Problem: Different test runs require different configuration on Sauce Labs. In this post, we will show you how to make your automated test runs to use specific configuration parameters of Sauce Labs.


TWEAKING SAUCE LABS CONFIGURATION

Some nightly automated selenium checks that we run at Qxf2 Services need to be run only on specific browsers. Further some of our nightly tests were running a lot slower on Sauce labs. We decided to tweak a few configuration parameters on Sauce Labs to improve performance. We also added some useful reporting parameters like build and tag to help testers identify different runs easily. This post will walk you through the steps you need to do to be able to map your automated test runs to use specific configuration parameters of Sauce Labs. For this example, we are going to show you how to set the following parameters:
1. browser
2. browser version
3. platform
4. name: name of the test run
5. build: version of your application the test was running against
6. tag: tag your test cases to different groups
7. record-video
8. video-upload-on-pass
9. record-screenshots

The configuration parameters will be listed in a file and read via the Python module ConfigParser.


OVERVIEW

1. Create a configuration file
2. Accept command line options for configuration file and test run name
3. Use ConfigParser to read the configuration
4. Run the tests
5. Check the result on Sauce Labs

STEP 1: Create a configuration file
We can create a configuration file that uses the test run name to identify sections and list the desired configuration under it. For this example we created a file called test_configuration.ini. Add the contents below to your configuration file and tweak the parameter values as you like. In this example we are showing you two test runs (Nightly Run and Acceptance Run) using different configurations.

[Nightly Run]
browser = ff
version = 28
platform = Windows 8
name = 20-May-2013 nightly run
build = Build-17.23
tag = capa_20-May-2013
record-video = false
video-upload-on-pass = false
record-screenshots = false
 
[Acceptance Run]
browser = ie
version = 8
platform = Windows 7
name = Acceptance test run
build = Build-17.23
tag = capa_20-May-2013
record-video = false
video-upload-on-pass = false
record-screenshots = false

NOTE: We have chosen to disable video recording, screenshot, video upload on pass to speed up our test runs.

STEP 2: Accept command line options for configuration file and test run name

In our previous post, we showed you an example of accepting command line arguments in our test scripts. In this post we will modify the command line parameters to accept the configuration file and test run name. The code snippet to accept the configuration file and command line is given below.

#---START OF SCRIPT
if __name__ == '__main__':
    #Lets accept some command line options from the user
    #We have chosen to use the Python module optparse 
    usage = "usage: %prog -c  -t  \nE.g.1: %prog -c C:\\tests\\conf\\test_configuration.ini -t \"Nightly Run\"\n---"
    parser = OptionParser(usage=usage)
    parser.add_option("-c","--config",dest="config_file",help="The full path of the configuration file")
    parser.add_option("-t","--test_run",dest="test_run",help="The name of the test run")
    (options,args) = parser.parse_args()

STEP 3: Use ConfigParser to read the configuration
Qxf2 Services loves, and I mean *loves*, Python. We have chosen to use the module ConfigParser to read our configuration file. Python newbies can use either this link (pmotw) or this link (Python wiki) to get started with ConfigParser. On our end, we are modifying our Selenium2OnSauce class to look like the snippet below:

class Selenium2OnSauce(unittest.TestCase):
    "Example class written to run Selenium tests on Sauce Labs"
    def __init__(self,config_file,test_run):
        "Constructor: Accepts the configuration file and the test run name as parameters"
        self.config_file = config_file
        self.test_run = test_run
 
    def setUp(self):
        "Setup for this test involves spinning up the right virtual machine on Sauce Labs"
        parser = SafeConfigParser() #THE MAGIC STARTS HERE
        parser.read(self.config_file) 
 
        if parser.get(self.test_run, 'browser').lower()=='chrome':
            desired_capabilities = webdriver.DesiredCapabilities.CHROME
        if parser.get(self.test_run, 'browser').lower()in ['firefox','ff']:
            desired_capabilities = webdriver.DesiredCapabilities.FIREFOX
        if parser.get(self.test_run, 'browser').lower()in ['internet explorer','ie']:
            desired_capabilities = webdriver.DesiredCapabilities.INTERNETEXPLORER
 
        desired_capabilities['version'] = parser.get(self.test_run, 'version')
        desired_capabilities['platform'] = parser.get(self.test_run, 'platform')
        desired_capabilities['name'] = parser.get(self.test_run, 'name') 
        desired_capabilities['build'] = parser.get(self.test_run, 'build')
        desired_capabilities['tags'] = parser.get(self.test_run, 'tag')
        desired_capabilities['record-video'] = parser.get(self.test_run, 'record-video')
        desired_capabilities['video-upload-on-pass'] = parser.get(self.test_run, 'video-upload-on-pass')
        desired_capabilities['record-screenshots'] = parser.get(self.test_run, 'record-screenshots')
 
        self.driver = webdriver.Remote(
            desired_capabilities=desired_capabilities,
            command_executor="http://$USERNAME:$ACCESS-ID@ondemand.saucelabs.com:80/wd/hub"
        )
        self.driver.implicitly_wait(30)

STEP 4. Run the tests
Bringing everything together, your script should look something like the code below

import logging, unittest, sys
from optparse import OptionParser
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from ConfigParser import SafeConfigParser
 
class Selenium2OnSauce(unittest.TestCase):
    "Example class written to run Selenium tests on Sauce Labs"
    def __init__(self,config_file,test_run):
        "Constructor: Accepts the configuration file and the test run name as parameters"
        self.config_file = config_file
        self.test_run = test_run
 
    def setUp(self):
        "Setup for this test involves spinning up the right virtual machine on Sauce Labs"
        parser = SafeConfigParser()
        parser.read(self.config_file)
 
        if parser.get(self.test_run, 'browser').lower()=='chrome':
            desired_capabilities = webdriver.DesiredCapabilities.CHROME
        if parser.get(self.test_run, 'browser').lower()in ['firefox','ff']:
            desired_capabilities = webdriver.DesiredCapabilities.FIREFOX
        if parser.get(self.test_run, 'browser').lower()in ['internet explorer','ie']:
            desired_capabilities = webdriver.DesiredCapabilities.INTERNETEXPLORER
 
        desired_capabilities['version'] = parser.get(self.test_run, 'version')
        desired_capabilities['platform'] = parser.get(self.test_run, 'platform')
        desired_capabilities['name'] = parser.get(self.test_run, 'name')
        desired_capabilities['build'] = parser.get(self.test_run, 'build')
        desired_capabilities['tags'] = parser.get(self.test_run, 'tag')
        desired_capabilities['record-video'] = parser.get(self.test_run, 'record-video')
        desired_capabilities['video-upload-on-pass'] = parser.get(self.test_run, 'video-upload-on-pass')
        desired_capabilities['record-screenshots'] = parser.get(self.test_run, 'record-screenshots')
 
        self.driver = webdriver.Remote(
            desired_capabilities=desired_capabilities,
            command_executor="http://$USERNAME:$ACCESS_KEY@ondemand.saucelabs.com:80/wd/hub"
        )
        self.driver.implicitly_wait(30)
 
 
    def test_search_in_python_org(self):
        "An example test: Visit python.org and search for BeautifulSoup"
        #Go to the URL 
        self.driver.get("http://www.python.org")
 
        #Assert that the title is correct
        self.assertIn("Python", self.driver.title)
 
        #Identify the xpath and send the string you want
        elem = self.driver.find_element_by_xpath("//input[@id='id-search-field']")
        print "About to search for the string BeautifulSoup on python.org"
        elem.send_keys("BeautifulSoup")
        elem.send_keys(Keys.RETURN)
        print "Search for the string BeautifulSoup on python.org was successful"
 
 
    def tearDown(self):
        "Finish the test run"
        self.driver.quit()
 
 
#---START OF SCRIPT
if __name__ == '__main__':
    #Lets accept some command line options from the user
    #We have chosen to use the Python module optparse 
    usage = "usage: %prog -c  -t  \nE.g.1: %prog -c C:\\tests\\conf\\test_configuration.ini -t \"Nightly Run\"\n---"
    parser = OptionParser(usage=usage)
    parser.add_option("-c","--config",dest="config_file",help="The full path of the configuration file")
    parser.add_option("-t","--test_run",dest="test_run",help="The name of the test run")
    (options,args) = parser.parse_args()
 
    #Create a test obj with our parameters
    test_obj = Selenium2OnSauce(config_file=options.config_file,test_run=options.test_run)
 
    #We are explicitly calling setup snd tearDown because what we are showing is technically not a unit test
    test_obj.setUp()
 
    #Run the test itself. NOTE: This is NOT a unit test
    test_obj.test_search_in_python_org()
 
    #We are explicitly calling setup snd tearDown because what we are showing is technically not a unit test
    test_obj.tearDown()

You can run the test script the normal way you do. We run it via the command prompt. Here is a screenshot from our machine.

Run the test on Sauce Labs
Here is a screenshot from our machine.

STEP 5: Check the result on Sauce Labs
You can see the results on your web account. Login to your Sauce Labs account and you should see a result table like the screen shot below.

Result on Sauce Labs

 


Voila! You can run your automated Selenium test runs with different configurations.

Avinash Shetty
I am a software tester with over 10 years of experience in software testing. Currently I am working at Qxf2 Services Bangalore. As a student of the context-driven approach to software testing I feel there is a lot to learn out there which keeps me very excited. My work has helped me gain good experience in different areas of testing like CRM, Web, Mobile and Database testing. I have good knowledge of building test scripts using Automation tools like Selenium and Appium using Java and Python. Beside testing I am a “Sports Fanatic” and love watching and playing sports.

© 2013-2017, Avinash Shetty. All rights reserved.

2 Comments

  1. Anonymous said:

    Really useful! I like this blog very much!
    Here i have a question:
    From this page https://saucelabs.com/account I can always see “Results” as “Finished”.
    Then I failed my unit test case deliberately, but still get “Finished”. Is there any good way I can see pass or fail in the sauce page obviously?

    July 7, 2015
    Reply

Leave a Reply to Testing Bits – 5/18/14 – 5/24/14 | Testing Curator Blog Cancel reply

Your email address will not be published.