{"id":4629,"date":"2016-08-23T01:04:03","date_gmt":"2016-08-23T05:04:03","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=4629"},"modified":"2021-02-16T06:18:49","modified_gmt":"2021-02-16T11:18:49","slug":"selenium-cross-browser-cross-platform-pytest","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/selenium-cross-browser-cross-platform-pytest\/","title":{"rendered":"pytest: Cross browser, cross platform Selenium tests"},"content":{"rendered":"

We showed you how to run tests on BrowserStack<\/a> using pytest in our previous post – pytest and Browserstack<\/a>. We had an example test and ran it across one browser from the command line argument. What if we need to run it across different browser versions, platforms and platform versions?<\/p>\n

Why this post?<\/h3>\n

pytest responds really well when we run it for one combination of browser, browser version, platform, and platform version. But to make it run across a combination of different browsers, browser versions, platforms and platform versions is a bit tricky. It is tricky because not all platforms support all browsers. E.g.: Safari on Windows does not make much sense. Further, your company may have promised different browser versions on different platforms as part of the minimum browser requirements. We are writing this post to explain a clean, maintainable way to run your Selenium tests against different browsers, browser versions, platforms and platform versions.<\/p>\n

\n

Overview<\/h3>\n

To get your tests running against a combination of browsers, browser versions and platforms, you need to do the following:
\n1. Parameterize the test in conftest.py<\/code>
\n2. Generate the different browser\/OS combinations you want
\n3. Generate the tests using pytest_generate_tests<\/code>
\n4. Put it all together
\n5. Run the test<\/p>\n

The rest of the post explains each of these steps in detail.<\/p>\n

\n

1. Parameterize the test in conftest.py<\/code><\/h4>\n

pytest allows easy parameterization of tests. To run a test against a list of browsers,<\/p>\n

\r\n#contents of conftest.py\r\ndef pytest_generate_tests(metafunc):\r\n    \"test generator function to run tests across different parameters\"\r\n    if \"browser\" in metafunc.fixturenames:\r\n        metafunc.parameterize(\"browser\",metafunc.config.option.browser)\r\n<\/pre>\n
This function would parameterize the test if it finds the browser argument in the test function.
\nThe corresponding changes that have to be made to the browser option are,<\/p>\n
\r\n#Contents of conftest.py\r\nparser.addoption(\"-B\",\"--browser\",\r\n                 dest=\"browser\",\r\n                 action=\"append\",\r\n                 default=[\"firefox\"],\r\n                 help=\"Browser. Valid options are firefox, ie and chrome\")\r\n<\/pre>\n
Now running the tests using the command py.test<\/code> would run it across Firefox<\/code> by default. py.test -B chrome<\/code> command would append Chrome to the browser list for the test to run against. Hence the test would run against Firefox and Chrome.<\/p>\n
\n
2. Generate the different browser\/OS combinations you want<\/h4>\n
Let us create a conf file to store the list of browsers, browser versions, platforms and platform versions for the test to run against. This conf file would hold a method to create a tuple of browser-appropriate browser versions and platform-appropriate platform versions. <\/p>\n
\r\n#contents of the browser_platform_conf\r\n\"\"\"\r\nThis is a contrived version of the browser_platfor_conf\r\nWe use a more detailed version in our code for clients\r\n\"\"\"\r\nbrowsers = [\"chrome\"]\r\nos_list = [\"windows\",\"OS X\"]\r\nchrome_versions = [\"49\",\"50\"]\r\nwindows_versions = [\"7\",\"8.1\"]\r\nos_x_versions = [\"yosemite\"]\r\ndef generate_configuration(browsers=browsers,firefox_versions=firefox_versions,chrome_versions=chrome_versions,os_list=os_list,windows_versions=windows_versions,os_x_versions=os_x_versions):\r\n\t\"Generate test configuration\"\r\n\ttest_config = []\r\n\tfor browser in browsers:\r\n\t\tif browser == \"chrome\":\r\n\t\t\tfor chrome_version in chrome_versions:\r\n\t\t\t\tfor os_name in os_list:\r\n\t\t\t\t\tif os_name == \"windows\":\r\n\t\t\t\t\t\tfor windows_version in windows_versions:\r\n\t\t\t\t\t\t\tconfig = [browser,chrome_version,os_name,windows_version]\r\n\t\t\t\t\t\t\ttest_config.append(tuple(config))\r\n\t\t\t\t\tif os_name == \"OS X\":\r\n\t\t\t\t\t\tfor os_x_version in os_x_versions:\r\n\t\t\t\t\t\t\tconfig = [browser,chrome_version,os_name,os_x_version]\r\n\t\t\t\t\t\t\ttest_config.append(tuple(config))\r\n\r\n\r\n\r\n\treturn test_config\r\n\r\n\r\ncross_browser_cross_platform_config = generate_configuration()\r\n<\/pre>\n
3. Generate the tests using pytest_generate_tests<\/code><\/h4>\n
pytest has a method called pytest_generate_tests<\/code>. This method will ‘generate’ the same test for each of our browser\/OS combinations. All we need to do, is to provide pytest_generate_tests<\/code> with the different browser\/OS combinations. The code snippet to do this, is:<\/p>\n
\r\n#contents of conftest.py\r\nfrom conf import browser_platform_conf\r\n\r\ndef pytest_generate_tests(metafunc):\r\n    \"test generator function to run tests across different parameters\"\r\n    if 'browser' in metafunc.fixturenames:\r\n        if metafunc.config.getoption(\"-M\").lower() == 'y':\r\n            if metafunc.config.getoption(\"-B\") == [\"all\"]:\r\n                metafunc.parametrize(\"browser,browser_version,platform,os_version\", \r\n                                    browser_platform_conf.cross_browser_cross_platform_config)\r\n<\/pre>\n
Refer this link<\/a> to know more about pytest_generate_tests<\/code> and the metafunc<\/code> object.<\/p>\n
\n
4. Put it all together<\/h4>\n
Now to run the test, have the following files that look like the corresponding code below:
\na. browser_platform_conf.py
\nb. conftest.py
\nc. Test file<\/p>\n
4a. browser_platform_conf.py<\/strong>
\nThe browser_platform_conf<\/code> file to generate the test run configuration<\/p>\n
\r\n\"\"\"\r\nThis is a contrived version of the browser_platform_conf\r\nWe use a more detailed version in our code for clients\r\n\"\"\"\r\nbrowsers = [\"chrome\"]\r\nos_list = [\"windows\",\"OS X\"]\r\nchrome_versions = [\"49\",\"50\"]\r\nwindows_versions = [\"7\",\"8.1\"]\r\nos_x_versions = [\"yosemite\"]\r\n\r\n\r\ndef generate_configuration(browsers=browsers,firefox_versions=firefox_versions,chrome_versions=chrome_versions,os_list=os_list,windows_versions=windows_versions,os_x_versions=os_x_versions):\r\n\r\n\t\"Generate test configuration\"\r\n\ttest_config = []\r\n\tfor browser in browsers:\r\n\t\tif browser == \"chrome\":\r\n\t\t\tfor chrome_version in chrome_versions:\r\n\t\t\t\tfor os_name in os_list:\r\n\t\t\t\t\tif os_name == \"windows\":\r\n\t\t\t\t\t\tfor windows_version in windows_versions:\r\n\t\t\t\t\t\t\tconfig = [browser,chrome_version,os_name,windows_version]\r\n\t\t\t\t\t\t\ttest_config.append(tuple(config))\r\n\t\t\t\t\tif os_name == \"OS X\":\r\n\t\t\t\t\t\tfor os_x_version in os_x_versions:\r\n\t\t\t\t\t\t\tconfig = [browser,chrome_version,os_name,os_x_version]\r\n\t\t\t\t\t\t\ttest_config.append(tuple(config))\r\n\r\n\r\n\r\n\treturn test_config\r\n\r\n\r\ncross_browser_cross_platform_config = generate_configuration()\r\n<\/pre>\n
4b. conftest.py<\/strong>
\nThe conftest.py<\/code> file to hold the fixtures and command line arguments and the pytest_generate_test<\/code> to parametrize the test<\/p>\n
\r\nimport pytest\r\nimport os\r\nimport browser_platform_conf\r\n\r\n\r\n@pytest.fixture\r\ndef browser():\r\n    \"pytest fixture for browser\"\r\n    return pytest.config.getoption(\"-B\")\r\n\r\n\r\n@pytest.fixture\r\ndef browserstack_flag():\r\n    \"pytest fixture for browserstack flag\"\r\n    return pytest.config.getoption(\"-M\")\r\n\r\n\r\n@pytest.fixture\r\ndef browser_version():\r\n    \"pytest fixture for browser version\"\r\n    return pytest.config.getoption(\"-V\") \r\n\r\n\r\n@pytest.fixture\r\ndef platform():\r\n    \"pytest fixture for platform\"\r\n    return pytest.config.getoption(\"-P\") \r\n\r\n\r\n@pytest.fixture\r\ndef os_version():\r\n    \"pytest fixture for os version\"\r\n    return pytest.config.getoption(\"-O\") \r\n\r\n\r\ndef pytest_generate_tests(metafunc):\r\n    \"test generator function to run tests across different parameters\"\r\n    if 'browser' in metafunc.fixturenames:\r\n        if metafunc.config.getoption(\"-M\").lower() == 'y':\r\n            if metafunc.config.getoption(\"-B\") == [\"all\"]:\r\n                metafunc.parametrize(\"browser,browser_version,platform,os_version\", \r\n                                    browser_platform_conf.cross_browser_cross_platform_config)\r\n\r\n\r\ndef pytest_addoption(parser):\r\n    parser.addoption(\"-B\",\"--browser\",\r\n                      dest=\"browser\",\r\n                      action=\"append\",\r\n                      default=[],\r\n                      help=\"Browser. Valid options are firefox, ie and chrome\")                      \r\n    parser.addoption(\"-M\",\"--browserstack_flag\",\r\n                      dest=\"browserstack_flag\",\r\n                      default=\"N\",\r\n                      help=\"Run the test in Browserstack: Y or N\")\r\n    parser.addoption(\"-O\",\"--os_version\",\r\n                      dest=\"os_version\",\r\n                      action=\"append\",\r\n                      help=\"The operating system: xp, 7\",\r\n                      default=[])\r\n    parser.addoption(\"-V\",\"--ver\",\r\n                      dest=\"browser_version\",\r\n                      action=\"append\",\r\n                      help=\"The version of the browser: a whole number\",\r\n                      default=[])\r\n    parser.addoption(\"-P\",\"--platform\",\r\n                      dest=\"platform\",\r\n                      action=\"append\",\r\n                      help=\"The operating system: Windows 7, Linux\",\r\n                      default=[])\r\n\r\n\r\n\r\n<\/pre>\n
4c. Test file<\/strong>
\nTest file to test the selenium-tutorial-main<\/a> from the previous post<\/p>\n
\r\n#Contents of test file\r\n\"\"\"\r\nQxf2 Services: Utility script to test example form on Browserstack\r\nNOTE: This is a contrived example that was written up to make this blog post clear\r\nWe do not use this coding pattern at our clients\r\n\"\"\"\r\n \r\nfrom selenium import webdriver\r\nfrom selenium.webdriver.common.desired_capabilities import DesiredCapabilities\r\nimport sys,time\r\n\r\n \r\ndef test_example_form(browser,browserstack_flag,browser_version,platform,os_version,):\r\n    \"Test example form\"\r\n    #Create an instance of Driver_Factory\r\n    driver = get_webdriver(browser,browser_version,platform,os_version)\r\n    #Create variables to keep count of pass\/fail\r\n    pass_check_counter = 0\r\n    total_checks = 0\r\n    #Visit the tutorial page\r\n    driver.get('http:\/\/qxf2.com\/selenium-tutorial-main') \r\n    #Check 1: Is the page title correct?\r\n    if(driver.title==\"Qxf2 Services: Selenium training main\"):\r\n        print \"\\n\"\r\n        print (\"Success: Title of the Qxf2 Tutorial page is correct\")\r\n        pass_check_counter += 1\r\n    else:\r\n        print (\"Failed: Qxf2 Tutorial page Title is incorrect\")\r\n    total_checks += 1 \r\n    #Check 2: Fill name, email and phone in the example form\r\n    total_checks += 1\r\n    name_field = driver.find_element_by_xpath(\"\/\/input[@type='name']\")\r\n    name_field.send_keys('Shivahari')\r\n    email_field = driver.find_element_by_xpath(\"\/\/input[@type='email']\")\r\n    email_field.send_keys('test@qxf2.com')\r\n    phone_field = driver.find_element_by_xpath(\"\/\/input[@type='phone']\")\r\n    phone_field.send_keys('9999999999')\r\n    submit_button = driver.find_element_by_xpath(\"\/\/button[@type='submit']\") #Click on the Click me button\r\n    try:\r\n        submit_button.click()\r\n        pass_check_counter += 1\r\n    except Exception,e:\r\n        print str(e)\r\n    #Quit the browser window\r\n    driver.quit() \r\n    #Assert if the pass and fail check counters are equal\r\n    assert total_checks == pass_check_counter \r\n\r\n\r\n#NOTE: This is highly simplified code to make this post illustrative\r\n#We do not use this code at clients\r\n#We use Driver_Factory to return apporpriate drivers within our framework\r\ndef get_webdriver(browser,browser_version,platform,os_version):\r\n    \"Run the test in browser stack browser stack flag is 'Y'\"\r\n    USERNAME = user_name #We fetch values from a conf file in our framework we use on our clients\r\n    PASSWORD = access_key\r\n    if browser.lower() == 'firefox':\r\n        desired_capabilities = DesiredCapabilities.FIREFOX\r\n    if browser.lower() == 'chrome':\r\n        desired_capabilities = DesiredCapabilities.CHROME\r\n    desired_capabilities['os'] = platform\r\n    desired_capabilities['os_version'] = os_version\r\n    desired_capabilities['browser_version'] = browser_version\r\n \r\n    return webdriver.Remote(command_executor='http:\/\/%s:%s@hub.browserstack.com:80\/wd\/hub'%(USERNAME,PASSWORD),\r\n                            desired_capabilities=desired_capabilities)\r\n \r\n<\/pre>\n
\n
5. Run the test<\/h4>\n
Run the test using the command py.test -v -M y -B all<\/code> where,
\n-v<\/code> is the verbose flag,
\n-M<\/code> is the Browserstack flag
\n-B<\/code> is the Browser option<\/p>\n
\"test_run\"<\/a><\/p>\n
\n
And from now, all you need to do to add new browsers, browser versions, platforms is to edit the browser_platform_conf.py file. Happy testing!<\/p>\n
If you liked what you read, know more about Qxf2<\/a>.<\/strong><\/p>\n
\n