{"id":10392,"date":"2018-12-20T10:09:16","date_gmt":"2018-12-20T15:09:16","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=10392"},"modified":"2018-12-20T10:09:16","modified_gmt":"2018-12-20T15:09:16","slug":"reportportal-integration-with-pytest","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/reportportal-integration-with-pytest\/","title":{"rendered":"ReportPortal integration with pytest and pytest markers"},"content":{"rendered":"

Problem:<\/strong> If QA wants to make a difference with their testing and influence decision-making, we should convey our test results clearly. Hence test reporting becomes an important part of testing.
\n 
\nThere are several test reporting tools available and recently many AI based test reporting tools are getting popular. Integrating with these reporting tools is useful. In this post, we will show you how to integrate a Python test which uses pytest<\/a> as the test runner to integrate with a popular test reporting tool ReportPortal<\/a>. ReportPortal helps to acquire, aggregate and analyze test reports to ascertain release health better. It also shows us trends over time.
\n 
\nThank you Lea Anthony<\/strong> of Secure Code Warrior<\/a> for introducing us to ReportPortal! <\/p>\n

\n

Why this post?<\/h3>\n

One of our clients was planning on using ReportPortal as the test reporting tool. The plain, vanilla integration of ReportPortal with pytest (which is the default test runner we use at Qxf2 Services<\/a>) is straightforward and covered well by ReportPortal’s docs. The plugin for reporting test results of Pytest to the ReportPortal needs the pytest.ini to be updated at a root level. This was a problem for us on two counts:
\na) we wanted to be able to have different ReportPortal projects (called “launch name” in ReportPortal) for different pytest markers. We know the marker only after the pytest.ini has been initialized. So the launch name used couldn’t be conditionally set in pytest.ini
\nb) we like the config parameters to be set somewhere other than declaring it directly in the pytest.ini file<\/p>\n

\n

How to integrate ReportPortal with pytest markers<\/h3>\n

To illustrate how to integrate ReportPortal with pytest markers, we will be running two tests which fall under API and GUI test groups. When we use the marker as gui_test<\/em>, the GUI test should run and report to ReportPortal with a launch name as gui_test<\/em>. When we use the marker as api_test<\/em>, the API test should run and report to ReportPortal with a launch name as api_test<\/em>.<\/p>\n

Below are the steps needed to achieve this
\n1. Set up ReportPortal
\n2. Create two tests with markers
\n3. Set up pytest plugin for ReportPortal
\n4. Create a conftest
\n5. Run the test<\/p>\n

1. Set up ReportPortal<\/strong>
\nYou can go about downloading and setting up ReportPortal<\/a> as mentioned or try their Demo server<\/a>. I have used the demo server for this blog. Navigate to profile link and make a note of UUID, endpoint and project details.<\/p>\n

\"ReportPortal_Profile\"<\/a>
\n2. Create two tests with markers<\/strong><\/p>\n

For the demo, we will create dummy tests with just the function names. We will add a couple of markers<\/a> for gui and api tests.
\nNote: With pytest markers, you can restrict a test run to only run tests only with specific markers<\/p>\n

\r\nimport pytest\r\n\r\n@pytest.mark.gui_test()\r\ndef test_gui():\r\n    print (\"GUI Test\")\r\n    pass\r\n\r\n@pytest.mark.api_test()\r\ndef test_api():\r\n    print (\"API Test\")\r\n    pass\r\n<\/pre>\n
3. Set up pytest plugin for ReportPortal<\/strong>
\nThe agent-python-pytest<\/a> plugin allows you to integrate pytest with ReportPortal. To install the plugin you can run<\/p>\n
\r\npip install pytest-reportportal\r\n<\/pre>\n
The below example is the default way suggested for integrating with ReportPortal. You can skip this if you want to use a more flexible pytest hook implementation instead(discussed next). <\/p>\n
Add the ReportPortal details like UUID, endpoint and project detail which you noted in step 2 and use it in pytest.ini file. Example:<\/p>\n
\r\n[pytest]\r\nrp_uuid = fb586627-32be-47dd-93c1-678873458a5f\r\nrp_endpoint = http:\/\/192.168.1.10:8080\r\nrp_project = user_personal\r\nrp_launch = AnyLaunchName\r\n<\/pre>\n
Note: The UUID and other details mentioned above are for my Demo server and needs to be updated as per your ReportPortal details.<\/p>\n
4. Creating a conftest<\/strong>
\nHowever, instead of the above example, we can to create a conftest file. If you use pytest regularly, you probably already have one! If not, create a conftest.py file where we can add the ini details conditionally which ReportPortal requires<\/p>\n
a. Use the parser.addini function inside pytest_addoption to add rp_uuid, rp_endpoint, rp_project and rp_launch ini option
\nb. Inside pytest_configure hook use config._inicache to add the parsed command line option to the ini value created above<\/p>\n
\r\nimport pytest\r\n\r\ndef pytest_addoption(parser):\r\n    # Method to add the option to ini\r\n    parser.addini(\"rp_uuid\",'help',type=\"pathlist\")\r\n    parser.addini(\"rp_endpoint\",'help',type=\"pathlist\")\r\n    parser.addini(\"rp_project\",'help',type=\"pathlist\")\r\n    parser.addini(\"rp_launch\",'help',type=\"pathlist\")              \r\n\r\n@pytest.hookimpl()\r\ndef pytest_configure(config):\r\n    # Sets the launch name based on the marker selected.\r\n    suite = config.getoption(\"markexpr\")\r\n    try:\r\n        config._inicache[\"rp_uuid\"]=\"f9ffd233-1c12-4bcc-87cd-89bc62865b4d\"\r\n        config._inicache[\"rp_endpoint\"]=\"http:\/\/web.demo.reportportal.io\"\r\n        config._inicache[\"rp_project\"]=\"default_personal\"\r\n        if suite == \"gui_test\":            \r\n            config._inicache[\"rp_launch\"]=\"gui_test\"\r\n        elif suite == \"api_test\":\r\n            config._inicache[\"rp_launch\"]=\"api_test\" \r\n           \r\n    except Exception,e:\r\n        print (str(e))\r\n\r\n<\/pre>\n
5. Running the test<\/strong>
\nYou can run the test in ReportalPortal using any of the markers as specified below<\/p>\n
\r\npy.test -m gui_test --reportportal\r\n<\/pre>\n
\"Run_Test_ReportPortal\"<\/a><\/p>\n
\"ReportPortal_Launch\"<\/a><\/p>\n
Hurray, now we are able to run tests and analyze report for specific tests in ReportPortal. Happy testing…<\/p>\n
\n
References:<\/h3>\n
1. Working with plugins and conftest files<\/a>
\n2. Adding INI file option<\/a>
\n3. Allow to dynamically modify all ini-values from plugins<\/a><\/p>\n
\n","protected":false},"excerpt":{"rendered":"
Problem: If QA wants to make a difference with their testing and influence decision-making, we should convey our test results clearly. Hence test reporting becomes an important part of testing.   There are several test reporting tools available and recently many AI based test reporting tools are getting popular. Integrating with these reporting tools is useful. In this post, we […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[107,18,181],"tags":[],"class_list":["post-10392","post","type-post","status-publish","format-standard","hentry","category-pytest","category-python","category-reportportal"],"_links":{"self":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10392","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/comments?post=10392"}],"version-history":[{"count":19,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10392\/revisions"}],"predecessor-version":[{"id":10476,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10392\/revisions\/10476"}],"wp:attachment":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/media?parent=10392"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/categories?post=10392"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/tags?post=10392"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}