{"id":3024,"date":"2015-10-20T01:04:56","date_gmt":"2015-10-20T05:04:56","guid":{"rendered":"http:\/\/qxf2.com\/blog\/?p=3024"},"modified":"2018-03-14T10:33:19","modified_gmt":"2018-03-14T14:33:19","slug":"reporting-to-testrail-using-python","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/reporting-to-testrail-using-python\/","title":{"rendered":"Reporting automation results to TestRail using Python"},"content":{"rendered":"

We have been hard at work integrating our Python based GUI automation framework<\/a> (that uses the Page Object pattern) with a test case management tool for reporting purposes. In this post we will show you how to report the results of your automated checks to a visual test case management tool – TestRail<\/a>. TestRail offers a free trial<\/a> – so you can sign up for free and follow this post.<\/p>\n

\n

Why this post?<\/h3>\n

In most automated checks, we are able to judge the outcome as ‘pass’ or ‘fail’ by comparing the expected result to the actual result. Tracking, reporting and sharing the test results is important. Excel and email rarely scale and need a lot of maintenance. Continuous Integration servers are great for engineers – but non-technical stakeholders prefer something more visual. At Qxf2, we love TestRail – a visual test case management software. We got you started with TestRail in an earlier post<\/a>. In this post, we are going to show you how automation can update test results directly on to TestRail.<\/p>\n

\n

Report automation results to TestRail using Python<\/h3>\n

We’ll need to perform the following steps to report test results automatically to TestRail
\n1. Get the TestRail API
\n2. Connect (authentication) to your TestRail instance
\n3. Write the logic to update test results to TestRail
\n4. Identify your test case id on TestRail
\n5. Modify your test script to report to TestRail
\n 
\n 
\n1. Get the TestRail API<\/strong>
\nTestRail provides a useful REST API for automated tests to directly report results on to TestRail. The API provides the functionality to authenticate API requests, provides seamless JSON encoding\/decoding and has generic support for read and write requests. We will write some useful wrappers around TestRail’s official API binding<\/a> for Python.
\nBefore we proceed, please download the testrail.py<\/strong> from here<\/a> and follow the installation instructions<\/a>.<\/p>\n

2. Connect to your TestRail instance<\/strong>
\nWe prefer separating any credentials we use into separate files<\/a>. For this tutorial, we have created testrail.env<\/strong> that looks something like this:
\ntestrail.env<\/p>\n

\r\nTESTRAIL_URL=https:\/\/.testrail.com\r\nTESTRAIL_USER=test@abc.xyz\r\nTESTRAIL_PASSWORD=test\r\n<\/pre>\n
We then created a file that wraps around TestRail’s API called TestRail.py<\/strong>. In this file we implemented a method to connect to the TestRail instance.<\/p>\n
\r\n\"\"\"\r\nTestRail integration\r\n\"\"\"\r\nimport dotenv,os,testrail,Conf_Reader\r\n\r\ndef get_testrail_client():\r\n    \"Get the TestRail account credentials from the testrail.env file\"\r\n    testrail_file = os.path.join(os.path.dirname(__file__),'testrail.env')\r\n    #Get the TestRail Url\r\n    testrail_url = Conf_Reader.get_value(testrail_file,'TESTRAIL_URL')\r\n    client = testrail.APIClient(testrail_url)\r\n    #Get and set the TestRail User and Password\r\n    client.user = Conf_Reader.get_value(testrail_file,'TESTRAIL_USER')\r\n    client.password = Conf_Reader.get_value(testrail_file,'TESTRAIL_PASSWORD')\r\n\r\n    return client\r\n<\/pre>\n
Now we have an object that can connect to your TestRail instance!<\/p>\n
3. Write the logic to update test results to TestRail<\/strong>
\nTestRail needs to know two things to go an update a test case result:
\na. the test run id
\nb. the test case id<\/p>\n
We need both because we can have multiple test runs for the same set of test cases. So our script should be able to update the test results into TestRail for a particular test run and specific test case. For this we need to know the test case id and the test run id. We implemented the update_testrail<\/strong> method. This method can be called from the test script with appropriate parameters so that the test results are recorded in TestRail for the specific case id and run id.<\/p>\n
\r\ndef update_testrail(case_id,run_id,result_flag,msg=\"\"):\r\n    \"Update TestRail for a given run_id and case_id\"\r\n    update_flag = False\r\n    #Get the TestRail client account details\r\n    client = get_testrail_client()\r\n\r\n    #Update the result in TestRail using send_post function. \r\n    #Parameters for add_result_for_case is the combination of runid and case id. \r\n    #status_id is 1 for Passed, 2 For Blocked, 4 for Retest and 5 for Failed\r\n    status_id = 1 if result_flag is True else 5\r\n\r\n    if run_id is not None:\r\n        try:\r\n            result = client.send_post(\r\n                'add_result_for_case\/%s\/%s'%(run_id,case_id),\r\n                {'status_id': status_id, 'comment': msg })\r\n        except Exception,e:\r\n            print 'Exception in update_testrail() updating TestRail.'\r\n            print 'PYTHON SAYS: '\r\n            print e\r\n        else:\r\n            print 'Updated test result for case: %s in test run: %s with msg:%s'%(case_id,run_id,msg)\r\n        \r\n    return update_flag\r\n<\/pre>\n
NOTE:<\/strong> Test case ids stay constant across test runs in TestRail. However each test run is assigned a unique id. To automatically obtain your run id based on your test run name, you can implement something similar to this code snippet:<\/p>\n
\r\n    def get_project_id(project_name):\r\n        \"Get the project ID using project name\"\r\n        client = get_testrail_client()\r\n        project_id=None\r\n        projects = client.send_get('get_projects')\r\n        for project in projects:\r\n            if project['name'] == project_name: \r\n                project_id = project['id']\r\n                #project_found_flag=True\r\n                break\r\n        return project_id\r\n\r\n    def get_run_id(test_run_name,project_name):\r\n        \"Get the run ID using test name and project name\"\r\n        run_id=None\r\n        client = get_testrail_client()\r\n        project_id = get_project_id(project_name)\r\n        try:\r\n            test_runs = client.send_get('get_runs\/%s'%(project_id))\r\n        except Exception,e:\r\n            print 'Exception in update_testrail() updating TestRail.'\r\n            print 'PYTHON SAYS: '\r\n            print e\r\n            return None\r\n        else:\r\n            for test_run in test_runs:\r\n                 if test_run['name'] == test_run_name:\r\n                     run_id = test_run['id']\r\n                     break\r\n            return run_id\r\n<\/pre>\n
4. Identify your test case id on TestRail<\/strong>
\nTo obtain the test case id, simply visit your project on TestRail, navigate to Test Cases<\/strong>  section and look for the integer in the ID column next to your test case.<\/p>\n
\"TestRail<\/a><\/p>\n
5. Modify your test script to report to TestRail<\/strong>
\nNow you can modify your existing test scripts to report to TestRail. Here is a code sample that shows you how to use the method you wrote earlier:
\nTest Script<\/p>\n
\r\n#Code to automate test case goes here\r\n.\r\n.\r\n#Update TestRail\r\n    case_id = 125 #Case id of your test case \r\n    if (result_flag):\r\n        msg = \"Successfully updated the example form\"\r\n    else:\r\n        msg = \"Failed to update the example form\"\r\n    Test_Rail.update_testrail(case_id,test_run_id,result_flag,msg=msg)\r\n<\/pre>\n
\n
Hope this post helps you offload the manual task of updating the test results in TestRail! Get all this code for free in our open-sourced Python test automation framework<\/a> based on the page object pattern. We strongly recommend you check it out!<\/p>\n
\n