{"id":10230,"date":"2018-12-27T11:30:17","date_gmt":"2018-12-27T16:30:17","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=10230"},"modified":"2019-02-01T03:07:01","modified_gmt":"2019-02-01T08:07:01","slug":"using-requests-instead-of-mechanize-in-our-automation-framework","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/using-requests-instead-of-mechanize-in-our-automation-framework\/","title":{"rendered":"Implementing Python Requests library in our automation testing framework"},"content":{"rendered":"

Qxf2 automation framework<\/a> supports API testing. We used Mechanize<\/a> library to make REST calls for our testing framework. As we migrated from Python 2 to Python 3 and since Mechanize is not supported on Python 3 we had to come up with an alternative module compatible with Python 2 & Python 3. We decided to use Requests<\/a> library. In this post, we will have a look at changes we made and how we implemented Requests library to our testing framework<\/p>\n

\n

Why this post?<\/h3>\n

Requests is the most popular Python library for making HTTP requests. Its the cleanest way we have seen to make API calls programmatically in Python. When creating the first version of Qxf2’s automation framework (way back in 2013!), we chose to use Mechanize to make API calls because all our clients were using CAS as the authentication server and we needed something that could mimic the browser to get past it. But now, most of our clients use different authentication mechanisms and it made sense to start using Requests. We have used Requests library with Python 2 in the past but when implementing Requests in Qxf2’s testing framework we learned more about it and wanted to share the details on changes we built our framework around it.<\/p>\n

\n

Brief on our API testing framework<\/h3>\n

API automation is fast, robust and much easier to maintain than GUI automation. The Qxf2’s API automation framework is based on the object-oriented approach. We built wrappers around Mechanize Python library to make API calls. With Mechanize not being supported in Python 3 we are now building our framework around the Requests library. For more details, on the architecture of our API framework, you can refer to our earlier blog here<\/a>.<\/p>\n

\n

Changes we made to implement Requests library<\/h3>\n

We renamed our Base method from Base_Mechanize to Base_API. Then we substituted the GET, POST, DELETE, PUT wrappers to use Requests wrappers. These methods return JSON response with status code or error details with status code for further debugging. <\/p>\n

Changes to Get request:<\/strong> For Get Request with Mechanize module, we had to create browser instance. But Requests handles sessions without creating browser instance. <\/p>\n

Get request implementation using Mechanize<\/p>\n

   \r\ndef get_browser(self):\r\n        \"Create and return a browser object\"\r\n        browser = mechanize.Browser()\r\n        browser.set_handle_robots(False) \r\n     \r\ndef get(self, url, headers={}):\r\n        \"Mechanize Get request\"\r\n        browser = self.get_browser()\r\n        response = browser.open(mechanize.Request(url))<\/pre>\n
Whereas making get request with Requests is pretty simple and straightforward.<\/p>\n
\r\ndef get(self, url, headers={}):\r\n        \"Get request\"\r\n        try:\r\n            response = requests.get(url=url,headers=headers)<\/pre>\n
\n
Get method using Requests.<\/strong><\/p>\n
Get method in requests is handled as below, which is very straightforward.<\/p>\n
\r\ndef get(self, url, headers={}):\r\n        \"Get request\"\r\n        json_response = None \r\n        error = {}\r\n        try:\r\n            response = requests.get(url=url,headers=headers)\r\n            try:\r\n                json_response = response.json()\r\n            except:\r\n                json_response = None\r\n        except (HTTPError,URLError) as e:\r\n            error = e\r\n            if isinstance(e,HTTPError):\r\n                error_message = e.read()\r\n                print(\"\\n******\\nGET Error: %s %s\" %\r\n                    (url, error_message))\r\n            elif (e.reason.args[0] == 10061):\r\n                print(\"\\033[1;31m\\nURL open error: Please check if the API server is up or there is any other issue accessing the URL\\033[1;m\")\r\n                raise e\r\n            else:\r\n                print(e.reason.args)\r\n                # bubble error back up after printing relevant details\r\n                raise e # We raise error only when unknown errors occurs (other than HTTP error and url open error 10061) \r\n\r\n        return {'response': response.status_code,'text':response.text,'json_response':json_response, 'error': error}<\/pre>\n
Post method using requests.<\/strong><\/p>\n
Post method in requests is handled as below:<\/p>\n
\r\ndef post(self, url,params=None, data=None,json=None,headers={}):\r\n        \"Post request\"\r\n        error = {}\r\n        json_response = None\r\n        try:\r\n            response = requests.post(url,params=params,json=json,headers=headers)\r\n            try:\r\n                json_response = response.json()\r\n            except:\r\n                json_response = None\r\n        except (HTTPError,URLError) as e:\r\n            error = e\r\n            if isinstance(e,HTTPError,URLError):\r\n                error_message = e.read()\r\n                print(\"\\n******\\nPOST Error: %s %s %s\" %\r\n                    (url, error_message, str(json)))\r\n            elif (e.reason.args[0] == 10061):\r\n                print(\"\\033[1;31m\\nURL open error: Please check if the API server is up or there is any other issue accessing the URL\\033[1;m\")\r\n            else:\r\n                print(e.reason.args)\r\n                # bubble error back up after printing relevant details\r\n            raise e\r\n\r\n        return {'response': response.status_code,'text':response.text,'json_response':json_response, 'error': error}<\/pre>\n
Put method using requests.<\/strong><\/p>\n
Put method in requests is handled as below:<\/p>\n
\r\ndef put(self,url,json=None, headers={}):\r\n        \"Put request\"\r\n        error = {}\r\n        response = False\r\n        try:\r\n            response = requests.put(url,json=json,headers=headers)\r\n            try:\r\n                json_response = response.json()\r\n            except:\r\n                json_response = None\r\n\r\n\r\n        except (HTTPError,URLError) as e:\r\n            error = e\r\n            if isinstance(e,HTTPError):\r\n                error_message = e.read()\r\n                print(\"\\n******\\nPUT Error: %s %s %s\" %\r\n                      (url, error_message, str(data)))\r\n            elif (e.reason.args[0] == 10061):\r\n                print(\"\\033[1;31m\\nURL open error: Please check if the API server is up or there is any other issue accessing the URL\\033[1;m\")\r\n            else:\r\n                print(str(e.reason.args))\r\n            # bubble error back up after printing relevant details\r\n            raise e\r\n\r\n        return {'response': response.status_code,'text':response.text,'json_response':json_response, 'error': error}\r\n<\/pre>\n
Delete method using requests.<\/strong><\/p>\n
Delete method in requests is handled as below:<\/p>\n
\r\ndef delete(self, url,headers={}):\r\n        \"Delete request\"\r\n        response = False\r\n        error = {}\r\n        try:\r\n            response = requests.delete(url,headers = headers)\r\n            try:\r\n                json_response = response.json()\r\n            except:\r\n                json_response = None\r\n        \r\n        except (HTTPError,URLError) as e:\r\n            error = e\r\n            if isinstance(e,HTTPError):\r\n                error_message = e.read()\r\n                print(\"\\n******\\nPUT Error: %s %s %s\" %\r\n                      (url, error_message, str(data)))\r\n            elif (e.reason.args[0] == 10061):\r\n                print(\"\\033[1;31m\\nURL open error: Please check if the API server is up or there is any other issue accessing the URL\\033[1;m\")\r\n            else:\r\n                print(str(e.reason.args))\r\n            # bubble error back up after printing relevant details\r\n            raise e\r\n\r\n        return {'response': response.status_code,'text':response.text,'json_response':json_response, 'error': error}<\/pre>\n
Running some example test<\/h3>\n
We are not going to cover this section in detail as we have already covered it in our previous post here<\/a>.
\nWe have created a sample API automation test to verify & validate the changes we keep making to the API framework. You can download the test(with the framework) from Qxf2 automation framework<\/a>. <\/p>\n
The sample API test runs against this application cars app<\/a>. We have created this application using Flask, it is a collection of cars & their attributes. It uses REST API to add or delete cars from the list.<\/p>\n
You can refer the README.md file in the cars-api GitHub page to run the flask application locally and try adding\/deleting cars using the Python request module using your Python interpreter
\nor
\nTo run the sample API test from the Qxf2 automation framework against the hosted app,<\/a> use the following command from the framework’s root dir<\/p>\n
\r\npython -m pytest tests\/test_api_example.py\r\n<\/pre>\n
\n
References<\/h3>\n
1. Requests Document to have more information <\/a><\/p>\n
2. Youtube video to know more about requests<\/a><\/p>\n
\n","protected":false},"excerpt":{"rendered":"
Qxf2 automation framework supports API testing. We used Mechanize library to make REST calls for our testing framework. As we migrated from Python 2 to Python 3 and since Mechanize is not supported on Python 3 we had to come up with an alternative module compatible with Python 2 & Python 3. We decided to use Requests library. In this […]<\/p>\n","protected":false},"author":21,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43,26,18,180,61],"tags":[],"class_list":["post-10230","post","type-post","status-publish","format-standard","hentry","category-api-testing","category-mechanize-2","category-python","category-requests","category-testing"],"_links":{"self":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10230","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/comments?post=10230"}],"version-history":[{"count":69,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10230\/revisions"}],"predecessor-version":[{"id":10675,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10230\/revisions\/10675"}],"wp:attachment":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/media?parent=10230"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/categories?post=10230"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/tags?post=10230"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}