{"id":225,"date":"2014-05-01T05:31:58","date_gmt":"2014-05-01T09:31:58","guid":{"rendered":"http:\/\/qxf2.com\/blog\/?p=225"},"modified":"2017-06-14T02:12:33","modified_gmt":"2017-06-14T06:12:33","slug":"python-mechanize-the-missing-manual","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/python-mechanize-the-missing-manual\/","title":{"rendered":"Python Mechanize – the missing manual"},"content":{"rendered":"

Problem:<\/strong> The online documentation for Mechanize in Python is lacking. Case in point, this question on stackoverflow<\/a> remained unanswered until we added the answer. This post hopes to provide you with the key missing pieces.<\/p>\n

We love Python at Qxf2 Services<\/a>. We chose the Mechanize module to test REST services and automate a lot of our test setup tasks by using REST end points that are used in the products. One of the BIG drawbacks of using the Python module Mechanize is the sparse online support. Let’s hope this post begins to change that. <\/p>\n

\n

1. How to encode parameters in Python<\/strong><\/p>\n

\r\n#Parameters can be URL encoded like this:\r\nmy_params={key1:'value1',key2:'value2'} \r\nmy_params_encoded = urllib.urlencode(my_params)\r\n<\/pre>\n
\r\n#For JSON parameters do this:\r\nmy_params={key1:'value1',key2:'value2'} \r\nmy_params_encoded = json.dumps(my_params)\r\n<\/pre>\n
2. How to do a HTTP GET with Mechanize and Python<\/strong>
\nMechanize supports HTTP GET and POST methods out of the box.<\/p>\n
\r\nurl = \"your desired url\"\r\n#NOTE: browser is the instance of mechanize.Browser()\r\nbrowser.open(mechanize.Request(url,data=my_params_encoded,headers={\"Content-type\":\"application\/json\"}))\r\n<\/pre>\n
Bonus: We just showed you how to make a request with an application\/json header<\/p>\n
3. How to do a HTTP PUT with Mechanize and Python<\/strong>
\nMechanize (the Python module) supports only HTTP GET and POST methods out of the box.  We arrived at this solution by digging into the mechanize code packages to find out where mechanize was setting the HTTP method. We noticed that when we call mechanize.Request, we are using the Request class in _request.py which in turn is extending the Request class in _urllib2_fork.py. The http method is actually set in get_method of the Request class in _urllib2_fork.py. Turns out get_method in _urllib2_fork.py was allowing only GET and POST methods. To get past this limitation, we ended up writing my own put and delete classes that extended mechanize. Request but over-rode get_method() only. <\/p>\n
\r\nclass PutRequest(mechanize.Request):\r\n    \"Extend the mechanize Request class to allow a http PUT\"\r\n    def get_method(self):\r\n        return \"PUT\"\r\n\r\n\r\n#NOTE: browser is the instance of mechanize.Browser()\r\nbrowser.open(PutRequest(url,data=sub_level_params_encoded,headers={\"Content-type\":\"application\/json\"}))\r\n<\/pre>\n
4. How to do a HTTP DELETE with Mechanize and Python<\/strong><\/p>\n
\r\nclass DeleteRequest(mechanize.Request):\r\n    \"Extend the mechanize Request class to allow a http DELETE\"\r\n    def get_method(self):\r\n        return \"DELETE\"\r\n\r\n\r\n#NOTE: browser is the instance of mechanize.Browser()\r\nbrowser.open(DeleteRequest(url))\r\n<\/pre>\n
\n
Other useful links to learn Mechanize+Python:<\/strong>
\n1. If you are new to Mechanize, use this link<\/a>
\n2. Using Mechanize to login to CAS<\/a> <\/p>\n
If you have questions regarding any of these solutions, drop a comment below. Qxf2<\/a> like helping testers!<\/p>\n
\n
WARNING:<\/strong> Mechanize has its limitations as an automated testing tool. The biggest drawback for web testers is that Mechanize does not mimic rendering of executed JavaScript in the browser. So DOM elements that are rendered by executing JavaScript cannot be accessed by Mechanize.<\/p>\n
\n