API Testing with Python Mechanize

This is the third part in our series on API testing. Rather than focus on traditional approaches to API testing, we have decided to arm you with tools that let you interact with the API at different levels of abstractions. In part I we showed you how to inspect the API calls of any web application. In part II, we gave examples of UI tools that could help you interact and mimic the API calls. In this third part, we will show you how you can use a higher level scripting language to interact with the API and write scripts that are repeatable and robust.

For this tutorial, we have chosen to use the Python module Mechanize. We use these API test scripts in a couple different ways. One, to test the API themselves. Two, to quickly setup complex test scenarios. Using scripts that work off the API to setup test scenarios is more robust and much faster than using GUI automation.


Why Mechanize?

We love Python. So naturally this tutorial uses Python. There are plenty of good Python modules to use for API tests. For simple tasks, we love the Requests module which has a very clean and intuitive interface. For this tutorial, we have chosen to use Mechanize. Mechanize emulates the browser and makes it easy to handle authentication, sessions and cookies. It also helps us get past some sticky areas where the application API is not very clean and you need the browser to make requests. Of course, Mechanize comes with some drawbacks – like a lack of support for PUT, DELETE, PATCH HTTP methods. Qxf2 went through the source code and wrote our own ‘Missing Manual‘ that now makes Python Mechanize extremely powerful. We have used Python Mechanize for the past two years now and have found it suitable for our API testing needs.


Overview

In part 1, we used the developer tools to inspect the http requests/responses. We used DropTask as the application under test. DropTask is a free web based task and project management application that offers a visual approach towards managing daily workloads. For this tutorial, we will:
1. Login to DropTask
2. Get the account details
3. Delete an existing task

While this workflow may seem odd/not-realistic to professional testers, we chose these specific steps to keep this blog post at a reasonable length while still illustrating some common ways in which we expect you to use Mechanize when testing your web application. We highly recommend you scan Part I of our tutorial if you have not yet already read it.


Implementation details using Python Mechanize

1. Login to DropTask
Here the browser url is set to DropTask login page. We make a POST request with the encoded parameters (email and password).
content-type login request

def login(self,username,password):
        " Login to Droptask "
        #Setup the login POST request
        my_params={'email':username,'password':password}                
        params_encoded = urllib.urlencode(my_params) //Content-Type = 'application/x-www-form-urlencoded'
        self.browser.method='POST'
        login_response= self.browser.open(self.url,data=params_encoded)
 
        #Verifying that the login was successful.
        #We have chosen to check if we were taken back to the login page or not
        forms = mechanize.ParseResponse(login_response, backwards_compat=False)        
        if (len(forms)!=0) and (forms[0].find_control("password") is not None): #If we were redirected to the login page forms[0].find_control("password") would not be None indicating that Login failed
            self.write("Login failed")
            return False
        else:
            self.write("Login success")
            return True

2. Get the account details
On successful login, we make a GET request with Unix timestamp in the url parameter to get the account details about templates, tasks, users, groups, workspaces etc. This call is an example where imagination and deduction shine through. We looked at the GET call in developer tools and were wondering what the cryptic number was. After trying a few calls in rapid succession and noticing that the number only slightly changed, we were able to guess that it had something to do with a timestamp. The number beginning with ’14’ made us guess its a Unix timestamp. And the guess turned out to be right! Given that DropTask seems to support real-time collaboration (see the calls to pubnub in the screenshots?), it makes sense for the GET request to indicate that it wants the latest account details.

def get_account_details(self):
        " Get account details."
        #Current unix timestamp in milliseconds is needed!
        curr_time = int(time.time())*1000
        url = "https://www.droptask.com/v1/account?%s"%curr_time
        response = self.browser.open(mechanize.Request(url=url))
        self.account_details = json.loads(response.read())
        return json.dumps(self.account_details)

3. Delete a given task
Mechanize only supports GET and POST methods out of the box. To extend Mechanize to support the other HTTP methods, you can refer to our Mechanize tutorial. To keep this post complete, you need to add this class to your test file:

class Mechanize_Delete_Request_class(mechanize.Request):
    "Extend the mechanize Request class to allow a HTTP DELETE"
    def get_method(self):
        return "DELETE"

We are assuming you already have a task in a particular project. If not create one in your workspace/project. To delete the task we need to make a DELETE request. Since different projects could have tasks of the same name, our method takes 2 parameters -project name and task name. Inspecting the delete API calls, we notice that DropTask’s API expects a workspace id and the task id to perform a delete. Using the project name we get the workspace id from the account details.

json response workspace

This workspace id and the task name(title) help us to get the task id. Using the task id we make a DELETE request.

json response tasks

Sample DELETE request when a task is deleted using the browser.
delete task

def get_task_id(self,task_name,workspace_name):
        " Returns the task id for a given task in a project"
        self.task_id = None
        workspace_id = self.get_workspace_id(workspace_name)
 
        if workspace_id is not None:            
            for tasks in self.account_details['tasks']:
                if tasks['workspace']== self.workspace_id and tasks['title']== task_name:
                    self.task_id= tasks['_id']
                    break
 
        return self.task_id
 
def delete_task(self,task_name,workspace_name):
        " Deletes the given task by finding the task id"
        self.task_id = self.get_task_id(task_name,workspace_name)        
        result = False
 
        if self.task_id is not None:
            self.browser.method='DELETE'
            del_task_res= self.browser.open(Mechanize_Delete_Request_class("https://www.droptask.com/v1/tasks/%s" % self.task_id))
            self.write("Response Code for delete task %s is: %s"%(task_name,del_task_res.code))
            return del_task_res.code
        else:
            self.write("Could not find the given task :%s in workspace :%s"%(task_name,workspace_name))
            return None

Notes

1. We did not have an opportunity to show you how to check response codes. With Mechanize, the response code for given response object is simply response.code
 
2. To the testing team at DropTask, as part of preparing for this post, we ended up writing a lot more code. We created a small library using the facade pattern to quickly automate many of your API calls. We believe it will help you setup complex test scenarios quickly and robustly. Hit us up at mak@qxf2.com and we can share it with you.
 
This ends our three part series on arming you with a range of tools for performing API testing. We would love to hear your feedback and questions in the comments section.


A weekly newsletter for testers


View a sample



Vrushali Toshniwal
My journey as a tester started at Sun Microsystems (now Oracle). I was part of the testing and sustaining team for the Portal Server and Identity Management products. My first assignment was to test the Rewriter module. I enjoyed understanding the big picture, writing test cases, finding bugs and sometimes suggesting the fix too! I was hooked onto testing. Testing felt natural and intuitive to me. I am technically inclined and can write automation in Java, C++, Perl and Python. I am well versed with SilkTest, Selenium, Appium and Selendroid. I am a Computer Science graduate from BITS-Pilani. I love travelling and listening to music.

© 2013-2017, Vrushali Toshniwal. All rights reserved.

One Comment

Leave a Reply

Your email address will not be published.