You might have come across a situation where you felt the need of creating a virtualized service for your testing purpose. This is especially true if your application is integrating with third party applications. One such situation led me to creating a virtualized service for MailChimp and I am going to talk about it in this blog. This post will cover the groundwork needed to be done by a team before they can leverage the convenience of using virtualized services for testing. In a future post (or two?), I will show you how to use MockLab to create the virtualized service.
Why we wanted to virtualize MailChimp
We have an internal application at Qxf2 that integrates with MailChimp. We wanted to implement some tests to verify some “what if” scenarios. What if someday the MailChimp is down or it returns some errors? Would the application fail gracefully in these instances? Would the user be notified about why the failure happened? We obviously could not bring down MailChimp to check these scenarios. We also could not setup a separate instance/account of MailChimp. So I set about creating a virtualized service for MailChimp in order to test these scenarios.
Figuring out the MailChimp endpoints we want to mimic
The first step was to figure out the MailChimp endpoints we needed to virtualize. This was surprisingly hard because our application was not directly calling the MailChimp API. We were using a third party library to make these calls. In our case, we had access to the code that used the (third party) MailChimp marketing module. As a tester, you might not always have access to code this way. In these cases, collaborating with your developers to figure out which endpoints need to be virtualized is the way to go.
We found out that, the methods that interest us are – create_campaign and set_campaign_content. From the methods, we found out the respective endpoints viz. /campaigns which is a POST method and /campaigns/{campaign_id}/content which is a PUT method.
Finding out the return objects for the endpoints
Once, we knew the endpoints, we had to find the return objects. We did it through the following documentation for /campaigns
In the POST /campaigns, click the ‘Expand’ and scroll down below to the Success Response section and check the return type. In our case, digging through the code, we found that we would only need the return object as “id”: ”string” and not the entire response JSON. This is one more place where your developers can help you.
Similarly, For /campaigns/{campaign_id}/content
In the PUT, /campaigns/{campaign_id}/content, click the ‘Expand’ and scroll down below to the Success Response section and check the return type.
Creating a service to mimic the endpoints via mocklab
Now that we knew what endpoints to mimic and the structure of the JSON being returned, we started off by looking for some virtualization tool that would allow us to virtualize the service. We came across MockLab which had an easy-to-use user interface. Another good use case is that it allows the teams to share the stubs created. MockLab provides a publicly accessible base URL for the mocked service.
Trying to change local DNS
Our first attempt to use the virtualized service did not involve changing the application. We first tried changing the local DNS. But the challenge was we did not have the IPs we needed – either a MailChimp IP nor the MockLab IP. We thought of virtualizing the service with a simple Flask app (instead of MockLab) too but it had a port associated with it and this approach of changing the local DNS would not work. The next option we had was to set up a proxy but we didn’t like to go down that path because that would add an extra layer of complexity to our CI tests.
Having done this work, we decided to ask the developers to change the application so we could configure it to use our virtualized service on local and test environments.
Improving the testability of the application
When the developers looked into the issue, they realized something. Our application code (rather the third party library) had tightly coupled the base URL of MailChimp with the library. This meant, we needed some code changes to make our application more testable. Testers, advocating for this sort of change falls well within our job responsibilities. We might not be able to make the code changes but we should understand enough technical details about the application in order to advocate for changes like these.
The developer modified a wrapper around the third party library that made MailChimp API calls such that it could optionally take a URL other than the hardcoded MailChimp API base URL. They also made the application capable of taking the MailChimp API URL from an environment variable. If the environment variable was set, the application would read it and then pass it to the helper.
Now, the user can set the virtualized URL in an environment variable MAILCHIMP_VIRTUALIZATION to point to the base URL of your virtualized service.
export MAILCHIMP_VIRTUALIZATION = https://$PROVIDED_BY_MOCKLAB.mocklab.io/ |
Now, once the user makes a call to the MailChimp, they can check the request logs in the mocklab.
Please note: The developers should make the required changes to make the application under test more testable.
Exploratory Testing
Once the application had been modified so we could use our virtualized service instead of the real MailChimp, we set out to run some exploratory tests.
One of the vectors of our exploration involved checking what our application would do when MailChimp returned an error. We wanted to simulate all the response error codes which MailChimp supports in errors for our applications.
We created stubs for each of the errors and have made sure that the request body contains the required keyword which if present would return back the necessary error code. The response status would contain the error code along with the body containing the details of the error. We will soon write a tutorial on how to configure MockLab so you can do something similar.
For example, with error 500, the response body would look like this.
{
"type": "https://mailchimp.com/developer/marketing/docs/errors/",
"title": "Internal Server Error",
"status": 500,
"detail": "A deep, internal error has occurred during the processing of your request. Please contact support.",
"instance": "995c5cb0-3280-4a6e-808b-3b096d0bb219"
} |
Our exploratory testing uncovered a slew of bugs with a common theme – our application was not handling most of the error responses correctly!
Conclusion
Service virtualization is very useful when we want to test an integration with an external service. It allows us to cut off the dependency with the external service and check how the application would behave when the service is down or throws some error because, in the real world, these workflows are difficult to test otherwise.
As we use MailChimp weekly once for our newsletter, we would never know what would be the scenario if the service throws some error and how the application would behave in that scenario. Now with the service virtualization done, our team has some confidence that we handle the integration with MailChimp properly.
My journey in software testing began with Calabash and Cucumber, where I delved into Mobile Automation using Ruby. As my career progressed, I gained experience working with a diverse range of technical tools. These include e-Discovery, Selenium, Python, Docker, 4G/5G testing, M-CORD, CI/CD implementation, Page Object Model framework, API testing, Testim, WATIR, MockLab, Postman, and Great Expectation. Recently, I’ve also ventured into learning Rust, expanding my skillset further. Additionally, I am a certified Scrum Master, bringing valuable agile expertise to projects.
On the personal front, I am a very curious person on any topic. According to Myers-Briggs Type Indicator, I am described as INFP-T. I like playing soccer, running and reading books.
