Our pytest failure summary until recently showed only Assertion errors when there was a failure. Even though we printed active logs displaying the failed steps on the console and in the log file, we realized that a more useful failure summary would greatly assist in investigating the error and speeding up our possible future actions. This is how the failure […]
Creating a custom pytest plugin for better failure summary
Creating Asynchronous Automation test using Playwright & pytest
Qxf2 explored an advanced capability with Playwright recently – writing asynchronous tests. Given our admiration for pytest we picked the Playwright-pytest plugin for our research. While we were impressed with the pytest integration, we noticed the official and community documentation only featured synchronous examples using pytest. This post will hopefully bridge this gap. We figured working on fitting pytest with […]
Asynchronous API Automation testing using Qxf2’s Framework
You can now create asynchronous API automation test using Qxf2’s API Automation Framework. In this post we will go about explaining: – Why we need Async? – How we modified our Synchronous framework to support Asynchronous HTTP calls – How to create an Async API Automation test using our framework – Cases where Async is not the right fit Note: […]
Working with WebView context in Android mobile automation test
Qxf2 had created an Android mobile app recently to help testers practice mobile test automation. As a next step, we wanted to provide a sample mobile test that comes out of the box with our framework. We worked on automating a scenario – click on options in the mobile app one after another and validate the URL opened in the […]
How we debugged a Flask session error
Qxf2‘s CI test started failing recently, debugging the error helped us learn about an important aspect of serving Flask application using Uvicorn workers. In this short post I will be going over the details on how we went about identifying the error. Understanding the problem We started with the pytest output summary for the failed API test to validate our […]
API Endpoint monitoring using Sematext
Qxf2 had a request to help identify a tool for monitoring API endpoints in one of our client engagements recently. The client wanted a tool with a UI, intuitive configuration and easy maintenance to monitor their changing API endpoints. Additionally, we needed to be able to verify a response body as part of monitoring the endpoint. Our usual tool belt […]
Build a microVM using Firecracker
This post will cover how to create a custom microVM using Firecracker. We will add a simple Flask app to the microVM. By doing so, you will learn how to install custom packages to your own microVM. This post assumes that you have used Firecracker and tried out a basic example already. Why this post? AWS Lambda stands out as […]
Use pytest to run Great Expectations checkpoints
At Qxf2, we’ve successfully integrated Great Expectations into majority of our projects. We now have GitHub workflows in place to run Great Expectations checkpoints before deploying our applications to production. However, as our test suite expanded, we encountered a few challenges: 1. Triggering valid checkpoints. 2. Aggregating checkpoint results. To address these issues, we turned to pytest. In this post, […]
Build a semantic search tool using FAISS
This post provides an overview of implementing semantic search. Why? Because too often, we notice testers skip testing more complex features like autocomplete. This might be ok in most applications. But in domain specific applications, testing autocomplete capabilities of the product is important. Since testers can benefit from understanding implementation details, in this post, we will look at how autocomplete […]
Context-based question answering using LLM
Companies are going to want to query their own internal documents – especially with the rise of LLMs and improvements in AI. Qxf2 has already heard of several CEOs that want to use AI/ML models to glean insights from internal knowledge stores. What does this mean for a tester? Well, you can expect to test such systems in the coming […]