Companies are going to want to query their own internal documents – especially with the rise of LLMs and improvements in AI. Qxf2 has already heard of several CEOs that want to use AI/ML models to glean insights from internal knowledge stores. What does this mean for a tester? Well, you can expect to test such systems in the coming […]
Context-based question answering using LLM
Baseline Model Comparison for Performance Evaluation
Machine learning models evolve. As a tester, how do we know the newer version of the model is better? How do we know that the model did not get worse in other areas? The most intuitive approach would be to design a ‘good’ labelled dataset and then calculate the evaluation score like the F1 score for the model under test. […]
Testing AWS Lambda locally using LocalStack and pytest
Testing loosely coupled microservices requires the use of diverse testing tools. Unlike monoliths, it is challenging to have a complete copy of the production system (System Under Test) for integration testing. Often, developers are hesitant to make changes in the pipeline due to uncertainty about potential downstream impacts. In such cases, it is valuable for developers to test integration components […]
Troubleshooting Common Issues when using Chef InSpec with GitHub Actions
At Qxf2, we have been using Infrastructure tests at our clients for a while now. We recently came up with the idea of adding some infrastructure tests for our own servers and applications within Qxf2. We used Chef Inspec to write and execute our tests. Chef InSpec is a popular open-source framework that helps you test and review your applications […]
Harnessing the Power of Pix2Struct for Testing Images
Machines struggle with verifying things that humans find easy to check. Images, like charts, graphs, diagrams, and maps, are especially challenging for machines to evaluate. In this post, we introduce a method to automatically check important parts of generated images using Pix2Struct, an advanced model from the Transformers library. We were amazed by the model’s effectiveness in our testing of […]
My experience with Auto-GPT
Qxf2 was intrigued by the rising trend of LLMs. We decided to venture beyond ChatGPT. With Auto-GPT’s increasing popularity and the widespread claims, we were eager to explore its capabilities. Given my background as an engineer, I was particularly equipped to dive into the intricacies of Auto-GPT. And as curious tester, I wanted to get a sense of how we […]
Data Validation with ChatGPT: Trials and Insights
We conducted a study to explore the feasibility of using large language models like ChatGPT for performing validation on numerical data. At Qxf2, we execute a set of data quality tests using Great Expectations. Our goal was to assess the efficiency of leveraging ChatGPT to carry out these validations instead. In order to achieve this, I selected two specific scenarios. […]
Testing OpenAI Whisper with Indian Languages
In previous blog, we tested OpenAI Whisper for English language with different accents and observed it did great job. We also provided details about how we generated audios, setup and test details. In this blog, we attempted to test OpenAI Whisper’s capability to transcribe and translate Indian Languages. At Qxf2, our teammates work from different regions of India, and everyone […]
Read and Write csv data using Rust
This post will discuss how to read and write csv data using Rust and the csv crate. We are aiming this at testers who are trying to do useful things with Rust just like us at Qxf2. The two most common operations a tester working with csv would like to do are: reading csv data and dynamically creating csv data. […]
Introduction to Bi-Directional Contract Testing
In our previous blog we gave you an introduction to contract testing, explained Consumer-Driven Contract Testing approach ,setting up Consumer-Driven Contract tests, using Provider states and some of its drawbacks as well. In this blog we will introduce you to the concept of Bi-Directional Contract Testing. We will also show you how to setup Bi-Directional contract tests with an illustrative […]