In this post, we will outline our usual strategy to get an entire team to quickly fill testing holes – like writing infrastructure tests. Why? Because we find ourselves using this approach frequently at clients. Some context: Qxf2 engineers work with early stage products that neglected writing tests in favour of shipping quickly. This means, we almost always enter situations […]
Getting everyone to write infrastructure tests
Integrating CloudWatch Alarms with Skype
At Qxf2, our AWS environment hosts a multitude of applications, so monitoring the AWS services in real time is crucial for maintaining system reliability and performance. We’ve been using CloudWatch alarms to help us watch over our resources, these alarms used to send us emails whenever something went wrong. The problem was, it is often easy to miss these alerts […]
Auditing AWS cloud resources with Chef InSpec
Continuing from the preceding blog, ‘Auditing OS Level Resources with Chef InSpec’ which delved into utilizing the Chef InSpec open-source tool for testing individual servers via OS level resources. Now, we embark into the domain of Chef InSpec’s facility in cloud environments. Chef InSpec extends its support to cloud platforms like AWS, Azure, and GCP. Referring to insights shared in […]
Exploratory Testing with Logmine
This post will discuss how to improve your Exploratory Testing using Logmine. For sometime, I have been looking out for a log analyzer when I started doing exploratory testing with a new product. Logs are a gold mine of information if used well. You can infer common problems, get testing ideas, understand typical behaviour of a product and more from […]
Dataset and Model Evaluation using Deepchecks
At Qxf2, we have always been curious on how to test and validate the datasets and models. A good machine learning team should continuously monitor the model to identify any changes in model performance. You need to be confident that your models are accurate, reliable, and fair. Deepchecks can help you achieve this by providing a comprehensive set of tools […]
Data Generation for Text Classification
I set out to evaluate a ML model (emotion classifier) from a human/user perspective. The heart of my attempt was going to be around designing the right set of data to evaluate the performance of the model. Very quickly, I realized that there is more to this task than meets the eye. In this post, I will share several problems […]
Auditing OS level resources with Chef InSpec
Many of Qxf2‘s clients store their Infrastructure as Code. This means that new infrastructure is spun using code and then the application is deployed. So, as testers, we now have to verify that the right infrastructure was spun up and configured correctly. In this post, I will show you how to use Chef InSpec – a tool to write tests […]
Test Lambda before deployment on CI using LocalStack
In my previous posts, I covered how to write tests for Lambdas, run them on LocalStack and addressed the issue I encountered while working on LocalStack. Now, in this post, I will show you how to set up a GitHub Action job to run tests on LocalStack before deploying Lambdas to production. At Qxf2, we place high importance on CI/CD, […]
Using Airflow to start and stop EC2 instances
In this post we will show you how to use Airflow to start and stop EC2 instances. Airflow is a popular open-source platform that engineering teams use to manage workflows. It uses a concept called Directed Acyclic Graphs (DAGs) which lets you chain multiple steps into a workflow. Airflow’s popularity is also partly due to an extensive library of operators. […]
Robustness Testing of Machine Learning Models
In the world of machine learning, assessing a model’s performance under real-world conditions is important to ensure its reliability and robustness. Real-world data is usually not perfect, it may contain messy data or data with noise, outliers, and variations. During model training, these types of data could be limited, and the model may not have received sufficient training to handle […]