I recently implemented a Python script to trigger AWS Lambdas conditionally as part of a Microservices app we built at Qxf2 . I experienced multiple invocations of the lambda issue, which made me go in circles. Hence, I decided to share the thinking behind and the fix. Spoiler: The culprit happened to be a default timeout setting in the boto3 […]
Multiple invocations of lambda – The issue and the fix
Triggering Chef InSpec tests with Airflow DAG
In this post, we will show you how to write some Python code to create an Airflow “job” (aka Directed Acyclic Graph or DAG) to trigger some infrastructure tests. By creating this Airflow DAG, everyone in the team that has access to Airflow can run those tests. This post is intended for testers who want to practice writing DAGs. 1. […]
Using Airflow to start and stop EC2 instances
In this post we will show you how to use Airflow to start and stop EC2 instances. Airflow is a popular open-source platform that engineering teams use to manage workflows. It uses a concept called Directed Acyclic Graphs (DAGs) which lets you chain multiple steps into a workflow. Airflow’s popularity is also partly due to an extensive library of operators. […]
Building Custom Operator in Airflow for external API calls
In this blog, I will show how to build a Custom Operator in Airflow to make calls to external APIs. As we do this, we will see how to use secrets in Airflow, make tasks communicate with each other and interpret the output of an SSH Operator. Although this blog refers to posting messages to Skype, all it does is […]