Qxf2 services provides testing services for startups, a majority of our client base is offshore so as to keep them informed about the days Qxf2 is going to be on PTO (public and optional holidays) we follow a practice of notifying them 3 to 7 days in advance. This helps in proper planning and estimating the work. Every year, Qxf2 publishes a list of holidays comprising of public holidays and optional holidays. We will consider this years – 2021 holiday list. Before the development of the Holiday Reminder, it was a completely manual process wherein an employee had to poll the holiday list on a regular basis, post which they had to inform the client about it which was a bit annoying for all employees. I wanted to get rid of this problem so the idea of developing Holiday Reminder Skype Bot came into force.

How I solved the problem?

I wanted some common medium where the employees could be notified about upcoming holidays. Since we use Skype as a medium of communication on daily basis we planned to post upcoming holidays on one of our private skype channels where all employees are members. I wanted to use AWS stack for this project for a simple reason to LEARN !! I found AWS Lambda as a perfect and low-cost service offered by AWS suitable for my project.
My colleague Arunkumar already had a skype sender web service up and running so I just had to get the data from the holidays page, process it, and then just send it to the skype sender web service.

Holiday Reminder AWS lambda function is divided into 3 parts:

1. Scraping Qxf2 Holidays page
2. Finding upcoming Qxf2 Holidays
3. Sending the upcoming holidays to common Qxf2’s skype channel

1. Scraping Qxf2 Holidays page
To scrape Qxf2 Holidays page I chose BeautifulSoup python package using HTML parser.

def fetch_holidays():
    "Fetch holiday list from holidays webpage"
    now = datetime.now()
    if (now.month ==  12 and now.day == 25):
        url = f'https://qxf2.com/{now.year+1}-holidays.html'
    else:
        url = f'https://qxf2.com/{now.year}-holidays.html'
 
    req = requests.get(url)
    soup = BeautifulSoup(req.text, 'html.parser')
    tbody = soup.find("tbody")
    rows = tbody.find_all('tr')
 
    for row in rows:
        cells = row.find_all("td")
        qxf2_holidays['public'][cells[0].text] = ''.join(cells[2].text.split())
 
    ul = soup.find_all('ul')
    li = ul[5].find_all('li')
 
    for items in li:
        items = items.text.split(',')
        qxf2_holidays['optional'][items[0]] = ''.join(items[2].split())</code>

2. Finding upcoming Qxf2 Holidays
The fetched Qxf2 Holidays collection was then looped through and compared with current date and eventually find out if upcoming holiday is 3 or 7 days from now. If so then return the details of the upcoming holiday as a formatted message to be sent to an SQS queue

def get_holidays():
    "Get a holiday message for employees"
 
    fetch_holidays()
    msg = ''
    today = datetime.today().strftime("%d-%b-%Y")
 
    for key,value in qxf2_holidays.items():
        for date,name in qxf2_holidays[key].items():
            delta = datetime.strptime(date, "%d-%b-%Y") - datetime.strptime(today, "%d-%b-%Y") 
            if key == 'optional':
                end_string = 'Kindly take it if you have not taken one this year by marking on calendar and informing client about it'
            else:
                end_string = 'Kindly inform client about it right now'
 
            if delta.days == 7  or delta.days ==3:
                msg += f"\n Reminder - {delta.days} days from now {date} is a {name} {key} holiday. {end_string}"
    return msg.replace("'","")

3. Sending the upcoming holidays to common Qxf2’s skype channel
When a specified SQS queue receives a message it triggers the Skype sender lambda service to send the message to Qxf2’s common skype channel.

def write_message(holiday_reminder_message, channel):
    "Send a message to Skype Sender"
    sqs = boto3.client('sqs')
    message = str({'msg':f'{holiday_reminder_message}', 'channel':channel})
    sqs.send_message(QueueUrl=QUEUE_URL, MessageBody=(message))

Complete code is made available on Github.

Triggering the event

This lambda function is configured such that it would get triggered every day once via a cron job.

What could have been optimized?

I tried to scrape the holiday list only once and save it to a JSON file so that future lambda executions can ignore scrapping if current year holiday data already exists in the JSON file but I observed that only /tmp/ is writable in AWS lambda. The other options were to have a database or to have a JSON file uploaded to the S3 bucket. Since this was a mini project that runs daily for small amounts of data I made a tradeoff but if you are working on a project that requires some file/data updates using AWS Lambda then having a DB or using S3 buckets is the way forward.

What next?

We will be coming up with a series of blog posts on how AWS lambda helped us to solve a couple of our internal manual tasks and save our time. Hope you liked the blogpost !! If you any questions please comment below

Related posts:

Work Anniversary Image – Skype Bot using AWS Lambda Posting messages on a Skype group channel using Python Skype sender as a web service