{"id":15036,"date":"2021-04-01T07:10:19","date_gmt":"2021-04-01T11:10:19","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=15036"},"modified":"2021-04-01T07:10:49","modified_gmt":"2021-04-01T11:10:49","slug":"holiday-reminder-skype-bot","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/holiday-reminder-skype-bot\/","title":{"rendered":"Holiday Reminder Skype Bot"},"content":{"rendered":"

Qxf2 services<\/a> provides testing services for startups, a majority of our client base is offshore so as to keep them informed about the days Qxf2 is going to be on PTO (public and optional holidays) we follow a practice of notifying them 3 to 7 days in advance. This helps in proper planning and estimating the work. Every year, Qxf2 publishes a list of holidays comprising of public holidays and optional holidays. We will consider this years – 2021 holiday list<\/a>. Before the development of the Holiday Reminder, it was a completely manual process wherein an employee had to poll the holiday list on a regular basis, post which they had to inform the client about it which was a bit annoying for all employees. I wanted to get rid of this problem so the idea of developing Holiday Reminder Skype Bot came into force.<\/p>\n

\n

How I solved the problem?<\/h3>\n

I wanted some common medium where the employees could be notified about upcoming holidays. Since we use Skype as a medium of communication on daily basis we planned to post upcoming holidays on one of our private skype channels where all employees are members. I wanted to use AWS stack for this project for a simple reason to LEARN !! I found AWS Lambda as a perfect and low-cost service offered by AWS suitable for my project.
\nMy colleague Arunkumar<\/a> already had a skype sender web service<\/a> up and running so I just had to get the data from the holidays page, process it, and then just send it to the skype sender web service. <\/p>\n

\n
Holiday Reminder AWS lambda function is divided into 3 parts: <\/h5>\n

1. Scraping Qxf2 Holidays page
\n2. Finding upcoming Qxf2 Holidays
\n3. Sending the upcoming holidays to common Qxf2’s skype channel <\/p>\n

1. Scraping Qxf2 Holidays page
\nTo scrape Qxf2 Holidays page I chose BeautifulSoup<\/a> python package using HTML parser.<\/p>\n

def fetch_holidays():\r\n    \"Fetch holiday list from holidays webpage\"\r\n    now = datetime.now()\r\n    if (now.month ==  12 and now.day == 25):\r\n        url = f'https:\/\/qxf2.com\/{now.year+1}-holidays.html'\r\n    else:\r\n        url = f'https:\/\/qxf2.com\/{now.year}-holidays.html'\r\n\r\n    req = requests.get(url)\r\n    soup = BeautifulSoup(req.text, 'html.parser')\r\n    tbody = soup.find(\"tbody\")\r\n    rows = tbody.find_all('tr')\r\n    \r\n    for row in rows:\r\n        cells = row.find_all(\"td\")\r\n        qxf2_holidays['public'][cells[0].text] = ''.join(cells[2].text.split())\r\n        \r\n    ul = soup.find_all('ul')\r\n    li = ul[5].find_all('li')\r\n    \r\n    for items in li:\r\n        items = items.text.split(',')\r\n        qxf2_holidays['optional'][items[0]] = ''.join(items[2].split())<\/code>\r\n<\/pre>\n
2. Finding upcoming Qxf2 Holidays
\nThe fetched Qxf2 Holidays collection was then looped through and compared with current date and eventually find out if upcoming holiday is 3 or 7 days from now. If so then return the details of the upcoming holiday as a formatted message to be sent to an SQS queue<\/p>\n
\r\ndef get_holidays():\r\n    \"Get a holiday message for employees\"\r\n    \r\n    fetch_holidays()\r\n    msg = ''\r\n    today = datetime.today().strftime(\"%d-%b-%Y\")\r\n    \r\n    for key,value in qxf2_holidays.items():\r\n        for date,name in qxf2_holidays[key].items():\r\n            delta = datetime.strptime(date, \"%d-%b-%Y\") - datetime.strptime(today, \"%d-%b-%Y\") \r\n            if key == 'optional':\r\n                end_string = 'Kindly take it if you have not taken one this year by marking on calendar and informing client about it'\r\n            else:\r\n                end_string = 'Kindly inform client about it right now'\r\n            \r\n            if delta.days == 7  or delta.days ==3:\r\n                msg += f\"\\n Reminder - {delta.days} days from now {date} is a {name} {key} holiday. {end_string}\"\r\n    return msg.replace(\"'\",\"\")<\/pre>\n
3. Sending the upcoming holidays to common Qxf2’s skype channel
\nWhen a specified SQS queue receives a message it triggers the Skype sender lambda service to send the message to Qxf2’s common skype channel.<\/p>\n
def write_message(holiday_reminder_message, channel):\r\n    \"Send a message to Skype Sender\"\r\n    sqs = boto3.client('sqs')\r\n    message = str({'msg':f'{holiday_reminder_message}', 'channel':channel})\r\n    sqs.send_message(QueueUrl=QUEUE_URL, MessageBody=(message))\r\n<\/pre>\n
Complete code is made available on Github<\/a>.<\/p>\n
\n
Triggering the event<\/h3>\n
This lambda function is configured such that it would get triggered every day once via a cron job.<\/p>\n
\"Holiday<\/a><\/p>\n
What could have been optimized?<\/h3>\n
I tried to scrape the holiday list only once and save it to a JSON file so that future lambda executions can ignore scrapping if current year holiday data already exists in the JSON file but I observed that only \/tmp\/<\/strong><\/em> is writable in AWS lambda. The other options were to have a database or to have a JSON file uploaded to the S3 bucket. Since this was a mini project that runs daily for small amounts of data I made a tradeoff but if you are working on a project that requires some file\/data updates using AWS Lambda then having a DB or using S3 buckets is the way forward.<\/p>\n
\n
What next?<\/h3>\n
We will be coming up with a series of blog posts on how AWS lambda helped us to solve a couple of our internal manual tasks and save our time. Hope you liked the blogpost !! If you any questions please comment below <\/p>\n","protected":false},"excerpt":{"rendered":"
Qxf2 services provides testing services for startups, a majority of our client base is offshore so as to keep them informed about the days Qxf2 is going to be on PTO (public and optional holidays) we follow a practice of notifying them 3 to 7 days in advance. This helps in proper planning and estimating the work. Every year, Qxf2 […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[282,135,18],"tags":[],"class_list":["post-15036","post","type-post","status-publish","format-standard","hentry","category-aws-lambda","category-data-scraping","category-python"],"_links":{"self":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/15036","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/comments?post=15036"}],"version-history":[{"count":21,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/15036\/revisions"}],"predecessor-version":[{"id":15583,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/15036\/revisions\/15583"}],"wp:attachment":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/media?parent=15036"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/categories?post=15036"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/tags?post=15036"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}