We have noticed that sometimes we want to see test results in our email inbox even though good reporting tools are being used. We at Qxf2 Services, use pytest as our test runner. A simple utility email_pytest_reports.py has been added to our framework which can send pytest results as simple HTML reports. This blog is on how we built an email utility to send pytest results to any email address.
Steps to build and run the utility
Step 1: Created a email_conf to get the email details
You can add the email credentials, sender and target details in email_conf.py
Step 2: Install pytest-html plugin:
pip install pytest-html |
Step 3: Added the required functions to send email in email_pytest_report util file
We need to write functions to create the html report, make it as an attachment and finally send the email with the attachment.
a. Creating html report is accomplished by get_test_report_data function
def get_test_report_data(self,html_body_flag= True,report_file_path= 'default'): "get test report data from pytest_report.html or pytest_report.txt or from user provided file" if html_body_flag == True and report_file_path == 'default': #To generate pytest_report.html file use following command e.g. py.test --html = log/pytest_report.html test_report_file = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','log','pytest_report.html'))#Change report file name & address here elif html_body_flag == False and report_file_path == 'default': #To generate pytest_report.log file add ">pytest_report.log" at end of py.test command e.g. py.test -k example_form -r F -v > log/pytest_report.log test_report_file = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','log','pytest_report.log'))#Change report file name & address here else: test_report_file = report_file_path #check file exist or not if not os.path.exists(test_report_file): raise Exception("File '%s' does not exist. Please provide valid file"%test_report_file) with open(test_report_file, "r") as in_file: testdata = "" for line in in_file: testdata = testdata + '\n' + line return testdata |
b. Get and attach the report file
def get_attachment(self,attachment_file_path = 'default'): "Get attachment and attach it to mail" if attachment_file_path == 'default': #To generate pytest_report.html file use following command e.g. py.test --html = log/pytest_report.html attachment_report_file = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','log','pytest_report.html'))#Change report file name & address here else: attachment_report_file = attachment_file_path #check file exist or not if not os.path.exists(attachment_report_file): raise Exception("File '%s' does not exist. Please provide valid file"%attachment_report_file) # Guess encoding type ctype, encoding = mimetypes.guess_type(attachment_report_file) if ctype is None or encoding is not None: ctype = 'application/octet-stream' # Use a binary type as guess couldn't made maintype, subtype = ctype.split('/', 1) if maintype == 'text': fp = open(attachment_report_file) attachment = MIMEText(fp.read(), subtype) fp.close() elif maintype == 'image': fp = open(attachment_report_file, 'rb') attachment = MIMEImage(fp.read(), subtype) fp.close() elif maintype == 'audio': fp = open(attachment_report_file, 'rb') attachment = MIMEAudio(fp.read(), subtype) fp.close() else: fp = open(attachment_report_file, 'rb') attachment = MIMEBase(maintype, subtype) attachment.set_payload(fp.read()) fp.close() # Encode the payload using Base64 encoders.encode_base64(attachment) # Set the filename parameter attachment.add_header('Content-Disposition', 'attachment', filename=os.path.basename(attachment_report_file)) return attachment |
c. Finally write the send email function like below
To run the utility using python command and to see the full file, please check the file in our framework
def send_test_report_email(self,html_body_flag = True,attachment_flag = False,report_file_path = 'default'): "send test report email" #1. Get html formatted email body data from report_file_path file (log/pytest_report.html) and do not add it as an attachment if html_body_flag == True and attachment_flag == False: testdata = self.get_test_report_data(html_body_flag,report_file_path) #get html formatted test report data from log/pytest_report.html message = MIMEText(testdata,"html") # Add html formatted test data to email #2. Get text formatted email body data from report_file_path file (log/pytest_report.log) and do not add it as an attachment elif html_body_flag == False and attachment_flag == False: testdata = self.get_test_report_data(html_body_flag,report_file_path) #get html test report data from log/pytest_report.log message = MIMEText(testdata) # Add text formatted test data to email #3. Add html formatted email body message along with an attachment file elif html_body_flag == True and attachment_flag == True: message = MIMEMultipart() #add html formatted body message to email html_body = MIMEText('''Hello,Please check the attachment to see test built report.<strong>Note: For best UI experience, download the attachment and open using Chrome browser.</strong>''',"html") # Add/Update email body message here as per your requirement message.attach(html_body) #add attachment to email attachment = self.get_attachment(report_file_path) message.attach(attachment) #4. Add text formatted email body message along with an attachment file else: message = MIMEMultipart() #add test formatted body message to email plain_text_body = MIMEText('''Hello,\n\tPlease check attachment to see test built report. \n\nNote: For best UI experience, download the attachment and open using Chrome browser.''')# Add/Update email body message here as per your requirement message.attach(plain_text_body) #add attachment to email attachment = self.get_attachment(report_file_path) message.attach(attachment) message['From'] = self.sender message['To'] = ', '.join(self.targets) message['Subject'] = 'Script generated test report' # Update email subject here #Send Email server = smtplib.SMTP_SSL(self.smtp_ssl_host, self.smtp_ssl_port) server.login(self.username, self.password) server.sendmail(self.sender, self.targets, message.as_string()) server.quit() |
d. Conftest.py changes
In pytest, there is a provision to run scripts/code after the execution of all tests. pytest_sessionfinish plugin hook executes after the whole test run finishes. So we used pytest_sessionfinish plugin hook auto-email the report every time you run the test. To achieve this we added a method pytest_terminal_summary in conftest. This method will be run only after tests are exited which means after the execution of self.driver.quit(), this method will be called.
#Contents of conftest.py from utils.email_pytest_report import Email_Pytest_Report #Test arguments @pytest.fixture def email_pytest_report(request): "pytest fixture for device flag" return request.config.getoption("--email_pytest_report") #Command line options: parser.addoption("--email_pytest_report", dest="email_pytest_report", help="Email pytest report: Y or N", default="N") def pytest_terminal_summary(terminalreporter, exitstatus): "add additional section in terminal summary reporting." if not hasattr(terminalreporter.config, 'workerinput'): if terminalreporter.config.getoption("--email_pytest_report").lower() == 'y': #Initialize the Email_Pytest_Report object email_obj = Email_Pytest_Report() # Send html formatted email body message with pytest report as an attachment email_obj.send_test_report_email(html_body_flag=True,attachment_flag=True,report_file_path= 'default') |
Step 4: Running the utility
You can run the test using below command
pytest --email_pytest_report Y --html=log/pytest_report.html --self-contained-html --capture=sys |
After running the command you will see console output as shown below and an email with an HTML report attached.
I hope this blog has helped you get started with emailing your pytest reports.
I started my career as a Dotnet Developer. I had the opportunity of writing automation scripts for unit testing in Testcomplete using C#. In a couple of years, I was given the opportunity to set up QA function for our in-house products and that’s when I became full-time Tester. I have an overall 12 years of experience. I have experience in Functional – Integration and Regression Testing, Database, Automation testing and exposure to Application Security and API testing. I have excellent experience in test processes, methodologies and delivery for products in Information security and Pharma domain. On the Personal side, I love playing with kids, watch Football, read novels by Indian Authors and cooking.
Hey
I beleive there is no point to send an email if HTML file generated the after theexcution and take place and given location in command line.
Please do update me if this is post havingh different understanding.
Hey Akhil,
To send generated HTML file via email refer our util here You can directly run util just you need to modify import statement to point email config file.
In case, if you want auto email report after every run, you need to add method “pytest_terminal_summary” under conftest.py file and inside that method, you need call method to the sent email. Look at here, how we added method “pytest_terminal_summary” under contest here Akhil, ignore the flags we are checking here, look at Line 258 and 260, how we calling send email method. You can also add the flags the way we added for more control.
Note: Method “pytest_terminal_summary” executed only after tests get existed, I mean after the execution of self.driver.quit()
Hi , my generated html report has attachments of failure screenshot embedded in it. But with this solution on sharing over email it is not fetching screenshots. Can you help me with that?
Hi,
Can you please share what error you have hit and how you are attaching the failed screenshots to the email message.
So I’ve been able to get this to work and generate an email response. Yet, when I run my tests with pytest-xdist, I’m sent multiple emails instead of one, I’ve been trying to figure out where the problem arises, but I can’t seem to figure out why, even though I’ve sent this through a debugger and still only have the process running once.
Hey, I’ve been able to get this process working, but when I run my tests with pytest-xdist, I run into a problem where the report is emailed multiple times. the results only display once when I see this being emailed. Is there some method to only send this off once with running in xdist?
Thanks, Logan and Eric for reporting this issue. We will try to get a fix for this issue soon and update you
Have you found a fix for this issue. Facing the same.
Hi Sherlyn, we implemented the following fix:
To avoid multiple emails being sent, we used the master-worker concept where the master is the last process to finish and will be sending the email.
The fix is in conftest.py in the pytest_terminal_summary hook. The following line needs to be added:
if not hasattr(terminalreporter.config, ‘workerinput’):
which checks if the process is a worker or master based on the attribute ‘workerinput’. This attribute is present only for workers.
Step 3.d:
if not hasattr(terminalreporter.config, ‘workerinput’):
if terminalreporter.config.getoption(“–email_pytest_report”).lower() == ‘y’:
email_obj = Email_Pytest_Report()
We will update the snippet in the blog shortly.
Hi, Is there any solution found for sending reports only once after the multiple browser execution.
Hi Dhanshree, we implemented the following fix:
To avoid multiple emails being sent, we used the master-worker concept where the master is the last process to finish and will be sending the email.
The fix is in conftest.py in the pytest_terminal_summary hook. The following line needs to be added:
if not hasattr(terminalreporter.config, ‘workerinput’):
which checks if the process is a worker or master based on the attribute ‘workerinput’. This attribute is present only for workers.
Step 3.d:
if not hasattr(terminalreporter.config, ‘workerinput’):
if terminalreporter.config.getoption(“–email_pytest_report”).lower() == ‘y’:
email_obj = Email_Pytest_Report()
We will update the snippet in the blog shortly.
In the content of conftest.py. Is there a def pytest_addoption(parser): missing before parser_addoption
Yes the code snippet doesn’t show that. You can find the complete code of our framework here
I could able to send and get the mail through gmail account.
Can you let me know the email_conf details to use office email(ex: outlook) to send and receive emails.
I need to run my testcases in VM so i need to get the email triggers from VM.
Thanks,
Swathi
Hi Swathi,
Good to know that you are able to send and get mail through your Gmail account.
Can you please refer to the following link for outlook configuration
https://www.techhit.com/how-to/configure-outlook-to-use-IMAP-with-outlook.com-account/
Thanks,
Rohan
Thanks for a very quick reply Rohan!
before trying this out, can you please clarify me one more thing, does this outlook setup works for the code which runs from VM Server?
bcz, VM server does not contain the outlook setup with these configurations.
please clarify.
Thanks!
Hi Swathi,
It really does not matter where you are running the code. As long as you provide the correct email configuration details to code, the code will work fine.
Thanks,
Rohan
HTML report in the email attachment is not performing any actions on the result table.
Script content is not getting copied to the attached file and also results table header is not performing any actions like Sorting etc.,
any help on this?
Hi Swathi,
Can you please provide more details about the issue?
And can you please check report looks good locally or not?