Why this post
XRAY is a Test management plugin for Jira. This plugin helps you automate your complete testing process. Also, it has easy integration with BDD which is something we have implemented at our client.
There are server and cloud versions of XRAY. This article talks about simple methods which we implemented to get XRAY Server integration done with behave BDD.
Few terms you will see used in the post:
Test ExecutionID – The XRAY ID of type Test Execution. There can be many test executions based on your test cycles.
Test PlanID – This will map to the test plan in Jira. Test execution are mapped to test plans.
TestcaseID – These are actual test cases in Jira. TestcaseID is mapped to Test ExecutionID and is part of many test executions.
Current flow
The client generates a TestPlanID for the new cycle, and based on the environments, a TestExecutionID is created under this TestplanID. When the tests are run with the ExecutionID(s) the TestcaseID is automatically aligned with them.
One thing to remember, XRAY always aligns tests to execution. So if you don’t want to create TestExecutionID initially and want teams to create their own, teams feel it’s an overhead to go to Jira and create TestExectionID and then use that while running test. Phew!!!
XRAY, on the other hand, allows you to pass TestPlanID while running the test, and it will generate the TestExecutionID and align your tests to it. That’s cool right?
In the example, I am passing TestPlanID. And call methods to create TestExectionID and link them, finally upload results to Jira.
When TestPlanID is passed, first check if the planID is valid using XRAY API. If it’s valid then call API to create ExecutionID and link them.
Verify if the TestPlanID and TestExecutionID are valid
def find_if_valid_plan_id(test_plan_id): """ Function calls the xray api for checking test plan id. Returns the response status code to callee function. :param test_plan_id: xray test plan id :returns statuscode_plan : api response status code """ print "running find_if_valid_planid with plan id with plan id:", test_plan_id try: url = "{}rest/raven/1.0/api/testplan/{}/test?limit=10".format(JIRA_BASE_URL, test_plan_id) headers = header() response = requests.request("GET", url, headers=headers, data=None) statuscode_plan = response.status_code print "status code returned for plan id and url used are", statuscode_plan, url return statuscode_plan except Exception as e: print "Unable to execute find_if_valid_plan_id", e raise def find_if_valid_exec_id(xray_execution_id): """ Function calls the xray api for checking test execution id. Returns the response status code to callee function. :param xray_execution_id : xray test execution id :returns statuscode_exec : api response status code """ print "running find_if_valid_exec_id with execution id : ", xray_execution_id try: url = "{}rest/raven/1.0/api/testexec/{}/test?limit=10".format(JIRA_BASE_URL, xray_execution_id) headers = header() response = requests.request("GET", url, headers=headers, data=None) statuscode_exec = response.status_code print "status code returned for exececution id and url used are", statuscode_exec, url return statuscode_exec except Exception as e: print "Unable to execute find_if_valid_exec_id", e raise |
Create TestExecutionID
When TestplanID is passed, we need to call the API to create a new TestExecutionID and then call another API to link them.
def create_xray_test_execution_id_and_link_with_test_plan(test_plan_id): """ This function using xray api call creates new test execution id with the details mentioned in the Payload parameter. Newly created execution id is then passed to another xray api call which links it with the passed plan id. The execution id is returned by this function. :param test_pan_id : xray test plan id :returns xray_test_execution_id : newly created execution id :returns execution_creation_resp_status_code : response status for execution id created """ print "creating new execution id" try: xray_test_execution_id ="" url = "{}rest/api/2/issue/".format(JIRA_BASE_URL) payload = { "fields": { "project":{ "id": "xxxx" #pass the actual id }, "summary": "Creating a Test Execution issue", "description": "Creating of a test execution issue using IDs for projects and issue types using the REST API", "issuetype": { "id": "xxx" #pass the actual id } } } payloadJson = json.dumps(payload) headers = header() response_execution_creation = requests.request("POST", url, headers=headers, data=payloadJson) execution_creation_resp_status_code = response_execution_creation.status_code if execution_creation_resp_status_code != 404: result = json.loads(response_execution_creation.text) if execution_creation_resp_status_code != 400: xray_test_execution_id = result['key'] print(result['key']) #Link Test Plan to Test Execution ID print "Linking newly created execution id to plan id" link_execution_id(xray_test_execution_id, test_plan_id) return xray_test_execution_id except Exception as e: print "Unable to execute create_xray_test_execution_id_and_link_with_test_plan", e raise |
Link the newly created TestExecutionID to the passed TestPlanID
def link_execution_id(xray_test_execution_id, test_plan_id): """ This function will link the newly created execution id with the plan id :param xray_test_execution_id : newly created execution id :param test_plan_id : test plan id :returns response_plan_linking_status : returns the status code of linking execution with plan """ try: print "Linking newly created execution id to plan id" url = "{}rest/raven/1.0/api/testplan/{}/testexecution".format(JIRA_BASE_URL, test_plan_id) print(url) payload_plan = { "add": [ xray_test_execution_id ] } payloadJson_plan = json.dumps(payload_plan) response_plan_linking = requests.request("POST", url, headers=headers, data=payloadJson_plan) response_plan_linking_status = response_plan_linking.status_code print "This is response for test plan link to newly created test execution id", response_plan_linking if response_plan_linking.status_code != 200: print "plan id to execution id linking has failed" xray_test_execution_id = None return response_plan_linking_status except Exception as e: print "Unable to execute link_execution_id", e raise |
Now, you can upload the results to XRAY
def update_xray_results(xray_execution_id, feature_files, config_json_files): """ This is the main function for xray result update which calls various functions in sequence and updates xray results to jira. :param xray_execution_id : xray test execution id :returns : None """ try: print "inside update_xray_results" final_json_files = "c:/behave_json_files" update_xray_execution_results_using_behave_api(final_json_files) except Exception as e: print "Unable to execute update_xray_results", e raise def update_xray_execution_results_using_behave_api(final_json_files): """ This function makes the XRAY api call to upload the final results from the json files. :param final_json_files : folder where final json files are stored after cleaning and updating :returns : None """ try: url = "{}rest/raven/1.0/import/execution/behave".format(JIRA_BASE_URL) headers = header() response_status = None for file in final_json_files: if file.endswith(".json"): print "Found json for api call. JSON file name is ", file with open(XRAY_JSON_DIR_FINAL + file, 'r+') as json_file: json_data = json.load(json_file) data = json.dumps(json_data) response = requests.request("POST", url, headers=headers, data=data) response_status = response.status_code if response.status_code == 500: print "Something has gone wrong, check the json file for more details", file else: continue return response_status except Exception as e: print "behave api call failing", e raise |
All the code put together
import requests import json JIRA_AUTH = ('authid authpassword') JIRA_BASE_URL = "http://abc/" XRAY_JSON_DIR_FINAL = "<mention the path where final json files are created>" def header(): """ This function holds all the headers required for xray behave api. The Jira Authorization details are passed here. This function is called in methods which make jira xray api calls. :returns headers : headers required for xray api call """ headers = {'Accept' : 'application/json', 'Content-Type': 'application/json', 'Authorization': JIRA_AUTH} return headers def check_and_update_xray_results(xray_execution_id, test_plan_id, feature_files, config_json_files): """ This function checks if from CLI test plan id or execution id is passed. :param xray_execution_id : xray test execution id :param test_plan_id : xray test plan id :returns : None """ try: if test_plan_id is not None: print "Test Plan ID is passed" check_update_plan_id(test_plan_id, feature_files, config_json_files) if xray_execution_id is not None: check_update_execution_id(xray_execution_id, feature_files, config_json_files) except Exception as e: print "Unable to execute check_and_update_xray_results", e raise def check_update_plan_id(test_plan_id, feature_files, config_json_files): """ This functions calls method to verify passed plan id is valid or not. If passed plan id is valid then function to create and link test execution id is called. With newly created execution id the xray test results upload is proceeded. :param test_plan_id : xray test plan id :returns : None """ try: statuscode_plan = find_if_valid_plan_id(test_plan_id) xray_test_execution_id = None if statuscode_plan == 200: print "going to create execution id and link it to plan id" execution_creation_resp_status_code, xray_test_execution_id = create_xray_test_execution_id_and_link_with_test_plan(test_plan_id) print "new execution id is ", xray_test_execution_id if xray_test_execution_id is not None: xray_execution_id = xray_test_execution_id update_xray_results(xray_execution_id, feature_files, config_json_files) else: print "status code returned for plan id being valid", statuscode_plan except Exception as e: print "Unable to execute check_update_plan_id", e raise def check_update_execution_id(xray_execution_id, feature_files, config_json_files): """ This function will be called when execution id is passed directly from CLI or commander. First function will check if the passed id is valid. If excution id is valid then xray test result upload function is invoked. :param xray_execution_id : xray test execution id :returns : None """ try: statuscode_exec = find_if_valid_exec_id(xray_execution_id) if statuscode_exec == 200: print "Execution ID is passed directly" update_xray_results(xray_execution_id, feature_files, config_json_files) else: print "status code returned for execution id being valid", statuscode_exec except Exception as e: print "Unable to execute check_update_execution_id", e raise def find_if_valid_plan_id(test_plan_id): """ Function calls the xray api for checking test plan id. Returns the response status code to callee function. :param test_plan_id: xray test plan id :returns statuscode_plan : api response status code """ print "running find_if_valid_planid with plan id with plan id:", test_plan_id try: url = "{}rest/raven/1.0/api/testplan/{}/test?limit=10".format(JIRA_BASE_URL, test_plan_id) headers = header() response = requests.request("GET", url, headers=headers, data=None) statuscode_plan = response.status_code print "status code returned for plan id and url used are", statuscode_plan, url return statuscode_plan except Exception as e: print "Unable to execute find_if_valid_plan_id", e raise def find_if_valid_exec_id(xray_execution_id): """ Function calls the xray api for checking test execution id. Returns the response status code to callee function. :param xray_execution_id : xray test execution id :returns statuscode_exec : api response status code """ print "running find_if_valid_exec_id with execution id : ", xray_execution_id try: url = "{}rest/raven/1.0/api/testexec/{}/test?limit=10".format(JIRA_BASE_URL, xray_execution_id) headers = header() response = requests.request("GET", url, headers=headers, data=None) statuscode_exec = response.status_code print "status code returned for exececution id and url used are", statuscode_exec, url return statuscode_exec except Exception as e: print "Unable to execute find_if_valid_exec_id", e raise def create_xray_test_execution_id_and_link_with_test_plan(test_plan_id): """ This function using xray api call creates new test execution id with the details mentioned in the Payload parameter. Newly created execution id is then passed to another xray api call which links it with the passed plan id. The execution id is returned by this function. :param test_pan_id : xray test plan id :returns xray_test_execution_id : newly created execution id :returns execution_creation_resp_status_code : response status for execution id created """ print "creating new execution id" try: xray_test_execution_id ="" url = "{}rest/api/2/issue/".format(JIRA_BASE_URL) payload = { "fields": { "project":{ "id": "xxxx" #pass the actual id }, "summary": "Creating a Test Execution issue", "description": "Creating of a test execution issue using IDs for projects and issue types using the REST API", "issuetype": { "id": "xxx" #pass the actual id } } } payloadJson = json.dumps(payload) headers = header() response_execution_creation = requests.request("POST", url, headers=headers, data=payloadJson) execution_creation_resp_status_code = response_execution_creation.status_code if execution_creation_resp_status_code != 404: result = json.loads(response_execution_creation.text) if execution_creation_resp_status_code != 400: xray_test_execution_id = result['key'] print(result['key']) #Link Test Plan to Test Execution ID print "Linking newly created execution id to plan id" link_execution_id(xray_test_execution_id, test_plan_id) return xray_test_execution_id except Exception as e: print "Unable to execute create_xray_test_execution_id_and_link_with_test_plan", e raise def link_execution_id(xray_test_execution_id, test_plan_id): """ This function will link the newly created execution id with the plan id :param xray_test_execution_id : newly created execution id :param test_plan_id : test plan id :returns response_plan_linking_status : returns the status code of linking execution with plan """ try: print "Linking newly created execution id to plan id" url = "{}rest/raven/1.0/api/testplan/{}/testexecution".format(JIRA_BASE_URL, test_plan_id) print(url) payload_plan = { "add": [ xray_test_execution_id ] } payloadJson_plan = json.dumps(payload_plan) response_plan_linking = requests.request("POST", url, headers=headers, data=payloadJson_plan) response_plan_linking_status = response_plan_linking.status_code print "This is response for test plan link to newly created test execution id", response_plan_linking if response_plan_linking.status_code != 200: print "plan id to execution id linking has failed" xray_test_execution_id = None return response_plan_linking_status except Exception as e: print "Unable to execute link_execution_id", e raise def upload_xray_test_report(xray_execution_id, test_plan_id, files, config_json_files): """ The function checks if either of the xray ids(execution id or plan id) are passed. If none of the ids are found then further execution is stopped. :param xray_execution_id : xray test execution id :test_plan_id : xray test plan id :returns : None """ print "Test Execution ID is ", xray_execution_id, "Test Plan ID is ", test_plan_id try: if xray_execution_id and test_plan_id: print "Pass only one of the IDs, cannot upload results" elif xray_execution_id or test_plan_id: print "Found XRAY id" check_and_update_xray_results(xray_execution_id, test_plan_id, feature_files, config_json_files) else: print "Test execution ID or Test plan ID not passed, not uploading results to XRAY" except Exception as e: print "Unable to update xray with test results", e raise def update_xray_results(xray_execution_id, feature_files, config_json_files): """ This is the main function for xray result update which calls various functions in sequence and updates xray results to jira. :param xray_execution_id : xray test execution id :returns : None """ try: print "inside update_xray_results" final_json_files = "c:/behave_json_files" update_xray_execution_results_using_behave_api(final_json_files) except Exception as e: print "Unable to execute update_xray_results", e raise def update_xray_execution_results_using_behave_api(final_json_files): """ This function makes the XRAY api call to upload the final results from the json files. :param final_json_files : folder where final json files are stored after cleaning and updating :returns : None """ try: url = "{}rest/raven/1.0/import/execution/behave".format(JIRA_BASE_URL) headers = header() response_status = None for file in final_json_files: if file.endswith(".json"): print "Found json for api call. JSON file name is ", file with open(XRAY_JSON_DIR_FINAL + file, 'r+') as json_file: json_data = json.load(json_file) data = json.dumps(json_data) response = requests.request("POST", url, headers=headers, data=data) response_status = response.status_code if response.status_code == 500: print "Something has gone wrong, check the json file for more details", file else: continue return response_status except Exception as e: print "behave api call failing", e raise if __name__ == '__main__': test_plan_id = "XRAY-123" feature_files = "some_feature.feature" config_json_files = "file_name.json" xray_execution_id = create_xray_test_execution_id_and_link_with_test_plan(test_plan_id) print xray_execution_id upload_xray_test_report(xray_execution_id, test_plan_id, feature_files, config_json_files) update_xray_results(xray_execution_id, feature_files, config_json_files) |
I hope this post gets you started with the XRAY integration quickly.
Note- code example is compatible with python2 only.
Reference – https://docs.getxray.app/display/XRAY/About+Xray
I started my career as a Dotnet Developer. I had the opportunity of writing automation scripts for unit testing in Testcomplete using C#. In a couple of years, I was given the opportunity to set up QA function for our in-house products and that’s when I became full-time Tester. I have an overall 12 years of experience. I have experience in Functional – Integration and Regression Testing, Database, Automation testing and exposure to Application Security and API testing. I have excellent experience in test processes, methodologies and delivery for products in Information security and Pharma domain. On the Personal side, I love playing with kids, watch Football, read novels by Indian Authors and cooking.
in the method, create_xray_test_execution_id_and_link_with_test_plan, i am a bit confuse. What correspond to the xxxx and xxx?
As mentioned in the create_xray_test_execution_id_and_link_with_test_plan method, xxxx and xxx corresponds to the project id and issuetype id respectively of your JIRA project.