{"id":10107,"date":"2019-01-31T06:03:15","date_gmt":"2019-01-31T11:03:15","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=10107"},"modified":"2019-01-31T06:03:15","modified_gmt":"2019-01-31T11:03:15","slug":"time-analysis-of-jira-statuses-using-python","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/time-analysis-of-jira-statuses-using-python\/","title":{"rendered":"Time analysis of Jira statuses using Python"},"content":{"rendered":"

We have developed a time view of statuses metrics\u00a0based on Jira data as part of our engineering benchmarks application
\nNote:<\/strong>\u00a0This post is written in continuation with the other blog\u00a0on the engineering metrics<\/a>.<\/p>\n

What is time view of statuses metrics?<\/h3>\n

It is the time view graph that shows how many tickets were present in a given set of statuses for each day of a time period. The graph serves as an excellent tool to help with your hiring estimates. It also helps you identify when new hires start hitting their stride.
\n <\/p>\n

Background<\/h3>\n

We wanted to develop this graph because we did not find an easy way to look at a time view of statuses of tickets on Jira tool. As QA, we sense our workload increase but it is hard to make a case for more QA engineers with just a count of tickets. The metric described in this post also helps us to understand workload on daily basis.
\n 
\nThis is an example of the time view of statuses graph. The x-axis shows dates and the y-axis shows the number of tickets. The three lines plotted correspond to the statuses “in progress”, “feature test” and “peer review”.
\n\"Jira<\/p>\n

The chart above shows us three statuses over the first seven months of this year at one of Qxf2’s clients. Around May, we realized that the QA load over the last couple of months had been increasing steadily. We requested and got a new QA engineer joining us at the end of May. You can see the immediate impact she had – the number of tickets in the status ‘feature test’ started to trend downwards. Now, we are not only in a position to show this sort of impact but we can also alert our engineering management to other potential problems like the ballooning number of tickets in peer review. <\/p>\n

 <\/p>\n

A time view of statuses metrics technical details<\/h4>\n

In a previous post, we showed you how to use Python to analyze Jira data. We will build upon that work. To follow along, you need to know we developed two important modules:
\na) ConnectJira which contains wrappers for many Jira API calls. The sample code is here<\/a>
\nb) JiraDatastore which saves individual ticket information as a pickled file. The sample code is here<\/a><\/p>\n

We have also developed a backend module called \u2018TimeViewStatuses.py\u2019. As you see in the above screenshot, inputs for this metrics are Jira URL, authentication details, start date, and end date
\n <\/p>\n

How did we logically break it down?<\/h4>\n

1. Get tickets that are present in the given timeframe<\/strong><\/p>\n

\r\njql_result = self.connect_jira_obj.execute_query(query=\"'project'='%s' AND createdDate <= '%s' ORDER BY updated DESC\" % (self.connect_jira_obj.project, end_date))\r\n<\/pre>\n
2. Collect all possible statuses for all tickets and their start and end dates <\/strong><\/p>\n
\r\n   def get_ticket_status_dates(self, ticket_key):\r\n        \"Get ticket all statuses and their start date and end date\"\r\n        # This method returns a list of lists [[status1,start1,end1][status2,start2,end2]]\r\n        status_dates,status_journey,error = [],[], None\r\n        ticket = self.get_individual_tickets(ticket_key)\r\n        ticket_changelog = self.datastore_obj.get_item_in_json_ticket(ticket['ticket_in_json'],'changelog')\r\n        #order the ticket change log histories in descending order \r\n        ticket_changelog_histories = ConnectJira.order_list_of_dicts(ticket_changelog['histories'],'id','desc')\r\n        ticket_fields = self.datastore_obj.get_item_in_json_ticket(ticket['ticket_in_json'],'fields') \r\n\r\n        try:\r\n            ticket_creation_date = arrow.get(ticket_fields['created'])\r\n            current_status = str(ticket_fields['status']['name']).lower()\r\n            for action in ticket_changelog_histories:\r\n                for item in action['items']:\r\n                    if item['field'] == 'status':\r\n                        status_journey.append(\r\n                            [item['toString'].lower(), arrow.get(action['created'])])\r\n                        status_journey.append(\r\n                            [item['fromString'].lower(), arrow.get(action['created'])])\r\n            if len(status_journey) > 0: #For tickets created but never worked on, status_journey can be empty\r\n                status_journey.append([status_journey[-1][0], ticket_creation_date])\r\n                status_journey.reverse()\r\n                # We'll end on the current status and add now() as the end date \r\n                status_journey.append([current_status, arrow.now()])\r\n                for i in range(0,len(status_journey)-1):\r\n                    if status_journey[i][1] != status_journey[i+1][1]:\r\n                        status_dates.append([status_journey[i][0],status_journey[i][1],status_journey[i+1][1]])\r\n        except Exception as error:\r\n            print error.message, ticket_key\r\n            error = error.message + \" \" + ticket_key + \" \" + str(len(status_journey))\r\n         \r\n        return {'status_dates':status_dates, 'error': error}\r\n<\/pre>\n
3. Loop through each date and see how many tickets were present per status <\/strong><\/p>\n
\r\n def count_tickets_per_date(self, startdate, enddate, status_start_end, statuses_list):\r\n        \"Return a count of tickets per day in each status for a date range\"\r\n\r\n        start, end = arrow.get(startdate), arrow.get(enddate)\r\n        date_series, status_series, status_done = [], {}, {}\r\n            \r\n        for status in statuses_list:\r\n            status_series[status] = [] \r\n        \r\n        #Iterate through all dates provided by the user and have a count\r\n        for date in arrow.Arrow.range('day',start,end):\r\n            date_series.append(date.format('DD-MM-YYYYYYY'))\r\n            for status in statuses_list:\r\n                status_series[status].append(0)\r\n            for ticket_status_journey in status_start_end:\r\n                for status in statuses_list:\r\n                    status_done[status] = False\r\n                for val in ticket_status_journey:\r\n                    #val is of the format [status_name, startdate, enddate, ticket_key]\r\n                    if val[1].floor('day') <= date <= val[2].ceil('day'):\r\n                        if status_series.has_key(val[0]) and not status_done[val[0]]:\r\n                            status_series[val[0]][-1] += 1\r\n                            status_done[val[0]] = True\r\n                            print date.format('DD-MM-YYYYYYY'), val[0], val[3]\r\n        time_view_statuses = {'dates':date_series,'status_series':status_series}\r\n\r\n        return time_view_statuses\r\n<\/pre>\n
4. Added GET api call to render the time view of statuses metrics front end template<\/strong><\/p>\n
 \r\nif request.method == 'GET':\r\n   return render_template('\/get-time-view-statuses.html', title='Time view of statuses')\r\n<\/pre>\n
5. Added POST api call to interact with the backend for the data processing and format the backend data as per the high chart heap map configuration requirement<\/strong><\/p>\n
\r\nif request.method == 'POST':\r\n        #required jira logic calls for the given form input\r\n        status_list = split_str_to_list(statuses, ',')\r\n        jira_obj = TimeViewStatuses(jira_url, username, password, project)\r\n        backend_result = jira_obj.get_time_view_statuses(status_list, start_date, end_date)\r\n        error = backend_result['error']\r\n        data = []\r\n\r\n        if error is None:\r\n            for k,v in backend_result['time_view_statuses']['status_series'].iteritems():\r\n                data.append({'name':k,'data':v})\r\n            api_response = {'dates': backend_result['time_view_statuses']['dates'], 'data': data,'error': error}\r\n        else:\r\n            api_response = {'dates': [], 'data': [],'error': error}\r\n\r\n        return jsonify(api_response)\r\n<\/pre>\n
6. We have used High charts JS library to generate column chart for the jira sprint\/bucket metrics<\/strong><\/p>\n
\r\n    src=\"https:\/\/code.highcharts.com\/highcharts.js\"\r\n    src=\"https:\/\/code.highcharts.com\/modules\/heatmap.js\"\r\n<\/pre>\n
\n
NOTE:<\/strong> While Qxf2 has the habit of open sourcing many of our R&D projects<\/a>, we will not be open sourcing this code in the near future. <\/p>\n
\n","protected":false},"excerpt":{"rendered":"
We have developed a time view of statuses metrics\u00a0based on Jira data as part of our engineering benchmarks application Note:\u00a0This post is written in continuation with the other blog\u00a0on the engineering metrics. What is time view of statuses metrics? It is the time view graph that shows how many tickets were present in a given set of statuses for each […]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[173,172,171,18],"tags":[],"class_list":["post-10107","post","type-post","status-publish","format-standard","hentry","category-engineering-metrics","category-flask","category-jira","category-python"],"_links":{"self":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10107","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/comments?post=10107"}],"version-history":[{"count":21,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10107\/revisions"}],"predecessor-version":[{"id":10437,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/10107\/revisions\/10437"}],"wp:attachment":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/media?parent=10107"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/categories?post=10107"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/tags?post=10107"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}