{"id":4852,"date":"2016-12-08T07:18:39","date_gmt":"2016-12-08T12:18:39","guid":{"rendered":"https:\/\/qxf2.com\/blog\/?p=4852"},"modified":"2018-04-03T10:26:30","modified_gmt":"2018-04-03T14:26:30","slug":"testing-the-fitbit-heart-rate-monitoring-system-part-2","status":"publish","type":"post","link":"https:\/\/qxf2.com\/blog\/testing-the-fitbit-heart-rate-monitoring-system-part-2\/","title":{"rendered":"Testing the Fitbit heart rate monitoring system: Part 2"},"content":{"rendered":"

In this post, we<\/a> outline a cheap and creative way to perform repeatable and controlled tests on the Fitbit<\/a> heart rate monitor. Our test has a reasonable degree of accuracy. We hope it helps the Fitbit developers rapidly test their prototypes. To be clear: thorough, accurate and precise testing is needed in the later stages of development. The test outlined here is not suitable for later stages of development.<\/p>\n

This is the second part in a 2-part series of posts where we outline a quick and dirty tests for testing Fitbit\u2019s heart rate monitor. We strongly suggest you begin by reading the first part<\/a> of this post. In the\u00a0first part<\/a>, we wrote about the working principle of optical heart rate sensors, how we exercised Fitbit heart rate reading with different colored paper, plastic bottles and how we went about testing the Fitbit heart rate detector. In this part, we will cover how we mimicked the heart rate flow, dive into the details of our setup, share the Arduino code to control our stepper motor, provide readings and measurements we took, identify limitations with our approach and finally talk about how we could improve our system.<\/p>\n

\n

Setup<\/h3>\n

We want to be able to mimic the heart rate in a controlled manner. To mimic blood flow rate for Fitbit heart rate monitor testing, we decided to use two different reflected index materials which together mimic a pulse. We used two different colored craft paper to generate heart pulse and placed them on a wheel. We controlled the pulse rate by using a stepper motor to control the speed of the wheel. We developed this heart pulse wheel with different colored paper for 2 or 3 heart pulse per wheel as shown in Fig. 1<\/p>\n

<\/a>
Fig. 1 Wheel with strips of paper to mimic a pulse<\/figcaption><\/figure>\n
\n

Validating the idea<\/h3>\n

To test Fitbit heart rate monitor system, we need to rotate heart pulse wheel at different speeds to mimic blood flow. We initially choose 12V DC motor for rotating wheel and speed of DC motor controlled with a\u00a0potentiometer. We observed that at different speeds of the motor, Fitbit shows different readings as shown in following Fig. 2. Fitbit readings are directly proportional to a\u00a0speed of wheel i.e. speed of moving craft paper strips.<\/p>\n

<\/a>
Fig. 2 Fitbit heart rate monitor system testing using DC motor and Heart pulse wheel<\/figcaption><\/figure>\n

This step was helpful to validate the idea that our concept will work and we can exercise the heart rate monitor over a range of heart rates in a controlled manner.<\/p>\n

Note:<\/strong> We\u00a0observed that the Fitbit readings depended on the distance between heart rate sensor and target. It took us a while, but we fixed the distance between sensor and target by developing a nicely fitting box frame like in Fig 4.<\/p>\n

\n

Controlling the speed of the heart pulse wheel<\/h3>\n

To test the Fitbit heart rate monitoring system against a range of heart rates, we need to control the speed of heart pulse wheel accurately. So we decided to use a stepper motor to control the speed of heart pulse wheel. We programmed an Arduino to control the speed of our stepper motor.<\/p>\n

We have used a 6-wire hybrid type stepper motor, an Arduino Uno and L293D motor driver. The stepper motor interfaced with the Arduino Uno using an L293D motor driver in a bipolar fashion. We chose high torque stepper motor as we needed to rotate a wheel of approximate 400 gms.\u00a0Look at following Fig. 3 for the block diagram.<\/span><\/p>\n

\"Block<\/a>
Fig. 3 Block Diagram of Fitbit Heart Rate Tester<\/figcaption><\/figure>\n

After interfacing the stepper motor with Arduino Uno, we\u00a0moved to test stepper motor speed range. From our speed test, we found that the stepper motor works well between 1 to 100 RPM. After 100 RPM it starts missing pulses. We also found that at low and high-speed motor makes heavy noise. So we decided to run stepper motor between 35 to 75 RPM and then used 2 and 3 pulse wheel model. That way we were able to accurately mimic a heart rate of up to 180. Look at following Fig. 4 for the hardware setup.<\/p>\n

<\/a>
Hardware Setup for Fitbit testing.<\/figcaption><\/figure>\n

We used Arduino’s inbuilt ‘Stepper.h’ library to control the\u00a0stepper motor speed. Below is a code snippet you can use if you are performing this test at home.<\/p>\n

\/*This is sample code for this blog post. \r\nThis code does not reflect Qxf2's coding habits*\/\r\n#include \r\n\/\/Define pin numbers\r\nint in1Pin = 12;\r\nint in2Pin = 11;\r\nint in3Pin = 10;\r\nint in4Pin = 9;\r\n\/\/ change this to the number of steps on your motor\r\n#define STEPS 200\r\nStepper motor(STEPS, in1Pin, in2Pin, in3Pin, in4Pin);\r\nvoid setup()\r\n{\r\nSerial.begin(9600); \/\/Enable serial monitor\/communication\r\nSerial.println(\"Set motor speed (in RPM) : \"); \/\/ print msg on serial monitor for user\r\nwhile(Serial.available()==0); { } \/\/ wait here until user enter input data\r\nint user_speed= Serial.parseInt(); \/\/ Read entered integer value and store it in a variable\r\nSerial.print(user_speed); \/\/ print user entered speed on serial monitor\r\nmotor.setSpeed(user_speed); \/\/ Set stepper motor speed to user defined speed\r\n}\r\nvoid loop()\r\n{\r\nmotor.step(STEPS); \/\/Run stepper motor\r\n}\r\n<\/pre>\n
\n
The experiment<\/h3>\n
Once we were done with the hardware setup, we started testing of heart rate monitor system and took readings. Here the expected reading depends on the speed of motor and heart pulse wheel.<\/p>\n
If 2 beat heart pulse wheel is used for testing, then the expected reading is as follows:<\/p>\n
Expected Reading = 2 * speed of motor<\/p>\n
If 3\u00a0beat heart pulse wheel is used for testing<\/span>, then the expected reading is as follows:<\/p>\n
Expected Reading = 3 * speed of motor<\/p>\n
Test Steps<\/h5>\n
    \n
      \n
    1. Check and note down, you are using 2 or 3 beat heart pulse wheel<\/li>\n
    2. Calculate the speed of stepper motor required based on heart pulse wheel and expected reading. Use above formula to find the\u00a0speed of stepper motor.<\/li>\n
    3. Set stepper motor speed from Arduino serial monitor as shown in the\u00a0following fig. 5.\n
      \"Arduino<\/a>
      Fig. 5 Arduino Serial Monitor<\/figcaption><\/figure><\/li>\n
    4. Once you are done with above step, the stepper motor starts to rotate at defined speed along with heart pulse wheel. Keep your Fitbit above heart pulse wheel and lock the Fitbit belts in frame notches to fix its position.<\/li>\n
    5. After some time (say about 30+ sec), Fitbit shows heart beat reading on its display. Note down that reading as the actual reading. Look at following videos for the\u00a0demo.<\/li>\n
    6. To stop the stepper motor or to change stepper motor speed, press ‘reset’ button on Arduino board.\u00a0Look at following video 1 for the\u00a0demo of testing.<\/li>\n<\/ol>\n<\/ol>\n
      \nhttps:\/\/qxf2.com\/blog\/wp-content\/uploads\/2016\/11\/expected_100_actual_98.mp4<\/a><\/video><\/div>\n
      Video. 1 Fitbit heart rate monitoring system testing demo<\/p>\n
      \n
      Disappointing results …<\/h3>\n
      We repeated the test 10 times for each expected readings and took notes of expected and actual reading as shown in Table 1. We collected and analyzed the data and plotted a candlestick chart as shown in Fig. 6. We were highly disappointed with the margin of error.<\/p>\n
      <\/a>
      Fig. 6 Fitbit readings when Fitbit kept parallel to heart pulse wheel<\/figcaption><\/figure>\n
      Formulae used for Calculations:<\/strong><\/p>\n
      1.Error:<\/p>\n
      Error = Actual Heart Rate – Expected Heart Rate<\/p>\n
      2.Error in %:<\/p>\n
      Error in % = (Error\/Expected heart rate) * 100<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n
      Table 1. Results when Fitbit is kept in parallel with heart pulse wheel<\/td>\n<\/tr>\n
      Expected heart rate<\/td>\nActual heart rate readings<\/td>\nError range<\/td>\nError range (%)<\/td>\n<\/tr>\n
      80<\/td>\n85, 77, 82, 75, 81, 86, 80, 79, 88, 76<\/td>\n-5 to +8<\/td>\n-6 to +10%<\/td>\n<\/tr>\n
      90<\/td>\n84, 90, 95, 96, 91, 88, 90, 86, 90, 93<\/td>\n-6 to +6<\/td>\n-7 to +7 %<\/td>\n<\/tr>\n
      100<\/td>\n95, 98, 97, 94, 105, 101, 100, 96, 107, 104<\/td>\n-6 to +5<\/td>\n-6 to +5 %<\/td>\n<\/tr>\n
      120<\/td>\n111, 118, 122, 125, 123, 117, 115, 120, 119<\/td>\n-9 to +5<\/td>\n-8 to +7 %<\/td>\n<\/tr>\n
      135<\/td>\n132, 138, 140, 135, 139, 141, 131, 130, 135<\/td>\n-5 to +6<\/td>\n-4 to +4 %<\/td>\n<\/tr>\n
      150<\/td>\n143, 148, 152, 150, 154, 155, 155, 158, 146<\/td>\n-7 to +8<\/td>\n-5 to +5 %<\/td>\n<\/tr>\n
      165<\/td>\n159, 158, 170, 168, 165, 164, 162, 158, 161<\/td>\n-7 to +5<\/td>\n-4 to +3 %<\/td>\n<\/tr>\n
      180<\/td>\n174, 170, 173, 178, 179, 172, 175, 173, 176<\/td>\n-10 to -1<\/td>\n-5 to -1 %<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
      You’ll notice that the margin of error is quite high. We seem to have gotten errors up to +\/- 8 beats per minute. That won’t do. Clearly, our heart pulse wheel is not exercising Fitbit in the same as our wrist exercises Fitbit.<\/p>\n
      Side note:<\/strong> Fitbit heart rate displays a ‘–‘ if you continue the test for beyond a minute or so.<\/p>\n
      D’oh! The breakthrough<\/h3>\n
      We were not happy with our results. So we started thinking to find other ways to test Fitbit heart rate monitor system. We thought about to use pipes and liquid to mimic blood flow. But that seemed pretty involved and we needed to study fluids and fluid dynamics. But then, suddenly, we realized we had made a mistake in the way Fitbit was oriented with respect to the heart pulse wheel. We had kept Fitbit in parallel to heart pulse wheel i.e. kept in parallel to blood flow. But blood in our wrists flows perpendicular to Fitbit. We corrected the\u00a0placement of Fitbit by keeping it perpendicular with heart pulse wheel.<\/p>\n
      And voila!! Look at the\u00a0following fig. 7 for actual reading and expected reading after placing Fitbit in perpendicular with heart pulse wheel.<\/p>\n
      \"Expected<\/a>
      Fig. 7 Expected & Actual readings after keeping Fitbit perpendicular with heart pulse wheel<\/figcaption><\/figure>\n
      After keeping Fitbit perpendicular with heart pulse wheel, we observed, it working in a\u00a0similar way like when we wear on our wrist. I exercised it for 8 to 10 mins and also varied motor speed i.e. heart rate. Fitbit kept up with the variations. We repeated 6 test steps mentioned above for approximately 10 times for each expected reading and took notes of expected and actual reading as shown in Table 2. We analyzed collected data and plot a candlestick chart as shown in fig. 8.<\/p>\n
      \"Fitbit<\/a>
      Fig. 8 Readings when Fitbit kept perpendicular with heart pulse wheel<\/figcaption><\/figure>\n\n\n\n\n\n\n\n\n\n\n\n\n
      Table 2. Results when Fitbit is kept in perpendicular with heart pulse wheel<\/td>\n<\/tr>\n
      Expected heart rate<\/td>\nActual heart rate readings<\/td>\nError range<\/td>\nError range (%)<\/td>\n<\/tr>\n
      80<\/td>\n85, 85, 84, 84, 84, 84, 84, 84, 83, 83<\/td>\n+3 to +5<\/td>\n+4 to +6 %<\/td>\n<\/tr>\n
      100<\/td>\n99, 99, 97, 99, 101, 102, 99, 100, 99, 98<\/td>\n-3 to +2<\/td>\n-3 to +2 %<\/td>\n<\/tr>\n
      120<\/td>\n118, 121, 120, 121, 120, 118, 119, 119, 118, 121<\/td>\n-2 to +1<\/td>\n-2 to +1 %<\/td>\n<\/tr>\n
      130<\/td>\n132, 134, 133, 130, 133, 134, 129, 135, 132, 131<\/td>\n-1 to +5<\/td>\n-1 to +4 %<\/td>\n<\/tr>\n
      140<\/td>\n142, 137, 143, 144, 137, 142, 143, 145, 143, 143<\/td>\n-3 to +5<\/td>\n-2 to +3 %<\/td>\n<\/tr>\n
      150<\/td>\n146, 145, 146, 149, 150, 151, 153, 148, 149, 147<\/td>\n-5 to +3<\/td>\n-3 to +2 %<\/td>\n<\/tr>\n
      165<\/td>\n172, 173, 166, 175, 172, 169, 173<\/td>\n+1 to +10<\/td>\n-1 to +6 %<\/td>\n<\/tr>\n
      180<\/td>\n174, 179, 175, 181, 173, 172, 175, 174<\/td>\n-8 to +1<\/td>\n-4 to +1 %<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
      \n
      Suggested improvements<\/h3>\n
      To solve the problem of getting high error tolerance and to minimize error in % to the\u00a0even lower value say +\/- 2, we think using a tracked wheel arrangement will help. Look at\u00a0Tracked wheel\/ tank wheel arrangement shown here<\/a>. It\u00a0requires single stepper motors to rotate both wheels. On the\u00a0belt, we need to stick different colored craft paper to mimic heart pulses. Connect stepper motor to one of tank wheel and place Fitbit near to another wheel. This tank arrangement keeps Fitbit away from stepper motor noise and may we get more accurate result.<\/p>\n
      \n
      There you have it! We used colored paper, a stepper motor and an Arduino to test the Fitbit heart rate monitor. If you found it interesting, drop me a note below in the comments.<\/p>\n
      If you are a startup finding it hard to hire technical QA engineers, learn more about Qxf2 Services<\/a>.<\/strong><\/p>\n
      \n","protected":false},"excerpt":{"rendered":"
      In this post, we outline a cheap and creative way to perform repeatable and controlled tests on the Fitbit heart rate monitor. Our test has a reasonable degree of accuracy. We hope it helps the Fitbit developers rapidly test their prototypes. To be clear: thorough, accurate and precise testing is needed in the later stages of development. The test outlined […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43,117,82],"tags":[119,118],"class_list":["post-4852","post","type-post","status-publish","format-standard","hentry","category-api-testing","category-hardware-testing","category-thoughts-on-testing","tag-arduino","tag-fitbit"],"_links":{"self":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/4852","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/comments?post=4852"}],"version-history":[{"count":88,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/4852\/revisions"}],"predecessor-version":[{"id":5188,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/posts\/4852\/revisions\/5188"}],"wp:attachment":[{"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/media?parent=4852"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/categories?post=4852"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qxf2.com\/blog\/wp-json\/wp\/v2\/tags?post=4852"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}