Disclaimer: This sleep tracking study was conducted by the Pillows.com staff and volunteers. This study was not conducted by a sleep research team or sleep scientists. The Pillows.com staff is passionate about discovering new methods and products to help people sleep better and improve their overall health and life.
At Pillows.com we are constantly researching and keeping up with the latest sleep trends in order to help people achieve the best sleep possible. As many of you know, there is a lot of buzz right now about sleep tracking devices and how they can monitor your sleep cycle to help you better understand the quality of your sleep. With so many devices out there, how are people supposed to know which one is the most accurate at tracking? Well, we took it upon ourselves to put on a fun experiment so that we can help answer that question!
The purpose of this study was to determine which popular sleep tracking device is the most accurate at monitoring movement while you sleep, which is a determinant in analyzing quality of sleep. When we conducted the study we also took a subjective approach and reviewed each app based on how user friendly the apps were as well as how insightful the data and reports were.
In order to determine which app was objectively most accurate at monitoring movement, we had a volunteer, Max M., act as the sleeper and a Pillows.com team member, Chelsea Duckham., Director of Marketing, act as an observer. The participant (Max) wore three different sleep tracking devices and slept next to two phones with sleep apps for 3 nights. All of the apps/devices were activated for the entirety of the study. In addition, he slept in front of a video camera that visually recorded his sleep movement. Every morning Chelsea observed his movements via the video recording and compared it to the sleep monitoring data from the tracking devices. The above graph compares the sleep tracking devices’ data to Chelsea’s observations of movement from the video. Chelsea also recorded her opinions and reviewed the apps’ usability and reports/data.
Observer’s findings and review of sleeping tracking devices
How to use the sleep tracking graph: Compare the different sleep tracking devices to the observed movement that was noted in the video by clicking on the device name in the legend. Hover over the observed movement (dark grey dots on the graph) for more detail on the extent of movement. Each day is separated in different tabs.
Standards for data analysis: Movement measurement findings, review of user interface, and review of detailed reporting/data. We classified movement calibration by minor (scratching nose, adjusting bedding), moderate (adjusting sleep position), and extensive (sleeper participant is awake).
Fitbit Flex: This tracking device is a wearable wrist band and has a sleep tracking element to it. The app claims to measure your sleep quality by tracking how many times you woke up and how long you slept. This data is supposed to help you learn how to sleep more soundly. Wristband also includes a silent sleep alarm to gently wake sleeper.
movement measurement: There was only a few instances that Fitbit did not pick up on movement in some way, and the movement it did not pick up was always very minor ( He re adjusted his position on the same side, scratched his nose, etc). There was even a time on June 5th where he stretched, which caused moderate movement and Fitbit was the only device that tracked this movement. The movement was seen as “restless” or “awake” in the app and it even did a good job at recording how long he was restless or awake for.
user interface: The user interface was very unique. It was the only app that summarized the total time that he was restless or moving in the night. The app is interactive where you can touch the “restless” points in the chart to see the time frame of restless or awake. However, sometimes it was a bit finicky and I could not pull up the exact time.
detailed reporting: I like how the app described his movement as “restless” since it’s a very accurate description of his movement. Sometimes awake and restless were interchangeable from my video observations. The app doesn’t include anything regarding sleep cycles or phases, which I think is helpful in understanding your sleep quality.
The Jawbone UP is also a wearable wristband that tracks daily activity along with sleep. The app tracks hours slept, light sleep vs. deep sleep, awake time, and overall sleep quality. Up wristband also has a smart alarm that wakes the sleeper at an optimal time.
movement measurement: UP categorized sleep movement as light sleep and wake. When I compared Up to the video, I noticed that moderate to extensive movements were marked as transitions from deep sleep to light sleep. All movement from slight to extensive was recorded as light sleep. There were a few instances of moderate movement (e.g. adjusted blanket, sits halfway up, etc) where UP recorded his movement as sound asleep.
user interface: The user interface includes a graph that is also interactive. You can touch one of the sleep phases recorded in the graph (wake, light or sound) and it will give you detail on time. I really liked how the app also includes how long it took the sleeper to fall asleep.
detailed reporting: The only thing that I didn’t really like about categorizing the movement as light sleep, deep sleep, or wake is that it seemed a little too general and not detailed enough to my liking. The app didn’t take into account the range of his movement and how this could indicate his overall sleep quality.
**Winner for user interface
Pebble Watch With Sleep By Android
This specific sleep tracker was a combination device and phone app. The Pebble Watch is a customizable watch that syncs with apps and has the ability to track movement. The Pebble Watch can track a sleeper’s movement and then syncs with the app, Sleep By Android, where the data is analyzed.
movement measurement: Movement was tracked in both a time linear graph and bar graph that tracked sleep phases. The Pebble Watch With Sleep By Android (PW) was extremely accurate and showed at least some calibration of movement for every time the participant moved.
user interface:The user interface was slightly confusing and it was not interactive. However, this means that the two graphs were the main focus, which I think is necessary since they provide you valuable information. You can also compare different nights of the linear movement graph.
detailed reporting: THIS is the kind of detailed reporting that I’ve been waiting for. I love how the app tracks everything that is important in sleep tracking and displays it in two simple graphs. This app is the only one that included the REM (rapid eye movement) which is essential when determining sleep quality. The reports also allowed you to compare graphs over multiple nights so that you can get a better understanding of your overall sleep quality over a period of time.
** Winner for movement measurement (most accurate)
Sleep Cycle – IOS
Sleep Cycle is an app that acts as a bio alarm clock that tracks your movement and then determines which sleep phase you are in. The alarm will wake the sleeper up during their lightest sleep phase. The sleeper sleeps with their phone tucked between their sheet and mattress.
movement measurement: Sleep Cycle tracked movement via a linear graph. The app was accurate and documented the sleeper’s movement and how this correlated with falling in and out of the sleep stages. During the study, there was only one major movement where Sleep Cycle did not indicate anything that summarized this movement.
user interface: The user interface is very intuitive and simple. After three nights, you have the ability to view more detailed reporting by turning the phone to the side. There is no interactive element. The smart alarm is a nice touch since it gently wakes up the sleeper in a given time frame (30 minutes, etc).
detailed reporting: I really liked Sleep Cycle’s reporting and especially how the app classifies movement as awake, sleep, and deep sleep. The graph makes it easy to show how long you spent in deep sleep and the peaks of movement that brought the sleeper into different stages. The app gives you more insight on your overall quality of sleep the more you use it. I liked this detail reporting the best since it was clean, simple, and gave you all the necessary info.
After 3 nights, the app includes more detailed reports on sleep quality (percentage), sleep quality per day of the week, time in bed per day of the week, bedtime, and activity (steps), effect on sleep quality.
** Winner for detailed reporting
Sleep Bot – Android
Sleep Bot is also a phone app that tracks sleep movement and uses that information to trigger the smart alarm to wake the sleeper.
movement measurement: Sleep Bot was not as accurate as the other apps used in our study. The first night, the app was the least accurate and barely recorded any movement. The 2nd and 3rd night saw some improvement where the app did record extensive movement and some minor movement.
user interface:The interface was simple and only really displayed the graph of movement, how long the sleeper was in bed, and sleep time. There was no interactive element.
Reporting/Data: I liked how Sleep Bot clearly reported movement and the graph included low, medium, and high movement. The app was also the only linear graph that included detailed time ranges in the graph. The app did not mention any sleep phases, which could help users understand the correlation between movement and quality of sleep.
Although three days is a brief time period in which to conduct a sleep study, it was long enough to give us a look into the world of sleep devices and sleep technology. Variables such as age, health, exercise habits, eating habits, etc should all be taken into account when evaluating your sleep needs. Sleep devices are not guaranteed to automatically improve your sleep; however, they can help to give you a better understanding of your sleep patterns. Evaluating this information and adjusting your daily habits can greatly improve your quality of sleep.
Furthermore, quality of sleep is just as important, if not more, than quantity, which is what these sleep devices aim to do. On a given day, keeping track of what you eat, how much you exercise, and how you feel can give you much insight into what’s most conducive for your sleep. No one knows you better than you, and there is no one-size-fits-all approach to sleep. Focus on maintaining a regular schedule and consistency, as the human body likes predictability. The amount of money you want to spend and the amount of data you desire will ultimately determine which device/app you choose to purchase. If you do choose to use these sleep apps, or any others, use them in a manner that helps you to be more attuned to your body and overall health.