Battle of the sleep apps! Sleepbot vs Jawbone UP vs Sleep as Android vs Sleep Cycle vs Fitbit

Disclaimer: This sleep tracking study was conducted by the Pillows.com staff and volunteers. This study was not conducted by a sleep research team or sleep scientists. The Pillows.com staff is passionate about discovering new methods and products to help people sleep better and improve their overall health and life.

Purpose:

At Pillows.com we are constantly researching and keeping up with the latest sleep trends in order to help people achieve the best sleep possible. As many of you know, there is a lot of buzz right now about sleep tracking devices and how they can monitor your sleep cycle to help you better understand the quality of your sleep. With so many devices out there, how are people supposed to know which one is the most accurate at tracking? Well, we took it upon ourselves to put on a fun experiment so that we can help answer that question!

The purpose of this study was to determine which popular sleep tracking device is the most accurate at monitoring movement while you sleep, which is a determinant in analyzing quality of sleep. When we conducted the study we also took a subjective approach and reviewed each app based on how user friendly the apps were as well as how insightful the data and reports were.

The Study

In order to determine which app was objectively most accurate at monitoring movement, we had a volunteer, Max M., act as the sleeper and a Pillows.com team member, Chelsea Duckham., Director of Marketing, act as an observer. The participant (Max) wore three different sleep tracking devices and slept next to two phones with sleep apps for 3 nights. All of the apps/devices were activated for the entirety of the study. In addition, he slept in front of a video camera that visually recorded his sleep movement. Every morning Chelsea observed his movements via the video recording and compared it to the sleep monitoring data from the tracking devices. The above graph compares the sleep tracking devices’ data to Chelsea’s observations of movement from the video. Chelsea also recorded her opinions and reviewed the apps’ usability and reports/data.

Devices

Sleep Tracking Devices

 

max sleep

Screen shot of observed video footage

Observer’s findings and review of sleeping tracking devices

How to use the sleep tracking graph: Compare the different sleep tracking devices to the observed movement that was noted in the video by clicking on the device name in the legend. Hover over the observed movement (dark grey dots on the graph) for more detail on the extent of movement. Each day is separated in different tabs.

Standards for data analysis:  Movement measurement findings, review of user interface, and review of detailed reporting/data. We classified movement calibration by minor (scratching nose, adjusting bedding), moderate (adjusting sleep position), and extensive (sleeper participant is awake).

Fitbit Flex: This tracking device is a wearable wrist band and has a sleep tracking element to it. The app claims to measure your sleep quality by tracking how many times you woke up and how long you slept. This data is supposed to help you learn how to sleep more soundly. Wristband also includes a silent sleep alarm to gently wake sleeper.

movement measurement: There was only a few instances that Fitbit did not pick up on movement in some way, and the movement it did not pick up was always very minor ( He re adjusted his position on the same side, scratched his nose, etc). There was even a time on June 5th where he stretched, which caused moderate movement and Fitbit was the only device that tracked this movement. The movement was seen as “restless” or “awake” in the app and it even did a good job at recording how long he was restless or awake for.

user interface: The user interface was very unique. It was the only app that summarized the total time that he was restless or moving in the night. The app is interactive where you can touch the “restless” points in the chart to see the time frame of restless or awake. However, sometimes it was a bit finicky and I could not pull up the exact time.

fitbit2

Fit Bit – graph with touch capabilities

detailed reporting: I like how the app described his movement as “restless” since it’s a very accurate description of his movement. Sometimes awake and restless were interchangeable from my video observations. The app doesn’t include anything regarding sleep cycles or phases, which I think is helpful in understanding your sleep quality.

Fitbit 1

Jawbone Up

The Jawbone UP is also a wearable wristband that tracks daily activity along with sleep. The app tracks hours slept, light sleep vs. deep sleep, awake time, and overall sleep quality. Up wristband also has a smart alarm that wakes the sleeper at an optimal time.

movement measurement: UP categorized sleep movement as light sleep and wake. When I compared Up to the video, I noticed that moderate to extensive movements were marked as transitions from deep sleep to light sleep. All movement from slight to extensive was recorded as light sleep. There were a few instances of moderate movement (e.g. adjusted blanket, sits halfway up, etc) where UP recorded his movement as sound asleep.

Up

user interface: The user interface includes a graph that is also interactive. You can touch one of the sleep phases recorded in the graph (wake, light or sound) and it will give you detail on time. I really liked how the app also includes how long it took the sleeper to fall asleep.

detailed reporting: The only thing that I didn’t really like about categorizing the movement as light sleep, deep sleep, or wake is that it seemed a little too general and not detailed enough to my liking. The app didn’t take into account the range of his movement and how this could indicate his overall sleep quality.

**Winner for user interface

Pebble Watch With Sleep By Android

This specific sleep tracker was a combination device and phone app. The Pebble Watch is a customizable watch that syncs with apps and has the ability to track movement. The Pebble Watch can track a sleeper’s movement and then syncs with the app, Sleep By Android, where the data is analyzed.

movement measurement: Movement was tracked in both a time linear graph and bar graph that tracked sleep phases. The Pebble Watch With Sleep By Android (PW) was extremely accurate and showed at least some calibration of movement for every time the participant moved.

Screenshot_2014-06-05-21-27-58 (1)

user interface:The user interface was slightly confusing and it was not interactive. However, this means that the two graphs were the main focus, which I think is necessary since they provide you valuable information. You can also compare different nights of the linear movement graph.

detailed reporting: THIS is the kind of detailed reporting that I’ve been waiting for. I love how the app tracks everything that is important in sleep tracking and displays it in two simple graphs. This app is the only one that included the REM (rapid eye movement) which is essential when determining sleep quality. The reports also allowed you to compare graphs over multiple nights so that you can get a better understanding of your overall sleep quality over a period of time.

** Winner for movement measurement (most accurate)

Sleep Cycle – IOS

Sleep Cycle is an app that acts as a bio alarm clock that tracks your movement and then determines which sleep phase you are in. The alarm will wake the sleeper up during their lightest sleep phase. The sleeper sleeps with their phone tucked between their sheet and mattress.

movement measurement: Sleep Cycle tracked movement via a linear graph. The app was accurate and documented the sleeper’s movement and how this correlated with falling in and out of the sleep stages. During the study, there was only one major movement where Sleep Cycle did not indicate anything that summarized this movement.

user interface: The user interface is very intuitive and simple. After three nights, you have the ability to view more detailed reporting by turning the phone to the side. There is no interactive element. The smart alarm is a nice touch since it gently wakes up the sleeper in a given time frame (30 minutes, etc).

detailed reporting: I really liked Sleep Cycle’s reporting and especially how the app classifies movement as awake, sleep, and deep sleep. The graph makes it easy to show how long you spent in deep sleep and the peaks of movement that brought the sleeper into different stages. The app gives you more insight on your overall quality of sleep the more you use it. I liked this detail reporting the best since it was clean, simple, and gave you all the necessary info.

After 3 nights, the app includes more detailed reports on sleep quality (percentage), sleep quality per day of the week, time in bed per day of the week, bedtime, and activity (steps), effect on sleep quality.

** Winner for detailed reporting

Sleep Cycle

Sleep Cycle Reporting – Night 1

Sleep Bot – Android

Sleep Bot is also a phone app that tracks sleep movement and uses that information to trigger the smart alarm to wake the sleeper.

movement measurement: Sleep Bot was not as accurate as the other apps used in our study. The first night, the app was the least accurate and barely recorded any movement. The 2nd and 3rd night saw some improvement where the app did record extensive movement and some minor movement.

Sleepbot

Sleep Bot – Night 1

Screenshot_2014-06-06-06-47-51 (1)

Sleep Bot – Night 3

user interface:The interface was simple and only really displayed the graph of movement, how long the sleeper was in bed, and sleep time. There was no interactive element.

Reporting/Data: I liked how Sleep Bot clearly reported movement and the graph included low, medium, and high movement. The app was also the only linear graph that included detailed time ranges in the graph. The app did not mention any sleep phases, which could help users understand the correlation between movement and quality of sleep.

Study Conclusion

Although three days is a brief time period in which to conduct a sleep study, it was long enough to give us a look into the world of sleep devices and sleep technology. Variables such as age, health, exercise habits, eating habits, etc should all be taken into account when evaluating your sleep needs. Sleep devices are not guaranteed to automatically improve your sleep; however, they can help to give you a better understanding of your sleep patterns.  Evaluating this information and adjusting your daily habits can greatly improve your quality of sleep.

Furthermore, quality of sleep is just as important, if not more, than quantity, which is what these sleep devices aim to do. On a given day, keeping track of what you eat, how much you exercise, and how you feel can give you much insight into what’s most conducive for your sleep. No one knows you better than you, and there is no one-size-fits-all approach to sleep. Focus on maintaining a regular schedule and consistency, as the human body likes predictability. The amount of money you want to spend and the amount of data you desire will ultimately determine which device/app you choose to purchase. If you do choose to use these sleep apps, or any others, use them in a manner that helps you to be more attuned to your body and overall health.

This entry was posted in Live Well, Sleep Tips. Bookmark the permalink.

15 Responses to Battle of the sleep apps! Sleepbot vs Jawbone UP vs Sleep as Android vs Sleep Cycle vs Fitbit

  1. Cathy says:

    Great blog post!

  2. Nichole G says:

    Would love to win these pillows. I havent bought new pillows in over 10 years..

  3. Dana Rodriguez says:

    Nice post! Very interesting!

  4. susan king says:

    I’ve never used any of these but some I would love to try ! Thank you for giving us the chance to win these pillows !! They sound wonderful !!

  5. Rose Tente says:

    Love the Blog Post. These Pillows seem amazing, would Love to own one.

  6. Marichelle Claveria says:

    Great blog. Want to try your pillow and see it for myself

  7. Kendra W. says:

    Oh boy, I need a sleep study done on me! I haven’t slept well in years and I absolutely know a big part of it is my pillow! I’m still on the search for my perfect pillow, but the process is so darn expensive!

  8. Kyle Ross says:

    Loved the sleep study information!

  9. nancy fiorenza says:

    wow thank you for the chance

  10. Carmela Capano says:

    Love the blog post!

  11. rob fiorenza says:

    Great prize I need this

  12. Margaret Maggie Porter says:

    I had no idea all these gadgets existed. I am getting a fitbit to track daily exercise. Bonus that it tracks sleep!

  13. Jackie Vigliucci says:

    Nice post. I have never used any of these things but I have considered the FitBit Flex as a friend recommended to me. I also would consider the Jawbone Up.

  14. sean fiorenza says:

    sounds great

  15. Thank you! That has been a lot of my questions answered on these sleep things! I just wondered how in the world would a bracelet know when you were sleeping! Super great article , thank you. This has answered a lot of my questions! Now on to the pillows, do they help? Lol

Leave a Reply

Your email address will not be published. Required fields are marked *