By Carl Valle
Since 2010, sleep tracking has doubled in interest with coaches, as nearly any professional knows that getting good shut-eye can help athletes. I have used subjective logging for 20 years, starting with hours and quality sleep before progressing to actual wearable devices. My experience has taught me a few hard lessons about data, behavior, and, of course, the details of the technology.
This article covers all you need to know about data capture with sleep tracking, starting with the brands and options first, and then addressing what to do with the information in the second part. Countless articles rehash the same cycle of science on how sleep is valuable for everyone, not just athletes, but rarely do they talk about the necessary hardware and software. What coaches will find here is a breath of fresh air, meaning an honest look at the reasons sleep technology isn’t just about cool apps or what teams are “monitoring” sleep, but how accurate and useful the devices are.
Research Grade Data: What the Lab Uses
The strangest part of the validation process with devices is the lack of encouragement for coaches or medical professionals to actually do a sleep study with their wearable. While not cheap, a team can get up to three or four devices validated at the same time by having someone volunteer one night of data and see if the product comes close to reproducing the research grade data with the device of choice. Several companies have done research internally, but very few have independent peer-reviewed research to show they are in the ballpark of full Polysomnography (PSG). The use of a full set of metrics is the gold standard of sleep measurement, and in the past I made the mistake of thinking it was brainwaves only.
Sleep is always estimated, but with multiple measures it’s easier to evaluate with more confidence when someone is sleeping deeply or just “resting their eyes.” PSG incorporates up to eight different indices, but the most common measures are heart (ECG), brain (EEG), muscle (EMG), and eye movements (EOG). Recently, breathing is captured and estimated for the record, using pulse oximetry and other indirect measures. Sleep is a state of consciousness so it’s very brain-centric, but the body and higher centers of the mind are not always aligned perfectly.
Sleep data from a single measure is just one metric, even if the system has millions of dollars of product development and world-class scientists. As you will see later, one measure can be gamed and invalid if you’re not careful. Most products on the market try to combine cheap but valid accelerometers to get actigraphy, a measurement of human activity with a wearable sensor. Some products are using Heart Rate Variability (HRV) to get sleep architecture from a shirt or sensor in the bed.
While all of the ideas are great ways to get sleep, even the best medical-grade products have limitations when applied in the real world. Even a sleep lab has a colossal problem with sampling sleep, since the environment for the data is not what is likely being experienced by athletes. For example, someone’s bed and the cumbersome wires are a major factor in recreating what “normal is.”
In the city of Newton, right outside Boston, there is a Marriott hotel that is home to a sleep diagnostic center that evaluates sleep disorders. While on paper a hotel is a perfect fit, recreating a sleep problem is nearly impossible, because a snoring spouse not present during the stay is an obvious flaw to addressing one potential problem. I have seen the actual data of PSG reports and, believe me, they are not very “app friendly” and would be a nightmare to visualize on a coach dashboard. While the validity of sleep evaluation for true diagnostics is far from perfect, sleep lab data is a medical solution to serious disorders that can ruin lives and impair recovery for athletes. Sleep is not just about helping athletes get better; it’s a major factor in all human health. Even today, culture and misinformation continue to diminish sleep quantity and quality.Sleep is not just about helping athletes get better; it’s a major factor in all human health. Click To Tweet
How Good Is Good Enough With Sleep Data?
Coaches want to measure sleep to monitor compliance of rest and recovery, not pretend they are sleep doctors. Unfortunately, sleep data that coaches or team professionals use will always have just a fraction of the accuracy of lab data. However, this is actually alright if the data gives enough information to assist in making smarter decisions.
The accuracy and precision of data is still murky, and I have experienced poor results with products caused by technology failing. Batteries not keeping charge, Bluetooth connectivity issues, sensors not contacting the body properly, and lazy software developers forgetting to beta test upgrades are all-too-common issues. All it takes is one algorithm sabotaged by a few lines of code and something near-magical becomes a very expensive random number generator. It’s great for lotteries like Powerball, but a major letdown to teams wanting to solve real problems. A single evening or even an eight-week pilot study is not tested enough to be in the trenches; coaches need bulletproof technology that is dependable as a guard dog, always watching and trusted like family.
The data integrity needed to make progress with sleep monitoring can start with something simple, such as a sleep log. But, as many coaches know, any time you ask for subjective information you should get ready for a rollercoaster ride of problems. Even with the limitations of self-reported data like sleep quality and sleep quantity, I still include it along with passive measures like sleep-tracking wearable devices. First, an athlete’s recall compared to more objective data allows me to calibrate their scale of reality. Second, athlete communication is vital to a healthy relationship.
It’s a bad idea to just implant sensors and skip the dialogue, because the causes of bad sleep are likely more important than the record of poor sleep architecture. To me, metadata is the cream of information and has a better signal-to-noise ratio than pure data without context. Some products capture a myriad of “environmental” factors that attempt to provide better analysis to explain the interaction of variables that cause poor sleep data. Other products try to manage sleep by assisting the wake period with ideal rise times with some success. But again, all of the efforts use estimated single-stream proxy data points.
Most of the products available can help move the needle with better sleep results if used creatively. The brutal disappointment is that even some of the companies themselves are often not clever about using their own devices with more effectiveness. In addition to the sleep device companies, other companies are not very creative about using their products to help paint a better and more confident picture of what is going on with sleep and activity patterns.
My favorite Wi-Fi scale is the Withings scale, and digital scales are undervalued in the simple need to get attendance while at home. An athlete sleeping and waking up in their own bed is a factor in getting better sleep and, while travel is sometimes unavoidable, deciding to sleep somewhere else is a lifestyle choice that may help or may harm the sleep experience. Many of the bed sensor products are great examples of why simple can work wonders, as being in bed and out of bed is a crude score of activity that can hint to possible sleep duration. An athlete in bed, provided it’s actually them and not part of their entourage, is rather binary but still a worthwhile start.
Adding more assessments of sleep can extract more information, but small hints to behavior patterns really add up over time. When coaches are looking at very precise measures of state of sleep, such as REM patterns, then the potential for error rises. The company Fatigue Science boasts a 93% accuracy rate, and to me that is enough to draw conclusions of restful sleep versus tossing and turning, if used right.
Classification of Sleep-Tracking Device Types
The best way to split up the companies is to place them into categories based on how they capture data. While not perfect, the benefit of using a taxonomy of sleep trackers is to illustrate the pros and cons in a fair way. Each company will share their opinions on why they went with a specific method of data capture: some have honest explanations and some are not to be trusted because they have investors they are trying to please.
While the commercial and enterprise markets to acquire sleep data only use two systems of the body, the way in which companies decide to position their product is an element I had to include. Briefly summarized, companies measure using the heart or body motion, and sometimes a combination of the two. Companies also decide to make it wearable on the wrist, the chest, or on the bed/pillow. Benefits and pitfalls exist with every option, and this is the reason I worry about anyone claiming they have the ideal product or solution. The better companies will be open and transparent about how data artifacts can emerge in the recording process, or the limitations of their calculations in some situations.
Wrist Accelerometers – You can pick up a gray market wrist wearable option from Alibaba for a dollar, spend $50 at Best Buy for a Misfit, or upgrade to Fatigue Science if you are a professional team or company. The most common wearable on the market is a wrist option, as engineers find this body zone the most useful for collecting data. Accelerometers are cheap, and most of the investment in the product is spent making it robust in both the design of the hardware and the development of a quality algorithm to filter and calculate the data it collects.
Wrist Heart Rate Variability – I thought the Apple Watch would get HRV from the wrist, or the Mio Watch would be able to get data as well, but the two companies elected to go with heart rate only. Whoop, a company here in Boston, elected to get HRV and other measures with the wrist years ago. I sat next to the founders at a Tech Stars meeting lead by Nike, and they showed me their prototype. I was impressed with what the Harvard guys were attempting to do.
Heart Rate Strap or Shirt Options – Restwise, Hexoskin, and other companies are collecting HRV data and then creating sleep scores with their calculations. The use of biometric shirts as athlete pajamas is clever, but not all athletes find shirts and straps comfortable. The Suunto material is silky smooth and very ergonomic, but many athletes still don’t like wearing anything, including a watch or something similar.
Bed or Pillow Options – A growing area with companies is the pillow or bed. Old iPhone apps cleverly used the vibration from the mattress to get estimations of sleep, but those were not valid enough to rely on. Sleepace and Beddit use ballistocardiography from a sensor strip to get HRV data from the torso. While they are popular in the consumer market, there are few teams experimenting with it. Sleepace and Hello use accelerometers for capturing data from pillows with an entry-point product, and both provide an added-value option to support the users.
Other systems exist or existed, such as the Zeo sleep system that used a consumer-grade EEG headband that was a major turnoff for my athletes, as it was very geeky-looking. The people behind the company were awesome, but the market will only support what is widely adopted and the company folded after a few years. I remember sitting down with Dave Tenney about five years ago, and it was clear on his face that he knew compliance was going to be near zero with athletes, but being the early adopter he tried it anyway.
Landon Evans invested in it and experienced the sizing problem I had when he woke up with a headache hours later. After a few months, the band conductive strap would eventually wear down, but the app was excellent and, again, the data very solid. Other innovative options exist now that use cameras and similar technologies, but you don’t have to be a rocket scientist to know that the market for spying on someone’s private quarters isn’t going to be popular, especially with the recent breaches by hackers in the news.
App, Team Software, and Data Export
Every device on the market wirelessly pairs with a smartphone or tablet, and provides the end user with an app to review sleep data. Some enterprise products designed for large groups provide a power-user- or management-style software product for aggregating many users, such as teams and companies, to one area. Other products allow for data to be pushed or shared in an automated fashion, and some need manual export. Whoop and Fatigue Science have dashboards and team software options, while the other products found in an electronics store are usually good enough to push the data to a third-party web product or even an AMS (athlete management system) provider.
Two perspectives matter with the sleep data: that of the athlete and of the support staff. While beyond the scope of this review article, athletes are the end decision makers on how seriously they commit to getting rest and sleep. Tools designed to capture data and push it up the food chain but forget the end user fail miserably in the long run. I spoke to one company that was having serious problems with compliance with all of its clients, save a few endurance athletes. I warned them years ago to learn from Zeo—it’s not the hardware, it’s the tribe. They forgot about culture and, the last time I checked with teams, the same vanilla user experience and branding is haunting them.
After two weeks, the new technology’s luster and the novelty of seeing your sleep data wears off, and most companies simply have very poor ability to spawn a movement in sleep adoption. Coaches want to see athlete’s sleep data to improve recovery, but most coaches simply give up on the process because it’s a struggle.
Coaches are infamous for working insane hours, and I have been guilty for years of putting in long weeks, but I still get my hours of sleep in. Support staff needs to buy into sleep first, and get their ducks aligned before promoting their sleep campaign to athletes. So much more can be written on adoption, behavior modification, routines, and motivation, but for now if you believe and do what you want your athletes to do, it’s a great start to convincing them to measure their sleep.
Sleep analysis software is not sleep diagnostics, meaning the purpose of the tools is to summarize patterns and visualize the data, not declare the medical problems if there are any. When data is severely compromised in any way, the natural and responsible choice is to bump up the problem to a medical professional. Most AMS products are now placing sleep data in a higher priority zone on the dashboard because sport science and the general population appreciate the value of sleep intellectually, but actions speak much louder than words. Just as obesity rates are still climbing even though the importance of diet and exercise are widely known, the average Joe or elite athlete is not getting better sleep than 10 years ago. Some athletes decide to sleep more and focus on better quality and patterns, but, for the most part, sleep with athletes, be it pro, student, or amateur, still needs a lot of work.
The Pros and Cons of Each Sleep-Tracking Option
Each choice of sleep tracking device and software has strengths and weaknesses, and a corresponding price tag to match. Most coaches would rather buy something cheap to get data, but don’t ever shop on price—shop on long-term value. I am all for coaches starting off small if budgets are thin with “over the counter” products, as I am a proponent of being creative with simple data before going hardcore with the better tools. The main problem is that plenty of coaches want to move forward to get more insight with the cheaper options, but that is the nature of the beast—you get what you pay for.When it comes to sleep technology, don’t ever shop on price—shop on long-term value. Click To Tweet
Teams that are traveling constantly, like basketball, baseball, and hockey, need a wearable tool because most bed products are just too cumbersome. The pillow products may be useful, but those are easily lost. If athletes are not buying the products, they don’t commit to being responsible for them. Some athletes feel like animals or prisoners at first when they know they have a device that is watching them, but the modern world has cameras watching them with the ubiquitous smartphone connected to every human being.
I have mentioned countless times that the only difference between an athlete thinking you are a guardian angel instead of a big brother is the actions you take after getting monitoring data. Coaches must reach out to athletes and explain that the purpose of sleep monitoring is not to remove fun or enjoyment from life, but to make sure those experiences are amplified with proper rest. Sleep is about paying the piper the right way at the right time, not cheating sleep or giving up a social life. While some sacrifices must be made, in reality the average athlete can have a balance if they manage their time better.
Some teams and organizations may want to use bed devices if they feel compliance will rise, but those systems only work if the athlete is there. As I have learned through painstaking trial and error, the activity pattern during the day usually dictates the pattern or sleep process later. My main interest is the wind-down three hours before sleep, as mental and physical stimuli can make or break sleep.
Shutting off isn’t easy for every athlete, and teams that compete at night know that the typical routine of athletes visiting a new city isn’t just to eat and get to bed, especially if they are young and single. It’s rough even at home, since athletes with kids or other responsibilities can make getting good sleep a wishful event. To me, sleep devices that capture data before sleep are hard to give up, but remember it’s about maxing out the data you get, not putting the cart before the horse and looking for the next big thing.
Many activity trackers are collecting data on movement, so an athlete passed out drunk for hours will falsely look like deep sleep to some. Some products are going to give false summary scores for athletes who might have unique circumstances that cause them to sleep longer. For instance, mononucleosis may cause an athlete to sleep longer, but that extension is a sign of illness, not rest. Again, it’s up to staff to decide what is best for them based on their needs, and not shop on features or testimonials.
Don’t Talk About Sleep, Measure It
It’s easy to fall into the trap of wanting athletes to get sleep and provide education formally or from face time. While the phrase, “what gets measured is what gets managed” is used all the time, it does happen to be true. But it’s incomplete. What is measured is valued, since most coaches that don’t measure something usually don’t prioritize the information. It’s fine and acceptable to fail repeatedly because of the lack of cooperation from groups of athletes, but not putting in an honest effort is just lazy. Change doesn’t happen overnight, but athletes are human and people will adopt what you believe in if you feel and convey it.
Many athletes will not ever wear a device because they feel they are being spied on, but for the most part, that is just an excuse because they value the fun of doing other things besides getting sleep. Truthfully, attempting to get athletes to sleep better is an enormous task, but the results are massive when they get deep restful hours. You don’t have to get everyone on board sleeping with measurement devices, but the first step is getting the few before getting the majority. Progress is what is needed, not perfection, so get going and do something with someone with sleep tracking.