Fitness Trackers Out of Step When Measuring Calories, Research Shows

Fitness devices can help monitor heart rate but are unreliable at keeping tabs on calories burned, research has revealed. From a report on The Guardian: Scientists put seven consumer devices through their paces, comparing their data with gold-standard laboratory measurements. "We were pleasantly surprised at how well the heart rate did -- under many circumstances for most of the devices, they actually did really quite well," said Euan Ashley, professor of cardiovascular medicine at Stanford University and co-author of the research. "At the same time we were unpleasantly surprised at how poor the calorie estimates were for the devices -- they were really all over the map." The team tested seven wrist-worn wearable devices -- the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and Samsung Gear S2 -- with 31 women and 29 men each wearing multiple devices at a time while using treadmills to walk or run, cycling on exercise bikes or simply sitting.

Not a particularly unique problem.

[jedidiah]

This is probably true of any fitness device that claims to track calories. The new shiny shiny is no exception.

Re:

[MightyMartian]

The best you can do outside a lab where you can monitor blood sugar levels and the like, is compare activity to various tables of average calories burned. I have no faith in any wearable device getting very close to calories burned.

Re: Not a particularly unique problem.

[Anonymous Coward]

There are wearable gas analyzes but you will look like wearing a gas mask. But those are medical and accurate

Re:

[AvitarX]

Do they capture anaerobic burn?

Re:

[Hognoxious]

If it releases carbon dioxide (I think it does) then yes.

This was 30 years ago, but IIRC by comparing 02 consumption to C02 output you can determine which type of respiration is occurring.

Re: Not a particularly unique problem.

[AvitarX]

That makes sense. It does release carbon, just less per unit of energy.

Also, without the oxidation, the work doesn't directly relate to the calories burnt, as it's a lot less efficient without the oxidation.

Re:Not a particularly unique problem.

[CastrTroy]

Apart from a power meter on a bicycle, you are probably right. It really hard to get a good estimate of calories burned without actually being able to measure how much work the person is doing. The heart rate is only very loosely correlated with how much work somebody is doing. Heart rate can be changed by a number of factors, many of which have little to do with how many calories you are burning. Power meters on bikes are a whole other story, because they can actually measure how much physical work you are doing, and will be able to measure calories burned within some reasonable level of accuracy. Even, then, you probably need to develop a base line, as each person will expend a varying amount of energy to produce the same output.

Re:

[toonces33]

Power meters for bicycles aren't cheap either - maybe 1000$ or so. If you are a pro I could see getting one, but that's way out of the range of what average people are going to spend.

Re:

[pointybits]

The price is falling these days, you can get a 4iii for $400, or a Powerpod (kind-of-a-power-meter) for $260.

Re:

[AlexSasha]

Single leg PMs can go for as low as $300 if you are willing to buy used.

Re:Not a particularly unique problem.

[necro81]

Power meters for bicycles aren't cheap either - maybe 1000$ or so.

For some models, yes, but by and large your information is dated. There are plenty of "consumer" models that are less than $500, and provide data that is quite good.

For more information than you can shake a bicycle crank at, I suggest taking a look at DC Rainmaker [dcrainmaker.com]. He owns every device he reviews* and does lots of hands-on testing. For reviews of power meters, he typically tricks out his bike with 2-4 power meters (in different locations: crank arm, spider, rear hub, etc.), each with their own head unit.

* As he explains on his website, he often is doing his hands-on reviewing with pre-production models from the company, but always returns is and purchases his own - to confirm that is wasn't a special snowflake - before posting.

Re:

[lgw]

Especially: any exercise equipment that claims to show calories burned will vastly "overestimate" the result (flat out lie). This is pure sales material: the makers want you using their exercise equipment.

Re:

[toonces33]

Yeah, funny how they always seem to overestimate. About the only thing they are good for is relative comparisons from one day to the next. But they ought to remove the word calories and put some other word there - how about "donuts".

Re:

[Mr D from 63]

I've always assumed they were useless measurements, except maybe for comparing your own individual workouts on that particular equipment.

Re:Not a particularly unique problem.

[nine-times]

I also feel like it should be fairly obvious, if you take the time to think about it. What can these devices measure? Heart rate and motion, generally. Some will ask you what kind of exercise you're engaged in, but it rarely gets more specific than "running" or "cycling". There might be a GPS, in which case, the device can know roughly how much distance you cover.

So take each individually:

* Heart rate: Different things can effect your heart rate to different degrees, and there's not a direct/exact correlation between hear rate and calories burned.
* Motion: One of these things just knows how much your wrist is moving, which often won't tell you much. Sit still and shake your arm a little, and then run around for an equal amount of time. The watch can't tell the difference.
* Kind of exercise: You can't know how many calories you've burned simply from the information, "You were riding a bike for 10 minutes." Were you going up-hill or down-hill? How heavy is your bike? How well maintained is your bike? What gear were you in? How bumpy is the road?
* GPS: It knows you went from point "A" to point "B", which gives a distance and theoretically an altitude and speed. Were you in a car or on a bike? Were you carrying anything? Were you doing jumping jacks while your travelled? When you went up 10 meters in altitude, did you climb the stairs or ride and escalator?

So on their own, none of these things would get you to an accurate reading. Admittedly, you could try to combine these measurements to make the reading more accurate. For example, if the device knows you're riding a bike, it can use the GPS to determine how fast you're going, whether you're going up-hill or down. It can measure your wrist movements and heart rate to guess about how hard you might be pedaling. However, doing this kind of calculation would probably take some machine learning to figure out how to combine these things for each different kind of exercise, and even then it would probably change for different people in different circumstances, requiring some kind of calibration. And even then, it might be fooled or confused somehow.

So I think, reasonably, all we can expect with the current technology is a vague estimate. However, that's not useless. If I use my fitbit every day, and yesterday it says I burned 200 calories, and then today it says I burned 2,000 calories, that is likely a good indication that I was far more active today than yesterday. Did I actually burn precisely 2,000 calories today? Probably not. Did I actually burn 10 times the number of calories as yesterday? That might be a decent estimate (I think that'd be worth studying). Did I do a significant amount of additional exercise today, probably improving my health? I think that's a safe bet, and mostly that's what people really need to know. After all, the numbers of calories listed on food packaging is also an estimate. If you're trying to line up your calories burned to calories eaten, you're fooling yourself.

Re:

[Carnildo]

Averaged over significant periods of time, heart rate is proportional to breath rate, breath rate is proportional to oxygen consumption, and oxygen consumption is proportional to calories burned. So in theory, starting from heart rate alone, you can measure calorie burn.

The problem is that the constants of proportionality vary from person to person. Further, since the number you're measuring (heart rate) is three steps away from the number you want (calorie burn), errors tend to magnify each other. In pr

Re:

[dbIII]

Indeed, it's kind of obvious but quantifying the difference was a different story.

Re:

[jellomizer]

The problem is that Calorie burn is very complex. Age, Height, Weight, Body Fat Percentage, How much mussel, their current diet, body heat....
While most fat shamers will just say Calories in vs Calories out. Measuring those calories is very complex, the body will react differently towards starvation, and differently from exhaustion. For some people those excess calories in will just get pooped out, while for others it will be stored, and for others people will just get a huge burst of energy. These devi

Big boned

[110010001000]

Big boned people like me burn calories differently. So that explains it.

Re:

[AmiMoJo]

Well, yeah, actually... Once you start trying to diet seriously, your body goes into starvation mode. It conserves as much energy as possible, makes you feel tired so you do less etc. You go from needing 2500 calories a day to maintain to 1500, so your huge effort of will to drastically cut down your intake has no effect.

You can overcome it by forcing your body to burn calories with exercise, but your body will fight you all the way and you will feel like absolute shit for a long time. And even then, if you

Run 5 miles, about 800 calories

[Anonymous Coward]

That's all I need to know.

Re:

[anjrober]

that is not accurate.
you do NOT burn 800 calories in 5 miles.
the average is 100 calories per mile.
800 is way high.
and thats includes BMR.

Not all listed/detailed but...

[RJFerret]

From the blurb, "...errors on energy expenditure... ...ranging from the lowest at 27.4% for the FitBit Surge to the highest error of 92.6% for the PulseOn device."

Re:

[Anonymous Coward]

So you are saying the FitBit Surge is the most accurate caloric expenditure measurement device on the market! It's AMAZING measurements blow the competition away!

'Tracking calories' is basically impossible

[Rick Schumann]

There are too many variables involved in determining calories burned by any biological entity, and these 'fitness bands' are not the only device that has this problem, either. The closest you can come are devices that measure power generated by your muscles (PowerTap hubs, SRM or Quarq cranksets and other bicycle-mounted direct measuring instrumentation like them, rowing machines with an ergometer, etc), and even then there is a variable 'biological efficiency' term in the equation that means you can only call it an estimate. Some of the worst accuracy devices are things like the treadmills and stationary bikes in a public gym, which just use statistical averages of a range for calories burned given a level of exertion, and even then they tend towards the high end of the range to keep people motivated to continue using the treadmill or stationary bike. Something like these fitness bands work in a similar way, and I'd fully expect that they too tend to estimate on the high end of the range of 'calories burned' to keep you motivated. The fact that they track heart rate means it's a little more accurate, given one more term in the equation, and if they have a way to enter your bodyfat percentage, that would improve the estimation also. There are other factors you can plug into such an equation to make it more accurate, but in the end it's still just an estimate because of the efficiency factor. Therefore: none of what is being claimed here about it's lack of accuracy in 'calories burned' is terribly surprising. Of course for the average person, moving their body for significant amounts of time (not just getting off the couch, going to the 'fridge for another soda/beer, then back to the couch) is good regardless of trying to track 'calories burned', and to be quite honest, your dietary intake is more important when trying to lose excess bodyfat than exercise is. Of course you don't need any 'fitness bands' for that, just a decent pair of running shoes, so that doesn't make the company any money now does it?

Re:

[AlexSasha]

As a runner of many years, I find it that covering 1 mile by foot costs most people about 100 kcals. So run 5 miles - you need 500 kcals give or take a bit.

Re:

[Anonymous Coward]

I'm curious how you figure being a runner tells you anything about the calorie expenditure of other people running. Are you doing some kind of experiments? I'd love to be able to get a quality estimate for running myself.

Re:

[Rick Schumann]

See, now, applying what I said above to what you just said: "Most people" == 'statistical average' to one extent or another. Are you talking about other avid runners? Or are you talking about average people who just jog occasionally? The degree to which their bodies use calories efficiently is going to vary depending on how well conditioned they are.

That being said, the tables that you find everywhere online that let you figure out about how many calories you're burning for running at 'X' pace for 'Y' ti

Re:

[AlexSasha]

Yes. Higher aerobic point factor -> higher VO2Max -> less calories required. But generally 100 per mile is a decent estimate.

Re:

[Rick Schumann]

Emphasis on the word estimate . I don't know how many people I've talked to over the years who try to play the numbers game with their intake and calories burned, only to get frustrated that they're not losing any weight, because they thought it was absolute.

Re: 'Tracking calories' is basically impossible

[Anonymous Coward]

You are wrong. Calories can be accurately measured by a gas analyzer device. They are wearable, but about the size of a gas mask with a pouch filter. Cost about $4k too. They are just miniaturized version of the stand equipment athlete testing used in the past.

Re:

[Rick Schumann]

I've had performance testing in a lab using that sort of technology. You have to have a mask covering your entire nose and mouth, so no air leaks in or out. It's bad enough when you're an athlete putting up with that for 10 or 20 minutes, the average overweight person who has to force themselves to exercise at all is not going to put up with that for even ONE minute, let alone spend $4000 for the privilege of doing so every time they go to the gym or go out to run. Also that's still just an estimate albeit

Re:

[sexconker]

Chart your weight, exercises, diet (exactly) every day.
Add in weights of your pees and poos, along with their calorie measurements (you'll need a bomb calorimeter).

Now that's science.

Re:

[jedidiah]

Exercise is certainly important in counteracting the body's inclination to lower your metabolic rate to account for a sudden decrease in intake.

Although extra energy burned during exercise is a relatively small amount compared to what kind of calorie deficit you need in order to lose weight.

Re:

[Rick Schumann]

Exercise is certainly important in counteracting the body's inclination to lower your metabolic rate to account for a sudden decrease in intake.

Sure, we're in agreement there, but we're discussing the accuracy (or lack thereof) of 'fitness bands' for calories burned. Everyone should exercise in some significant way on a regular basis for best overall health. Of course you don't need any sort of technology beyond a decent pair of walking/running shoes to do that, either. ;-)

And further

[Okian Warrior]

There are too many variables involved in determining calories burned by any biological entity, and these 'fitness bands' are not the only device that has this problem, either...

To further put this into context, if your body takes in 2 calories more per day than is needed, you will be obese in a year.

No diet has that level of accuracy, and there are variations within different samples of food as well. That 10oz of chicken might vary by dozens of calories, depending on random circumstance.

Laboratory animals grown with the same caloric intake and same access to exercise are obese, compared to ones grown decades ago. The trend over time is consistent and compelling. We're also seeing

Re:

[BradleyUffner]

To further put this into context, if your body takes in 2 calories more per day than is needed, you will be obese in a year.

That doesn't sound right to me. An extra 2 calories per day for one year is 730 calories. Even eating an extra 1000 calories in a year isn't going to qualify someone as obese unless they are practically right at that line anyway.

Re:

[doogles]

To further put this into context, if your body takes in 2 calories more per day than is needed, you will be obese in a year.

That doesn't sound right to me. An extra 2 calories per day for one year is 730 calories. Even eating an extra 1000 calories in a year isn't going to qualify someone as obese unless they are practically right at that line anyway.

My understanding is that one pound is roughly 3500 calories. So a 730 calorie overage, over the course of a year, would theoretically only represent ~0.2lb.

-jd

Re:

[tlhIngan]

That doesn't sound right to me. An extra 2 calories per day for one year is 730 calories. Even eating an extra 1000 calories in a year isn't going to qualify someone as obese unless they are practically right at that line anyway.

It isn't. Just take it this way, an extra 10 calories a day means you gain just over a pound a year (1 lb of fat == 3500 calories). 50 calories a day, or roughly a tablespoon of mayo, can cause serious weight gain (5lbs). Just think - an extra spoon of mayo in your sandwich can caus

Re:

[antifoidulus]

Except its not that simple, fat is not an inert tissue(which is why people who lose weight can never return to their old eating habits, even if they were at equilibrium before trying to lose weight). So every pound of fat will consume a certain # of calories every day for upkeep(the younger and well maler you are the more that number goes up). So while you will consume 17500 or so more calories, you won't actually gain 5 lbs of weight, you will gain something less, but thats much harder to accurately quan

Re:

[Rick Schumann]

"Variability in the caloric content of foods"

That's completely correct, too.

Over 900 environmental causes have been identified as potential causes of obesity (and researchers are working through the list)

One of the more alarming ones I've heard of is pesticide residues in common foods. For instance: glyphosate (active ingredient in Monstantos' Roundup) in wheat, peas, corn, and beans. They literally drench crops in it just before harvest to increase yield. It's a metabolic inhibitor. I'm starting to think that many people who think they have a problem with 'gluten sensitivity' actually have a problem with chronic exposure to glyphosate residue in wheat. Most 'gluten free' prod

Re:

[dbIII]

That may be a point for some, but there really are a few people who really are make seriously ill by gluten even if it's nowhere near the number of people who think they may have a problem.
The current gluten-free craze is at least helping those people who are actually harmed by it and highlighting odd things like gluten in cornflakes. I have no problem with gluten but I'm a little annoyed that many cornflakes are full of cheap filler materials instead of being mostly corn.
On the pesticide issue one example

Re:

[Dare nMc]

Wow, where do you get this crap from?
>I've heard of is pesticide residues in common foods. For instance: glyphosate
glyphosate is not a pesticide, it is a herbicide.
> They literally drench crops in it just before harvest to increase yield.
how would that increase yield? Roundup only kills plants, doesn't fertilize, doesn't directly affect bugs... The only reason to spray, is early enough in the growth cycle that the weeds do not compete with the crop. People with these theories really think farmers a

Re:

[Khashishi]

Well, if you breathe through the device, then maybe you can track calories. Who would want to wear a breathing apparatus to exercise though?

Re: 'Tracking calories' is basically impossible

[Dimitar Dimitrov]

My fitness tracker reports you are wrong.

Re:

[ausekilis]

I personally like Chris Pratt's method [onsizzle.com] for weight loss.

Color me surprised

[Anonymous Coward]

So strapping an accelerateometer to your wrist isn't an accurate way to measure calories, even with all that big-data, cloud computing, web 2.4 social media magic algorithms?

Next you'll be telling me this $500 web connected juice press was a waste of money, and that someone can hack my IoT light-bulbs!!!!!

Garmin?

[cmeans]

How on earth could they not include any one of the Garmin devices? I guess they had costs involved but still seems like they'd include one of the (I assume) most popular fitness trackers available.

Re:

[Dixie_Flatline]

There's really only two that would've worked, but it's true that they should've included them.

Re:

[geek]

9 times out of 10 these studies have to get some sort of approval to test the devices. Garmin likely chose not to participate.

Re:

[necro81]

9 times out of 10 these studies have to get some sort of approval to test the devices. Garmin likely chose not to participate.

Have to?! According to whom? I could walk into any number of retain outlets and get purchase one today. How is the company then going to prevent me from using it as part of a study?

Re:

[erice]

How on earth could they not include any one of the Garmin devices? I guess they had costs involved but still seems like they'd include one of the (I assume) most popular fitness trackers available.

It doesn't really matter, other than less exposure for Garmin. None of the consumer devices, Garmin included, have the instrumentation to get reliable calorie estimates. GPS tracks and heart rate are not the rate data.

Rather surprisingly, Garmin units will give you a calorie estimate even when there is no heart rate data. It is just guess work based on knowing what activity you are doing and how long, along with user entered data like height, sex, and weight. Garbage, really. It gets even worse if yo

Re:

[anjrober]

i have had 4 different garmin running watches over the years and recently switched to the apple watch.
the garmins all estimated a significantly higher calorie burn than the apple watch.
the garmin's were also higher than i believed was correct and what my diet tracking and weight indicated.
i believe they were estimating high. imho.

Must have shiny!

[WillAffleckUW]

Shiny makes feel good!

Pretend to self shiny is for health, not to be toy!

old devices

[bobm]

The bigger issue here is that most of the devices tested were gen 1 devices and I think that the technology and software has gotten better.

Basis Peak is not even on the market anymore.
The rest have been surpassed by multiple generations.

Re:

[dbIII]

It's not a technology issue it's an issue of measuring metrics that are poor indicators of the metric that you are actually interested in.
Increased accuracy in the wrong metric isn't going to help.

There's nothing wrong with using these devices as a rough indicator of the amount of activity, it's just that people expected (on the basis of advertising to an extent) that it would provide more than that.

A quick look at a map of where you've been is just as good as pedometer. Heart rate measurement is interes

Here is a link to the actual paper

[Xoc-S]

http://www.mdpi.com/2075-4426/... [mdpi.com]

If "calories" are measured, the device is defect

[ffkom]

or at least very obsolete, given that the official SI unit for food energy is the Joule.

Re:

[thegarbz]

I've yet to see Joules listed on nutritional labels, and I live in a metric country.

The only thing defect would be to give people units they don't understand to compare to units that aren't listed anywhere to appease idiots who insist on SI for SI sake.

People should use common units. Not SI units, but COMMON units. You hear that you weirdos with your miles?

Re:

[ffkom]

Like over 90% of the earths population I, too, live in a country that signed the SI treaty - and the "nutritional information" printed on food here is obliged to present the food energy in Joule. (Many still also print "calorie" values in addition, but that is optional.)

The most bizarre thing about "calories" is that most people who talk about them use numbers that are off by a factor of 1000 - just as if people were talking about "bytes" while in fact they mean kilobytes.

Re:

[thegarbz]

The most bizarre thing about "calories" is that most people who talk about them use numbers that are off by a factor of 1000 - just as if people were talking about "bytes" while in fact they mean kilobytes.

Oh true that. It's like the k in kcal on the label is meaningless. Food labels get this right, but what really annoys me is references like Google's search results could correct for this. Googling "Calories in an Apple" gets you "52 calories" per 100g. Not 52kcal not 52000cal but 52cal.

What chance have people got if their reference material is wrong. "You're saying it wrong"... "nuh uah Google it".

What does the expended energy even mean?

[SLi]

If you say a certain one-hour exercise burns, say, 600 kcal, is that in addition to the expenditure by the resting metabolism, or including it? I've tried to parse this from the instructions of different monitors, but so far utterly failed to find anything.

Re: What does the expended energy even mean?

[Anonymous Coward]

As far as I've been able to discern, they all include it. And I find it annoying because I'd like to know what the exercise netted me, but they all want to pad their numbers. It's not difficult math to figure it out, but it sure would be helpful if these devices would give the breakdown.