Blog

Accuracy

At the core of Pertinacity is a proxy for calories. To evaluate this proxy we need to consider precision and accuracy. The previous blog post addressed precision. In this post we address accuracy.

Recall the calorie proxy:

  • Imagine squashing the food item.
  • Make a fist. Estimate how many fists would be the same size as the squashed food item.
  • Exclude drinks known to have zero calories (water, diet soda, tea, black coffee, etc.), but include other drinks like smoothies, alcohol, regular soda, milk, juice, etc.

An evaluation of accuracy is best done in terms of bias: bias: The average value of errors in a measurement. Lower bias measure higher accuracy.

Biases in Calorie Counting

Calorie counting systems possess several biases. They still help people lose weight, but they do so in spite of the biases.

Calorie counting systems usually have two parts: (i) a measurement of the calories you consume in a day, and (ii) a limit on the number of calories you consume in a day. The limit is derived from a model like this:

Calories In - Calories Out < 0

where Calories In is the number of Calories you measures and Calories Out is given by

Calories Out = Estimated Energy Requirement (EER) = Calories expended to maintain weight as a function of age, gender, weight, height, and level of activity

Bias: EER

EER is defined in report Institute of Medicine Dietary Reference Intakes (DRI) as "the average dietary energy intake that is predicted to maintain energy balance in a healthy adult of a defined age, gender, weight, height, and level of physical activity, consistent with good health". The DRI presents a statistical model of EER based on age, gender, etc. Like any model there is some uncertainty (or spread as defined previously). The spread in the estimate of EER is about 200 Calories for males and 180 Calories for females Table 5-14.

When applying the EER estimate to calorie counting we'll get a number for Calories Out and try to eat less than that number for many days in a row. Imprecision in the estimate of that number thus appears as a persistent bias in our calorie counting system. The DRI describes the situation this way: "By definition, the estimate would be expected to underestimate the true energy expenditure 50 percent of the time and to overestimate it 50 percent of the time, leading to corresponding changes in body weight."

EER cross-sectional imprecision leads to a persistent bias in a calorie-counting system -- in Calories Out -- of around +/- 200 Calories.

Bias: Calories In

If you've ever dieted you'll be unsurprised to know that people underreport the amount of calories they've eaten. They even do it when they're not dieting. They even do it if they're dieticians (see below).

A paper, What are people really eating? The relation between energy intake derived from estimated diet records and intake determined to maintain body weight, based on the Beltsville study data, finds underreporting of calories by 18% for both men and women. This corresponds to about 550 Calories/day for men and 430 Calories/day for women.

Other studies also find that subjects underreport calories consumed. Validity of self-reported energy intake in lean and obese young women, using two nutrient databases, compared with total energy expenditure assessed by doubly labeled water. concludes "Both physically active lean and sedentary obese women under-reported TEI...". This paper measured underreporting of 23%, 30%, 39%, and 38% in the four different groups (lean and obese women using two different calorie databases) that were studied.

Dieticians did a better job than non-dieticians in Energy intake and energy expenditure: a controlled study comparing dietitians and non-dietitians., but dieticians still underreported by a little over 200 Calories/day. Non-dieticians in this study underreported by over 400 Calories/day.

Underreporting give a persistent bias to Calories In of around 400 Calories.

Bias: Activity

Activity level is specified categorically as Sedentary, Low Active, Active, or Very Active. These categories appear in the estimation formulas for EER given on p. 185 of the DRI.

Each category corresponds to a range of calories expended on physical activity. We can use the formulas to estimate the span of calories covered for a category. For example, for a man weighing 180 lb. (81.6 kg) and standing 5' 10" (1.78m) tall, we have

Sedentary   0-248 Calories, span = 248 Calories
Low Active  248-565 Calories, span = 317 Calories
Active      565-1084 Calories, span = 519 Calories
Very Active >1084 Calories

If we change the gender, height, or weight in the calculations we'll get different answers. Also, it is unclear what the span of "Very Active" is, since there is no specified or obvious value for the upper limit on the amount of calories a person can expend through physical activity.

The point, however, is that there is some uncertainty in the specification of activity level. If the example individual correctly chose "Low Active" as his activity level he would have an uncertainty in the calories expended of 317 Calories/2 = 158.5 Calories. This discrepancy is already included in the EER bias discussed above.

If he chose the wrong category he'd introduce a new, persistent bias that is not part of the EER bias. Since overreporting of physical activity has been observed, there might also exist bias in a person's selection physical activity category.

Dealing with Bias

These biases affect the quality of your calorie counting by creating a diet that, on average, has more calories than expected. Additionally the biases are persistent which means you should expect to be biased by the same amount every day. In other words, you can expect to overeat relative to your goal every day if you use the formulas directly.

Fortunately, the DRI also provides methods for dealing with bias.

Method I (Dynamic):

"This indicates that monitoring of body weight would be required when implementing intakes based on the equations that predict individual energy requirements. For example, if subjects were enrolled in a study in which it was important to maintain body weight, each individual would be fed the amount of energy estimated to be needed based on the EER equation. Body weight would be closely monitored over time, and the amount of energy provided to each individual would be adjusted up or down from the EER (or TEE) as required to maintain body weight."

Method II (Static):

"If the goal of planning is to prevent weight gain in an individual with specified characteristics, the appropriate EER equation could be used to derive the aver- age energy expenditure for the individual, and then subtract an amount equal to two times the SD. This would lead to an intake that would be expected to fall below the actual energy requirements of all but 2.5 per- cent of the individuals with similar characteristics. Using the above example for the 33-year-old, low-active woman, the energy requirement would be 2,028 – (2 × 160) kcal, or 1,708 kcal. This intake would prevent weight gain in almost all individuals with similar characteristics. Of course, this level of intake would lead to weight loss in most of these individuals."

Method I says to start with the amount recommended by the EER formula, monitor a person's weight, and adjust the calorie intake accordingly if weight is not maintained. Method II says to set the calorie intake so low that you overcome the "spread" (or imprecision) of the estimate and increase the probability of avoid weight gain or inducing weight loss.

Method I addresses weight maintenance rather than weight loss, but one could modify the suggestion to say, "If weight loss is not achieved, then adjust the energy provided down." In other words, if you're not losing weight, trying eating less. In fact, if the purpose is solely to lose weight, that adaptive strategy should work from any starting calorie intake -- not just from the EER estimate.

Pertinacity adopts Method I with the modification for weight loss just described. Additionally, Pertinacity uses as a starting point (instead of the EER formula) a measurement of your current caloric intake. In short, it says, "Try eating less than you have been and see if you lose weight." And it keeps saying that every day.

Specifically, Pertinacity recommends a maximum of one fist-size portion (FSP) reduction in your calorie intake relative to your current level of intake. Based on the data collected from Amazon's Mechanical Turk (see previous blog post for a description of the data), this reduction should be equivalent to 11% on average, and about 6%-13% for most people. Since the recommended limit on FSP is always based on the trailing two weeks of measurements this number will change slowly. (It won't decrease by 11% every day, for example. It'll start at an 11%-ish reduction then when you are successful at eating at the reduced level for a couple of weeks it'll make another reduction. The process is gradual.)

Notice that using this moving average method for recommending a limit on intake obviates the need to explicitly model calories in terms of FSP. That process would require more data from the user, so it represents another simplification and, thus, another improvement in ease-of-use. Additionally, due to the use of this method, Pertinacity does not need to collect data on activity or the other user characteristics needed to estimate required energy intake.

To deal with biases in calorie counting systems and avoid having to model calories in terms of FSP, Pertinacity employs a dynamic algorithm that works like this: Reduce your FSP intake and that will reduce your calorie intake which will, in turn, reduce your weight.

Conclusion

Calorie counting systems have several biases embedded in them, including (i) statistical error in the EER estimate of calories required (+/- 200 Calories), (ii) underreporting calories consumed (400-500 Calories), and (iii) overreporting of activity. Nevertheless, the systems work for their users, in part, because of the use of a bias-defeating method like (I) dynamically decreasing the calorie intake recommendation until the desired result is achieved, or (II) recommending a large, static, change in calorie intake.

Pertinacity uses a gradual, dynamically-changing limit on the number of fist-sized portions the user may consume. This makes the system accurate and simplifies it dramatically.