Policy Research for Front of Package Nutrition Labeling: Developing and Testing a Summary System Algorithm. 5.2 Limitations


Some of the limitations of our design are described in this section. The scoring of foods using the algorithms depends on the nutrient database used. The FNDDS has some limitations. For example, trans fat is not in FNDDS, and it would have been time prohibitive to add this information to each food in the database. Another limitation of the FNDDS is that the database uses a default choice of added salt, which resulted in high sodium content for some foods (e.g., baked potato) that was reflected in lower algorithm scores (the dietary recall in NHANES asks respondents if they added salt to foods at the time of eating, and the dietary intake data reflect the respondent's answer). The use of the major food grouping that is used in FNDDS to examine scores by food groups was limiting. The classification of some food items into major groupings was awkward, such as pickles and salsa in the vegetable group. Because of the high sodium content, various condiments had very low scores and skewed the overall vegetable score. Some food groups had a smaller number of foods and could be skewed by outliers. In theory, we could have examined scores of all 4,000+ foods in FNDDS, but it would have been difficult to assess the scores of that number of foods given the time constraints.

The selection of nutrients would need to be reassessed in the future as new evidence emerges for nutrients of importance and new federal dietary guidance is issued. The daily values used in the algorithm would also need to be updated when daily values change based on new federal dietary recommended intake guidance. The nutrient values per RACC would also need to be updated if standard serving sizes change in the future.

The two methods used in the project to test the algorithms for construct validity were to examine food scores for reasonableness and to validate composite algorithm scores for dietary intake against a measure of overall dietary quality. Examination of food scores is an important step but is a descriptive process rather than a formal quantitative assessment. It is difficult to compare food scores among all the algorithm modifications and identify which set of food scores is the best representative of nutritional quality. Validation against a measure of diet quality is a recommended approach to evaluate a nutrient scoring system, but it depends on the validity of the index used to measure dietary quality. The HEI is based on federal dietary recommendations found in MyPyramid and the 2005 Dietary Guidelines and has been validated (Guenther, Reedy, Krebs-Smith, & Reeve, 2008). The relationship between HEI and a nutrient scoring system also depends on the particular components in each score and how much these components influence the overall score. The HEI validation study demonstrated that certain components of the HEI have more influence on the score, that is, calories from SOFAAS (solid fats, alcohol and added sugars) and fruits (Guenther et al., 2008). Our algorithms demonstrated high food scores for fruits and showed better agreement with HEI scores when added sugar was present in the algorithm. With the new 2010 Dietary Guidelines, the HEI may be updated and it will be important to reevaluate the algorithm.

The focus of our formal validation was the ability of the algorithm to predict dietary quality, an important property of a summary scoring system. Nevertheless, if a summary scoring system is used in FOP labeling, the score will be shown on each food, and the ability of the algorithm to validly represent the nutritional value of each food would also be important. We qualitatively assessed the scores of foods and food groups, but we did not have an available method for formal validation of individual food scores.

The final algorithms were derived from dietary intake data from NHANES 2005-2008. Specifically, the weighting factors are the beta coefficients from regressions models of nutrient intakes against HEI scores. The reliability of the algorithm with other dietary intake data sets is unknown and will be important to test with future NHANES data when it becomes available.

Our results apply only to summary FOP systems; we did not consider nutrient-specific systems that are the other general type of FOP system. On the one hand, nutrient-specific systems can help consumers identify key nutrients in a food, but they do not provide an overall assessment of the product. On the other hand, summary systems can provide an overall assessment of the nutritional quality of a food, but they do not tell the consumer why the food received the score or rating (e.g., high calcium and low saturated fat content of low-fat milk).