Case Study: Pilot Test
USDA’s Food & Nutrition Service: Digital Nutrition Education for Middle School
Project Background
USDA’s Food and Nutrition Service continues to explore innovative avenues for promoting nutrition education and healthy, lifelong habits while adjusting to a changing educational landscape. FNS contracted CarrotNewYork to help in this pursuit and create digital, project-based learning units for 6th-8th grades that align to nutrition and ELA standards.
ACR was subcontracted to CarrotNewYork to assess the impact of the materials on students’ knowledge, attitudes, and healthy food and beverage choices; verify the materials were easy-to-use, clear, and feasible for teachers, and engaging and relevant for students; and identify barriers and supports for using the materials in middle school classrooms.
ACR’s Approach
First, we recruited a sample of teachers (n=19) that taught a range of subjects (e.g. family & consumer science, nutrition, ELA, social studies) from diverse schools across the country (AZ, IA, FL, MA, & MN) to pilot the materials. Schools and teachers were selected to obtain a diversity of perspectives about the materials.
Since the research objectives were to assess impact and also identify ways to improve the materials, we developed a mixed-methods evaluation, collecting data from multiple audiences. We developed and distributed student surveys (n=494) adapted from the Beverage and Snack Questionnaire (BSQ-1) and included program specific content for each grade level to measure program impact. In addition, we conducted 13 focus groups with students (n=74) and interviewed and collected implementation journals from all participating teachers (n=19) to learn how the materials were perceived and what barriers existed to leading the materials in different classrooms. Finally, by visiting each site we were able to observe obstacles and patterns that were not immediately clear from the data that strengthened the findings and recommendations.
The Impact
Analysis of quantitative data showed statistically significant knowledge gains in 12 of 18 grade-specific content questions and statistically significant healthy changes in the consumption of 19 snacks and beverages. Qualitative analysis supported and added nuance to these positive findings, and led to evidence-based recommendations that were integrated into the final materials available through Team Nutrition – Fueling My Healthy Life.
[1] Neuhouser, M. L., Lilley, S., Lund, A., & Johnson, D. B. (2009). Development and validation of a beverage and snack questionnaire for use in evaluation of school nutrition policies. Journal of the American Dietetic Association, 109(9), 1587-1592.
-
Content Development Partner
ACR brings the highest level of strategic thought, confidence, and professionalism to all projects. ACR has often been required to meet tight deadlines and manage multi-level client approval processes. They always deliver on time and without cutting corners, while managing expectations along the way.
-
Client, City Agency
ACR takes the time to understand what I’m looking to learn from each evaluation, and they design and revise their methods to answer my specific questions. As a result, their research captures meaningful data and evidence of impact, and their analysis and recommendations directly relate to my programs. Their work has directly influenced programmatic decisions and policy.
-
Client, Higher Ed
ACR’s knowledge, insight and understanding of our goals helped to foster a very collaborative working relationship. ACR’s team members became an invaluable part of my project team.
