Using Novel Multimethod Evaluation Approaches to Understand Complex Food System Interventions: Insights from a Supply Chain Intervention Intended to Improve Nutrition

Abstract

Background

A “food system” approach to improve diet quality by intervening within areas such as food supply chains is gaining prominence. However, evidence of such interventions’ impact, and understanding of appropriate methods to evaluate them, is lacking.

Objectives

We present an impact evaluation of an intervention that aimed to increase consumption of nutritious foods by supporting food-producing firms in Kenya. In doing so, we demonstrate how multiple methods, including those from other disciplines, can be used to evaluate a complex food systems intervention.

Methods

Four methods focused on food-producing firms and their management, including a survey of intervention participants ( n = 83 individuals), a “laboratory-in-the-field” experiment ( n = 83 individuals), baseline/endline data on firm performance ( n = 71 firms), and semistructured interviews ( n = 19 firms). Three methods focused on consumers in neighborhoods targeted by a supported firm: a randomized field experiment tested effects of making a supported product exhaustively available on consumers’ purchases and consumption ( n = 1295 consumers); 3 discrete choice experiments ( n = 1295 consumers) tested factors influencing consumers’ willingness to pay for foods with relevant characteristics.

Results

Among firms, we saw suggestive evidence of increased networking and business relationships, while laboratory-in-the-field experiments indicated the intervention might foster cooperation among participants. Qualitative interviews suggested that the intervention enabled firms to increase production, improve management, increase revenues, and lower costs. Baseline/endline data confirmed a positive effect only on the launch of new products and hiring workers. In the field experiment, consumption of the supported product increased in areas where it was made available relative to a control group, but this did not increase overall consumption of the food type or dietary diversity.

Conclusions

Results showed positive signs of the intervention improving firm-level outcomes but limited impact on consumers’ diet quality. The evaluation also demonstrates how diverse methods can be used to evaluate complex interventions.