The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations

Abstract

Background

Current evaluation methods are mismatched with the speed of health care innovation and needs of health care delivery partners. We introduce a qualitative approach called the lightning report method and its specific product—the “Lightning Report.” We compare implementation evaluation results across four projects to explore report sensitivity and the potential depth and breadth of lightning report method findings.

Methods

The lightning report method was refined over 2.5 years across four projects: team‐based primary care, cancer center transformation, precision health in primary care, and a national life‐sustaining decisions initiative. The novelty of the lightning report method is the application of Plus/Delta/Insight debriefing to dynamic implementation evaluation. This analytic structure captures Plus (“what works”), Delta (“what needs to be changed”), and Insights (participant or evaluator insights, ideas, and recommendations). We used structured coding based on implementation science barriers and facilitators outlined in the Consolidated Framework for Implementation Research (CFIR) applied to 17 Lightning Reports from four projects.

Results

Health care partners reported that Lighting Reports were valuable, easy to understand, and they implied reports supported “corrective action” for implementations. Comparative analysis revealed cross‐project emphasis on the domains of Inner Setting and Intervention Characteristics, with themes of communication, resources/staffing, feedback/reflection, alignment with simultaneous interventions and traditional care, and team cohesion. In three of the four assessed projects, the largest proportion of coding was to the clinic‐level domain of Inner Setting—ranging from 39% for the cancer center project to a high of 56% for the life‐sustaining decisions project.

Conclusions

The lightning report method can fill a gap in rapid qualitative approaches and is generalizable with consistent but flexible core methods. Comparative analysis suggests it is a sensitive tool, capable of uncovering differences and insights in implementation across projects. The Lightning Report facilitates partnered evaluation and communication with stakeholders by providing real‐time, actionable insights in dynamic health care implementations.