There are increasing demands for Participatory Arts-Based (PAB) programs involved in health research to better evidence outcomes using robust quantitative evaluation methodologies taken from science, such as standardized questionnaires, to inform commissioning and scale-up decisions. However, for PAB researchers trying to do this, barriers arise from fundamental interdisciplinary differences in values and contexts. Researchers are required to navigate the tensions between the practice-based evidence produced by the arts and the evidence-based practice sought by psychologists. Consequently, there is a need for interdisciplinary arts-science collaborations to produce alternative methods of evaluation that are better aligned to PAB approaches, and which combine systematic rigor with a sensitivity to the values, contexts and strengths of this approach. The current article centers on the development of an alternative transdisciplinary analytic tool, the Participatory arts Play Framework (PP-Framework), undertaken as part of an arts-psychology collaboration for a UK AHRC-funded PAB research project: Playing A/Part: Investigating the identities and experiences of autistic girls. We present details of three stages in the development of the PP-Framework: 1. preliminary emergence of the framework from initial video analysis of observational data from participatory music and sound workshops run for 6 adolescent autistic girls (aged 11–16); 2. identification and application of modes of engagement; and 3. further testing of the framework as an evaluation tool for use in a real-world setting, involving professional musicians engaged in delivery of a creative music project at a center for homeless people. The PP-Framework maps types of participation in terms of performative behaviors and qualities of experience, understood as modes of play. It functions as a vehicle for analyzing participant engagement, providing a tool predicated on the processes of working in creative participatory contexts while also being sensitive to the esthetic qualities of what is produced and capable of capturing beneficial changes in engagement. It offers a conceptual approach for researchers to undertake observation of participatory arts practices, taking account of embodied engagement and interaction processes. It is informed by understandings of autistic performativity and masking in conjunction with an ecological understanding of sense making as being shaped by environments, social relations and sensing subjectivity. The framework has the potential to be a bi-directional tool, with application for both practitioners and participants.