Our approach to program evaluation increases the usefulness of all estimates produced as part of an impact analysis. We replace the “no-treatment” counterfactual condition (i.e., children who were not exposed to an afterschool program) with low-implementation conditions (e.g., children who were exposed to lower-quality practices) in order to describe the impact of optimal implementation on child outcomes.
Pattern-centered theory and methods (PCM) can be used to tell simple and accurate stories about how real persons grow in real school and afterschool classrooms. Stories about the quality and outcomes (i.e., causes and effects) that are modeled using PCM are particularly useful because they can address questions related to “how” programs and classrooms work and “how much” children grow skills.
Are programs producing weak or no effects? Or are gold-standard designs missing something? In our third White Paper, Realist(ic) Evaluation Tools for OST Programs: The Quality-Outcomes Design and Methods (Q-ODM) Toolbox, we extend from the neuroperson framework for socio-emotional skills to a focus on evaluation design and impact evidence.
In this second white paper, Measuring Socio-Emotional Skill, Impact, and Equity Outcomes, we share hard-won lessons from two decades of work attempting to describe the actual outcomes of “broad developmentally-focused programs.” We answer this key question: Why are many SEL skill measurement constructs likely to be inaccurate despite psychometric evidence of reliability and validity?
In conjunction with the release of our first white paper, “A Framework for Socio-Emotional Skills, Quality, and Equity,” we answer the following questions: What are socio-emotional skills? Why are they important? Why are organizations and policies struggling to implement socio-emotional skill reforms? How can our framework help increase the effectiveness of reform?
The first in a series of four, “Socio-Emotional Skills, Quality, and Equity,” provides a translational framework for understanding our relatively unique view of the key parts of a socio-emotional skill set. We develop a case for supplementing the traditional focus on student beliefs and behavior with a much more extensive focus on students’ emotional life and the attention skills necessary for becoming the primary authors of their own development.
Share on linkedin LinkedIn Share on twitter Twitter Share on email Email Download Socio-emotional learning (SEL) skills are a partial but necessary cause of children’s developmental outcomes, and SEL skill growth is a key objective for nearly all out-of-school time (OST) programs. The Quality-Outcomes Design and Methods (Q-ODM) toolbox holds an integrated set of tools … Continue reading Realist(ic) Evaluation Tools for OST Programs: The Quality-Outcomes Design and Methods (Q-ODM) Toolbox
Share on linkedin LinkedIn Share on twitter Twitter Share on email Email Download The positivist theory and methodology used by most researchers and evaluators is poorly suited for addressing the formative explanations that guide continuous quality improvement (CQI) processes and the nuanced impact models that pertain to questions about how and how much. QTurn’s Quality-Outcomes … Continue reading Measuring Socio-Emotional Skill, Impact, and Equity Outcomes
Share on linkedin LinkedIn Share on twitter Twitter Share on email Email Download Evaluation evidence about the relations among children’s prior history, engagement in program settings, resulting SEL skill growth, and the ultimately desired transfer outcomes (e.g., agency to succeed in other settings) has been sporadic and fragmented. One reason for this may be that … Continue reading Socio-Emotional Skills, Quality, and Equity: The Multilevel Person-in-Context~neuroperson (MPCn) Framework
The Guidance for Out-of-School Time Learning at a Distance (GOLD) is a program quality assessment for 21st Century Community Learning Centers (21st CCLC) and other community-based programs (e.g., school-age childcare, YMCA, 4 H, Boys & Girls Clubs) that have transitioned virtual, socially distanced in-person, and blended service models. For these new models, the GOLD will: … Continue reading Guidance for Out-of-school time Learning at a Distance (GOLD)
