During times of crisis when programs are under tremendous pressures, evaluation and assessment can be challenging. Programs enter triage mode, putting their limited time and energy into the most urgent tasks. This heightens the need for evaluation that reduces strain and improves capacity. When conditions that created the crisis are long-lasting, like the coronavirus pandemic, it becomes necessary to revisit and restore vital activities that may have been moved to the backburner, but to do this successfully often requires intelligent redesign. How can the same needs be met in a new way? How can evaluation and assessment be adapted to succeed in challenging conditions?
QTurn has developed a comprehensive evaluation plan for afterschool programs at a moment when redesign of the service and delivery of the redesigned service are happening at the same time. This plan, ”Afterschool Evaluation Plan 2020,” was developed to address the unique needs of programs in the 2020-2021 school year – and support compliance with the specific requirements for the 21st Century Community Learning Centers. An evaluation plan for 2020 must remove burdens rather than adding to them and make life easier not harder. Aware of these needs, QTurn’s design includes short, validated assessment tools, guidance available through online trainings, and a reassuring, therapeutic approach referred to as the “lower stakes” model.
A Lower Stakes Approach
The lower stakes model is a strong component of QTurn’s work. Lower stakes means that the results of assessments are used to support and inform, and program staff are able to interpret the meaning of their own individual and group performance data. In a lower stakes model, the results of assessments do not influence funding or prompt sanctions. Instead, low assessments scores are opportunities for mutual learning, support, and growth.
While this approach has been integral to the work of QTurn’s founder, Charles Smith, for decades, lower stakes is especially critical in the 2020-2021 school year as program staff strive to adapt to a new normal. The AEP 2020 is intentionally designed to alleviate stress and confusion and help staff adapt to rapid change and achieve shared meaning.
User-Friendly Assessment Tools
The Afterschool Evaluation Plan 2020 includes three assessment tools designed for remote or in-person programming (or a combination of both). The first measures fidelity to best practices at the management level. The second captures quality at the point-of-service (and applies to home learning environments). The third charts the growth of social and emotional learning (SEL) skills among youth. The tools are short and easy to use. Designed to work together as part of the cycle, they also support impact evaluation.
Management Practices Self Assessment (MPSA). The MPSA was developed with extensive input from 21st CCLC program project directors and aligns with the core requirements for 21st CCLC programs in Michigan. With 24 indicators forming eight standards in four domains, the tool requires less than two hours for program managers to complete.
Guidance for Out-of-School Time Learning at a Distance (GOLD) Self-Assessment. Site managers and staff complete a self-assessment produced with extensive input from other expert 21st CCLC site managers. GOLD contains 27 indicators that form 11 standards in four domains. The four domains represent point-of-service quality in the individual learning environment.
Adult Rating of Youth Behavior (ARYB). Each child is rated on the ARYB in November and April. By completing the assessment at two time points (earlier in the school year then again toward the end of the year), the ARYB is able to capture growth in social and emotional skills across the school year. The ARYB has 30 items that form six skill domains, including emotion management, teamwork, and responsibility.
Additional Features:
Cost-Effective. The Afterschool Evaluation Plan (AEP) 2020 can be adapted to a wide range of cost structures. The Guidebooks and scoring forms are available for free to all users. Additionally, the demands on time are low. Completing the assessments can require as little as 2 hours for program directors and 3 hours for site managers and staff per year.
Guidance Through Online Training. QTurn will offer live online trainings covering the use of the MPSA, GOLD, and ARYB. Support also includes online training that equip leaders and staff to do data informed planning.
Emphasis on School-Day Alignment. QTurn’s AEP 2020 helps programs pivot toward greater integration with schools during a time when school has become more challenging for many children.
Support for Impact Evaluation. Finally, data obtained using the assessment tools can be used to evaluate the overall impact of programs, particularly across multiple programs.
One thought on “Continuous Quality Improvement and Evaluation in 2020: A Plan for 21st Century Community Learning Centers”