Moving the Needle on “Moving the Needle”

Summary

This paper introduces the nomenclature of performance-based accountability systems (PBAS) to the expanded learning field, provides a policy case study for a countywide system in southern Florida and uses data from that system to explore the issue of quality thresholds. We present an expanded design standard to guide development and improvement of PBAS policies and further develop a theory of lower-stakes accountability to guide effective use of incentives of various types. Findings suggest that (1) the PBAS framework defines critical concepts and improves our ability to describe existing quality improvement systems, (2) the Youth Program Quality Assessment (Youth PQA) can be used to produce a program rating of sufficient reliability for use in a PBAS, and (3) that the Palm Beach County PBAS
design is an exemplar for expanded learning policies.

General recommendations for PBAS designs include:

• PBAS design should differentiate roles and link performance measures to incentives targeted at specific management and service delivery roles.
• PBAS designs should include program ratings for multiple service domains linked to a mix of higher- and lower-stakes incentives.
• PBAS should emphasize participants’ understanding of performance levels and sense of fairness while evolving toward higher-stakes incentives over time.

Detailed recommendations for Weikart Center clients using the Youth Program Quality Intervention and related Program Quality Assessments as the basis for an expanded learning PBAS design include:

• Recommendations for best practice in each element of the seven elements in PBAS design
  standard.
• Detailed description of a composition map for program ratings and performance levels for nine commonly used measures in expanded learning PBAS.
• A PBAS design exemplar based on the Palm Beach County case describing specific combinations four types of incentives (financial, customer review, supervisory review, access to data) with two types of performance levels (high and low) and nine program ratings to achieve an optimal, lower-stakes, PBAS design with higher-stakes elements.