Evidence is central to PFS and plays a prominent role in three main stages of PFS projects: finding evidence-based programs, implementing programs in a manner consistent with past evidence and responsive to evidence emerging in real time, and generating evidence through evaluation to assess achievement of outcomes and (if applicable) payment.
Identifying programs that might be good candidates for PFS requires the ability to recognize and interpret evidence and distinguish strong evidence from weak. We identify four categories of evidence in ascending order of strength: potential, promising, model, and strong. Intuitively, programs with the strongest evidence have a stronger likelihood of success and lower risk. Conversely, programs with the weakest evidence bases carry greater risk, and funders will likely expect higher returns. For characteristics of high-quality evaluations, please consult Poulin, Orchowsky, and Trask (2011).
- Potential evidence. A program at this level should have an underlying theory of change that is rooted in the literature and clearly demonstrates (through evidence, including observational studies, from other interventions) how the intervention might cause the intended outcome. The program’s theory of change should link the intervention to a specific, defined outcome, but it lacks a high-quality program evaluation that confirms causality.
- Promising evidence. A program at this level meets three conditions. First, it clearly identifies the outcome the program is designed to change, the specific risk and/or protective factors targeted to produce change, the population for which it is intended, and how the components of the intervention work to produce this change. This condition is sometimes referred to as intervention specificity. Second, evaluations of the program trials produce valid and reliable findings through a minimum of (a) one high-quality randomized controlled trial or (b) two high-quality quasi-experimental evaluations. Third, the preponderance of evidence from the high-quality evaluations indicates significant positive change in intended outcomes that can be attributed to the program, and there is no evidence of harmful effects.
- Model evidence. A program at this level meets all the “promising evidence” qualifications and two more. First, the program has undergone a minimum of (a) two high-quality randomized controlled trials or (b) one high-quality randomized controlled trial and one high-quality quasi-experimental evaluation. Second, the evaluations demonstrate positive intervention impact for a minimum of 12 months after the program intervention end.
- Strong evidence. A program at this level meets the criteria of “model evidence” and also has sufficient evidence to conduct a meta-analytic review. This review should yield evidence of statistically significant effect sizes.