Critical Takeaways from the Success of the Denver Supportive Housing Pay for Success Initiative
By Roger Low and Stephanie Mercier
The Project
In Denver, Colorado, many residents experiencing chronic homelessness have long histories of rotating through multiple state and local systems — including jails, shelters, hospitals, and treatment centers. These cycles result in not only a lower quality of life for those experiencing homelessness, but also massive annual costs to taxpayers and the local community. In response, the City of Denver partnered with the Colorado Coalition for the Homeless and the Mental Health Center of Denver to deliver permanent supportive housing to chronically homeless Denver residents. The Corporation for Supportive Housing and Enterprise Community Partners served as co-intermediaries, providing overall project management and fiscal services.
The City of Denver committed to pay for the project via an innovative “Pay for Success” (PFS) financing approach, also often called a Social Impact Bond (SIB). Under this structure, the City set aside funding for the project, to be paid out only if the project succeeded, based on defined measurable outcomes (whether those served remained in stable housing and avoided time in jail). Eight foundations and impact investors provided $8.6 million in upfront capital for the project via an agreement that these funders would be repaid, with a modest return, over the next five years, only if the project yielded these specifically defined positive impacts.
The key project outcomes were measured via a rigorous randomized evaluation conducted by the Urban Institute, in partnership with a local evaluator at the University of Colorado, Denver. For the past five years, these independent evaluators painstakingly compared a group of over 360 eligible Denver residents randomized into the program against a control group who received services as usual. With limited funded spots in this program, randomization was not only ethical but also likely the fairest way to determine who were offered housing and services.
The Results
After five years, the final results of the project and related evaluation — to use a non-technical term — are incredibly exciting.
Among the significant findings:
· Days in Stable Housing: The final evaluation found that, of participants randomized into the treatment group, “79 percent (285 people) were located, engaged, and housed. After accessing supportive housing, participants maintained high housing stability rates. Of those who leased up and survived the full observation period, 86 percent remained in stable housing at one year … at year three, the rate was 77 percent.” These results are no small feat, given the long prior histories of homelessness that many individuals experienced.
· Days in Jail: The final evaluation found that “three years after being randomized into the [project], housed participants had an average of 85.93 jail days, and individuals in the control group had an average of 138.32 jail days — a difference of 38 percent.”
· Arrests, Visits to Local Shelters, and Visits to Local Detox Centers: Participants randomized into the program also had 40 percent fewer arrests, 40 percent fewer visits to local shelters, and 65 percent fewer visits to a local detox center.
· Emergency Care: Participants also used less emergency health care (a 40% decrease in emergency department visits) and more office-based health care (a 155% increase in office-based visits).
· Full Repayment with Modest Returns: The strength of the housing and public safety outcomes triggered full repayment of the principal, plus returns of just over $1 million, for the investors — a modest ROI that, impressively, almost precisely matches the original projections.
· Financial Impact: The Denver project offset more than half of the total annual per-person cost with reductions in other public services, primarily jail, ambulance and court costs. Further, because state and federal vouchers provided housing assistance, almost all of Denver’s locally funded supportive services were offset by local emergency services cost reductions.
Lessons Learned
The final evaluation overwhelmingly demonstrates the success of this project. As we work to scale proven approaches to address persistent challenges — housing-related and otherwise — facing our communities, these five important takeaways are worth highlighting:
1. Rigorous evaluations are worth the effort. The population that was the focus of this project faced many interrelated risk factors; even among those randomized into the program, arrests and time in jail appear high, compared to the general population. Absent a randomized evaluation, the project’s results likely would have raised questions about the project’s true impact. The randomized control group showed the contrast — that comparable Denver residents cycled through jails and other systems at a far higher frequency, leaving no doubt as to the intervention’s truly massive impact.
The evaluators measured many of these critical outcomes by leveraging administrative data. As the sophistication, security, and interoperability of government data systems improves, the accuracy and usability of this type of rigorous approach will increase, costs will fall, and disaggregating final outcome data by important risk factors (such as race, ethnicity, sex, and geography) will become ever more feasible. In short, with careful attention to privacy and equity, we expect ethically administered experimental evaluations and other rigorous evaluation tools to be an essential part of building evidence across the social sector.
2. Scaling what works can work. The final results also tracked remarkably closely with original projections. The results validate the hard work of the project’s architects, who spent years analyzing data for the target population and studying the available evidence. More importantly, these results underscore the power of an outcomes-driven analysis to improve public programs and lives. Scaling effective programs to new communities really can yield continued strong outcomes, particularly when replicated with the care that the Denver team demonstrated.
3. Strong, sector-transcendent partnerships and good leadership are essential. This evaluation validates the power of collaborative partnerships that transcend the public, private, nonprofit, philanthropic, and academic sectors in creating the conditions necessary for effective program implementation.
We recognize how many different stakeholders worked together to bring about this success — including eight funders, two nonprofit supportive housing providers with dozens of staff, the participants themselves, two evaluators, two intermediaries, and so many others. None of this would have happened without thoughtful leadership by key local elected officials including Denver Mayor Michael B. Hancock and the Denver City Council, as well as expert technical assistance providers such as the Harvard Kennedy School’s Government Performance Lab and Colorado-based Social Impact Solutions.
4. PFS financing remains a potent tool, deployed strategically, to build a case for scale. We continue to believe linking dollars to outcomes is the right long-term strategy. The innovative structure of the Denver project de-risked implementation because the City committed to pay only for demonstrated positive impacts.
The Denver project is, in fact, the latest in a series of successful and promising PFS-funded supportive housing initiatives in several communities, underscoring the value and reach of this model. For instance, the Massachusetts Chronic Homelessness PFS Initiative housed more than 1,000 individuals, 85 percent of whom remained housed or had a qualified/positive exit. Similarly, the evaluation of Project Welcome Home in Santa Clara County, California, showed that 86 percent of the participants who were randomized into supportive housing remained in their housing for several years, as compared to only about a third of the control group.
Collectively, these examples suggest that PFS, used well, can be a valuable tool to incent collaboration, adapt to a local context, deploy effective interventions, and generate cost avoidance across systems and levels of government. From a previous article in the Stanford Social Innovation Review in 2019:
“In some cases, governments can and should fund services that improve outcomes directly, without the complexity of PFS financing. But policy makers face many barriers to doing so. Public dollars for prevention are limited, and costs often diffuse across many systems. Local leaders encounter fierce opposition to redirecting funding streams that reliably flow to incumbent providers — even if those providers have failed to improve outcomes . . . In the face of such challenges, policy makers can use PFS financing as part of a sequential strategy to achieve reform. Starting projects with PFS financing, then scaling services that work with direct government funding . . . allows policy makers to embrace reform while mitigating the risks of innovation.”
We see the Denver project as a textbook example of what a well-designed, well-implemented PFS project can achieve. The goal should be to build a case for a broader scale, without the complexities and transaction costs currently inherent to even the best-managed PFS financing, that can serve not 363 program participants, but 363,000 — more than half of the individuals experiencing homelessness in a given night in the U.S — or more.
5. Scale quality supportive housing, using a “Housing First” approach, with robust and person-centered supportive services to solve this problem everywhere. Taken together with the other projects highlighted above, the Denver project outcomes make a strong case for scaling this approach more widely.
A growing body of rigorous studies have built a case for the efficacy of affordable housing paired with intensive supportive services as a solution to the ongoing crisis of housing affordability and chronic homelessness. This project’s replication of several strikingly consistent positive outcomes offers a near-definitive verdict on the potency and effectiveness of this model. Far too many Americans continue to suffer the devastating effects of chronic homelessness, even though a repeatedly proven model can put a roof over their heads and avoid the vast downstream costs of inaction. It’s fiscally reckless and morally indefensible.
In the short term, as state and local governments consider how to deploy an avalanche of American Rescue Plan Act (ARPA) federal stimulus funds, the release of this evaluation could not come at a better time. Policymakers and local elected officials should act now to direct ARPA funds toward scaling this approach, which will yield massive downstream benefits long after the ARPA funds are depleted. Denver provided communities a blueprint for implementation and replication, and metrics that can be used to manage performance and make sure these projects are on track to succeed.
In the longer term, Congress and the Biden-Harris Administration should reform, strengthen and expand federal funding streams (especially Medicaid) that could scale and sustain these services. The Denver project underscores how important stable housing is to the health of families and communities everywhere. Federal decision-makers now have an opportunity to seize on this blockbuster evaluation to put safe, affordable housing aligned with services in reach for every American who needs it, transforming this country into a wiser, more compassionate, and more equitable place where no person is forced to sleep on our streets.
Roger Low is Policy Director at America Forward, @RogerLow07
Stephanie Mercier is Director of Impact Investment at Corporation for Supportive Housing, @SHMercier