Results from the first generation of social impact bonds (also known as pay for success deals) are starting to come in. Last week, the field learned the results of the evaluation of the first social impact bond transaction in the United States.

The investment by Goldman Sachs and Bloomberg Philanthropies in a program to serve young men at the Rikers Island jail—the main processing and housing facility in New York City—did not show a sufficiently positive effect to warrant the continuation of this intervention. The program will terminate at the end of August.

The results seem to be a defeat for this approach. We see them as a partial victory for this disruptive innovation. Here’s why:

• The social impact bond transaction worked exactly as intended.

  The goal of pay for success deals is to encourage private investors to fund proven social programs by providing upfront support to programs that seek to improve long-term outcomes for those in need. If the programs are successful, governments pay the investors back; if they are not, then the investors absorb the cost, and governments pay nothing.

  In this case, the program has been found to have no impact –program participants did not spend fewer days in jail than a comparison group – and so the program will end. This alone represents a significant departure from status quo. Under typical government contracts, the program likely would have continued.

  Further, the rigorous process of constructing the social impact bond required a new type of program management and pushed New York City government toward more outcomes-based decisionmaking. The process forced public leaders to have conversations with parties inside and outside of government and likely helped create new capacity and infrastructure that can continue to benefit citizens, even though this particular intervention did not reach its goals.

  All of these changes will help policymakers as they start to develop the next important innovation—at no loss to taxpayers.

• The Rikers program shows why monitoring outcomes is essential to understanding the real-world impact of our social investments.

  Though the Rikers program was grounded in Moral Reconation Therapy (MRT), an evidence-based approach with a fairly strong track record of affecting recidivism outcomes for older youth, the program itself hadn’t been tested before. When you innovate to apply evidence-based practices in a new program, sometimes you create something that is effective and efficient. Sometimes it does not work.

  It is better to know something is not working; that enables you to make a decision either to change program delivery or end the program altogether. That in turn saves taxpayer dollars for more promising approaches. The only way to make such informed decisions is through rigorous testing of our social program investments.

• Implementation and context matters. A lot.

  Rikers Island is a remarkably disruptive—and potentially damaging—setting to rehabilitate juvenile delinquents. The challenge of serving a youth population in a huge jail setting like Rikers is enormous.

  According to the Vera Institute, 87 percent of adolescents who were at Rikers for more than seven days participated in at least one program session. Of those youth, “44 percent reached a programmatic milestone found in prior studies to be associated with positive outcomes.” While Vera did not report the completion rate of adolescents in the program, MRT is typically a three-month intervention at minimum.

  If juveniles at Rikers were not receiving the full “dose” of the intervention or the intervention was not delivered in the right way, of course they would be less likely to achieve intended outcomes. Recent lessons from the burgeoning field of implementation science have taught us that the setting of an implementation, and the extent to which it mirrors best practice, is critical.

In sum, disruptive innovation is inherently combustible. There will be big wins and big losses on the way to making government more evidence based, more creative, and more receptive to new ideas that better align our spending with the outcomes we want for our society.

The close of the Rikers Island transaction may be a first step along a long road toward more effective innovation. At the end of the road is better, more effective government.