Sometimes following the data, wherever they may lead, requires us to acknowledge mistakes and fix them. Our January 22, 2018, Urban Wire post (PDF) on student debt contained inaccurate results because of such mistakes. We retracted the post and wrote a new analysis based on corrected numbers. A detailed explanation of which numbers changed can be found here (PDF).

What happened

Our analysis of the 2016 Survey of Consumer Finances (SCF), based on code I had written for an earlier project, found that nearly half of student debt (49 percent) was held by households in the top income quartile. After a friend at the Brookings Institution, Adam Looney, informed us of a potential error, we double-checked our code and found we had indeed erred. Our corrected analysis finds that a smaller share of debt (34 percent) is held by the top quartile.

Although the overall trends found in the analysis still hold—households in the top quartile do hold a disproportionate amount of debt, and average debt increases with income—the concentration of debt among higher-income families is not as extreme as we thought.

How it happened

We wrote Stata code to analyze these data and had a researcher who was not an author of the piece double-check the code. Usually, that process catches any errors. But in this case, we made an incorrect decision about how to use weights with the SCF.

We applied analytic weights (aweights) throughout the analysis, when we should have been using probability weights (pweights). In most cases, aweights and pweights produce the same or very similar results, but when calculating sums using Stata’s collapse command—as we did to find the distribution of debt by quartile—the difference can be significant. After correcting the weighting, we noticed that the dollar amounts in the post were reported in 2010 dollars (for consistency with other survey years in a prior analysis). We updated all dollar figures in the post to 2016 dollars, as reported in the survey.

Moving forward

We have taken steps to alert those who relied on the data. We have also double-checked all our other work with the SCF—including last week’s post about Elizabeth Warren’s student loan forgiveness plan—and have verified that it’s correct. Going forward, we are reviewing and revising our internal quality assurance processes to fulfill our commitment to providing high-quality analyses that readers can trust.