Policymakers have expressed increased interest in program-level higher education accountability measures as a supplement to, or in place of, institution-level metrics. But it is unclear what these measures should look like. In this report, we assess the ways program-level data could be developed to facilitate federal accountability.

Evidence shows that what students study matters as much as, if not more than, where they study. Program-level measures can provide important data on student outcomes, which will allow policymakers to hold institutions and programs that receive federal funding accountable. But program-level measures are difficult to develop. Program definitions must be large enough to be assessed but distinct enough to be meaningful, and any measure should be insulated against potential gaming.

Using national data, we assess how pooling program data by year, as well as by subject (using Classification of Instructional Programs, or CIP, codes), can yield sufficient program-level metrics. In analyzing approaches to this question, we work from three principles: program definitions must include all eligible students, maintain student privacy, and provide a meaningful metric (i.e., not combine dissimilar programs or credentials). Using these principles to guide our analyses, we arrive at the following recommendations:

• Set a minimum program size. A size of 30 is typically cited as the goal for a measure that best represents the performance of a population, but smaller group sizes could be used, especially if the accountability criteria depend on a program consecutively failing to meet a completion or earnings threshold.
• Pool two years of data. One way to increase the size of cohorts measured for smaller programs is to pool years of data. We find that the share of programs and students included increases substantially when an additional year of data is added. Although adding third and fourth years includes more programs, it is only to a smaller degree, making two years of data the most effective option.
• Combine programs within credential levels. This approach ensures that all student outcomes are included in an accountability measure. The narrowest definition of a program is at the six-digit CIP code level. Though two- and four-digit CIP codes include more students, relying on these broader program categories may hide certain programs’ poor outcomes. We suggest rolling up programs within a credential level (e.g., combining small six-digit CIP codes into their broader four- or two-digit CIP code) until a given size is achieved.
• Monitor accountability measures over time. Policymakers should monitor their selected measures to ensure that institutions do not evade accountability.

Program-level accountability measures can produce targeted improvements in higher education outcomes and transparency. Through these recommendations, and with steps to protect student privacy, a program-level accountability metric is possible.