Urban Wire How Third-Party Data Can Strengthen the Strained Federal Data Landscape
Judah Axelrod, Jeremy Seeman, Sonia Torres Rodríguez
Display Date

An image of a data set on a computer screen.

The federal data landscape has experienced major disruptions this year. Many federal datasets and data products have been removed from their respective websites, leading thousands of data users to document and preserve public information, including America’s Data Index, the Data Rescue Project, and the Urban Institute.

Some federal data collection efforts will no longer be updated and will be too difficult to replicate elsewhere. In the absence of data, key players cannot serve affected communities. Consider the Billion-Dollar Weather and Climate Disasters database, which the National Oceanic and Atmospheric Administration announced will no longer be updated. Without it, we would not see the economic losses from disasters such as the 2025 Los Angeles wildfires.

On top of collection issues, the 13 federal statistical agencies that oversee the quality control and access to all of the major federal data sources used by public and private sector actors alike, are also experiencing major leadership and budget changes as well as mandated alterations to data sources they collect. In addition, from 2009 to 2024, most of these agencies had lost more than 14 percent of their purchasing power from funding cuts, despite growing responsibilities, while US residents are responding less frequently to government surveys and report less trust in government entities (PDF).

Third-party data offer a promising opportunity to help fill these federal gaps—but careful vetting and additional philanthropic investment is needed to address the limitations of nonfederal data sources.

The benefits and trade-offs of third-party data

Third-party data—collected by private firms, nonprofits, and community-based organizations—are underleveraged assets for public-interest use and could bridge federal data gaps. Urban Institute researchers, among those from other organizations, have studied the private data landscape to lift promising benefits and best practices when turning to these alternative sources. Third-party data can offer unique advantages:

  • Third-party data on how people use products or systems can offer high-frequency, granular insights at scale. This can lead to data that are more actionable and accurate than leading national surveys.
  • Third-party data can help us answer questions currently unanswerable with public data. For example, large digital records created by businesses (“digital traces”) offer insights into human behavior and activity with no current public-sector alternative.

On the other hand, third-party data can come with their own set of caveats and risks:

  • Repurposing third-party data carries potential liabilities. Private or vendor data are often originally intended to support commercial or administrative use cases, not public policy decisions, and therefore may exhibit poor representativeness of some populations or areas, contain measurement error in certain outcomes, undergo methodological changes over time, lack adequate documentation, or raise concerns over individual privacy.
  • Most third-party data are not easily accessible. Most privately held third-party data aren’t publicly available, though there are some notable exceptions (which can be subject to caveats). For example, Zillow provided free access to rich housing data through its ZTRAX (Zillow Transaction and Assessment Dataset) program until October 2023, and Epic’s Cosmos dataset provides deidentified, aggregate electronic health records to its participating organizations.

Put simply, most third-party data lack the vetting processes and availability of gold-standard federal survey and administrative data sources. Though they can offer immense utility, avoiding a “Wild West” of data providers who offer potentially flawed or costly alternatives requires strategic investment to ensure the data’s reliability and accessibility.

Third-party data in practice

One example of third-party data collected by organizations outside of government or the private sector is on renter needs and rental patterns, a critical issue amid today’s housing affordability crisis.

To address these gaps, nonprofits such as the National Low Income Housing Coalition are rolling out their own nationally representative survey of renters, which covers a broader range of questions about the rental experience than the American Community Survey. At the local level, Community Legal Services and the Housing Initiative at Penn administered a survey (PDF) to thousands of Philadelphia renters about their housing stability, costs, and experiences. Community-based and tenants’ rights organizations are also sourcing these data directly from members: The Community Alliance of Tenants has an annual survey open to Oregon renters to ask about their rental housing type, costs, assistance received, and other key experiences.

These third-party data sources are more geographically granular, timelier, and richer than federal data.

To maximize the benefits of third-party data, philanthropic investment is critical

Third-party data have long served as a complement to federal data but have mostly lacked the scale, accessibility, and trustworthiness of federal data. For third-party data to emerge as dependable assets in this turbulent time for the federal data landscape, philanthropic leaders can strategically invest in the infrastructure, governance, and grassroots capacity needed for evidence-based policymaking. Here are three steps they can take:

  1. Build shared infrastructure for ensuring data rigor and trustworthiness. Funders can support the development of an independent data clearinghouse that sits outside the federal government to host, document, and standardize third-party data. This includes investments in metadata, data documentation, and capabilities for tracking collection processes, assessing quality and known limitations, and determining appropriate data uses. The United Nations’ Citizen Data Collaborative highlights one possible framework (PDF) that involves direct citizen collaboration in the process.
  2. Invest in third-party data governance models. Third-party data need to be aggregated, distributed, and often harmonized with public data sources to unlock insights at greater scale, but doing so raises risks around liability, data sovereignty, and privacy. Philanthropic support is needed to fund templates for data sharing between sectors, as well as case studies of these agreements that align incentives across the public-, nonprofit, and private-sector stakeholders involved. 

    Promising case studies already exist: The Freedom Fund aggregates outcome metrics (e.g., the number of people liberated from modern slavery or the number of children enrolled in education programs thanks to its investments) across its frontline grantees. Opportunity Insights pooled anonymized transaction and payroll data from several private companies to track the economic impacts of the COVID-19 pandemic. But without the governance structures to enable such data sharing models, the analysis stalls even if the questions are straightforward to answer.
  3. Scale community-led data collection efforts. Investing in grassroots data collection can capture measures and insights embedded with local context not available in any federal datasets. The Neighborhood Vitality Index, a survey designed by and created for Detroit residents, provides data and tools to track neighborhood-level investment while fostering community ownership. The effort led to meaningful data-driven advances like the creation of Neighborhood Zones, new units of geography within Detroit that are large enough to be manageable for survey administration but small enough to provide meaningful, granular insights to community leaders.
Body

Let’s build a future where everyone, everywhere has the opportunity and power to thrive

Urban is more determined than ever to partner with changemakers to unlock opportunities that give people across the country a fair shot at reaching their fullest potential. Invest in Urban to power this type of work.

DONATE

Research and Evidence Research to Action
Expertise Data Governance and Privacy
Tags Data analysis Data collection Data science Quantitative data analysis Research methods and data analytics Research technology National Neighborhood Indicators Partnership (NNIP) Data resilience