In a baseline evaluation published in May 2026, we found that large language models (LLMs) often gave low-quality responses to questions about education and workforce pathway data. In response, we made targeted enhancements to improve the AI readiness of two Urban Institute data tools that centralize public data on these topics: the Education Data Portal and the Education-to-Workforce Framework Data Tool. We built Model Context Protocol (MCP) servers, which give models a structured way to find and query our data, and we expanded the documentation that explains how the data should be used. When we asked the same 100 questions again, response quality improved substantially.

Why This Matters

As more people turn to AI tools to find and interpret information, making public data usable by these tools is increasingly important. Without reliable access to authoritative data and documentation, a model can produce a confident answer that is wrong and difficult to check. Our results show that data providers can substantially improve how AI systems use their data. Our experience offers lessons for federal statistical agencies, state longitudinal data systems, city open data portals, and other organizations that maintain structured data.

What We Found

Compared with the same model’s baseline responses, we found the following:

• Response quality improved substantially across every dimension. Responses with no accuracy issues rose from 16 to 61 percent, and those with critical accuracy issues fell from 35 to 7 percent. Across every dimension, at least three-quarters of responses had no or only minor issues.
• Citations became traceable, and responses flagged limitations more consistently. Responses with no issues on references rose from 11 to 84 percent. Those with no issues on confidence, meaning they appropriately flagged uncertainty and limitations, rose from 26 to 71 percent. Specific citations also made the remaining errors easier for our evaluators to identify.
• Remaining errors were concentrated in judgment rather than retrieval. Models were usually able to locate and retrieve data reliably. Errors were most common when an answer required more than a lookup, such as choosing between two plausible sources, using the right denominator for a rate, or combining data that our tools do not link automatically.
• Responses required more tokens and time to produce. On average, a response cost roughly 12 times as much to generate and took substantially longer, reflecting the added work of retrieving and processing data.

For organizations working to make their data more AI-ready, our experience points to several lessons: test how AI tools handle the questions your users actually ask; build on a strong foundation of standardized, well-documented data; involve subject-matter expertise at every stage; and design for the range of AI tools people use to reach data.

How We Did It

Our baseline evaluation asked three LLMs 100 questions on topics that our tools are well-suited to answer. The questions were written from the perspective of users such as parents, Hill staffers, and career advisors. In August 2026, we sent the same prompts to Claude Sonnet 4.5, the same model version as the baseline, with our MCP servers connected. Subject-matter experts scored each response on six dimensions using a four-point scale from no issues to critical issues.