Many mission-driven organizations are exploring AI solutions to help stretch shrinking budgets and increase their impact. Deploying AI tools responsibly requires rigorously evaluating a tool’s effectiveness, consistency, fairness, and security before and during deployment.

Yet many public-serving organizations lack the in-house technical expertise and resources to assess AI tools. This may lead some to not adopt AI, leaving potential benefits on the table. Others might turn to vendor-reported evaluations that don’t reflect the organization’s policy context, data inputs, and target users.

The AI Studio helps fill this capacity gap. With a team of technologists, researchers, and policy experts, we conduct rigorous evaluations of AI tools, grounded in the relevant policy context, to give leaders actionable information they can trust.

Here, we highlight three examples of how we’ve designed evaluations to assess AI applications relevant to the public sector. These include investing in improving the AI readiness of public data, evaluating existing tools’ performance for policy applications, and vetting custom-built AI tools.

How can we make public data more AI ready?

As popular large language models (LLMs) like ChatGPT, Claude, and Gemini quickly become the new “front door” to information, public data providers like the Urban Institute want to ensure LLMs accurately answer questions grounded in their data.

To explore this, we tested whether the following two investments could measurably improve how LLMs answer questions using our Education Data Portal and Education to Workforce Data Tool:

• Developing model context protocol (MCP) servers for both tools, giving AI systems a direct, structured way to query their data
• Expanding the tools’ metadata and documentation layers, such as adding more-detailed definitions, guidance on appropriate comparisons, and explicit notes on known limitations

First, we ran a baseline test. Our data scientists partnered with Urban education and workforce experts to curate 100 questions written from the perspectives of our target user groups (parents, advocates, researchers, Capitol Hill staffers, career advisors, and others). At baseline, we found that 84 percent of LLM responses had accuracy issues, with more than a third having critical accuracy issues. Notably, inaccurate responses were presented with the same authoritative tone and confidence as accurate ones. This made errors very difficult to identify without the benefit of Urban’s subject matter experts.

After building the MCP and making the documentation improvements, we then reran our original 100-question test on one model. Response quality improved substantially across every dimension. Responses with no accuracy issues rose from 16 to 61 percent, and those with critical accuracy issues fell from 35 to 7 percent.

We also found that citations became traceable, and LLMs were more forthcoming about what they did not know. Models more reliably located and retrieved data, with some trade-offs, as the responses required more tokens and time to produce.

Though we are excited by these results, our evaluation doesn’t stop here. Both MCP standards and LLMs are continuously changing, and we will have to continue investing in the AI readiness of our data systems to keep up. Developing this evaluation framework will allow us to continuously test AI tools’ performance and target our resources toward the AI-readiness investments that have the biggest payoff in producing accurate, traceable, and trustworthy LLM responses.

Our experience designing evaluations can help you weigh which data AI-readiness investments make the most sense for your organization. We work with you to ensure AI tools—whether built in-house or externally—use your data accurately and create context-specific evaluation frameworks to monitor their performance over time.

How well can LLMs provide accurate information on a city’s zoning code?

Zoning documents are notoriously difficult for local government staff, residents, and developers to understand. Generative AI tools could make them easier to navigate, but can they return accurate information based on existing zoning code documents?

To test this, we ran a benchmarking exercise to evaluate how well the models at that time—including lightweight (GPT-5 mini), open-weight (Llama 3.2 1B Instruct and Mistral Pixtral Large 25.02), and state-of-the-art (ChatGPT 5.1) models —could answer questions about Minneapolis’s 467-page zoning code.

Working with a zoning expert familiar with Minneapolis’s regulations, we developed a set of prompts that represented common questions professional developers and nonexpert residents would have about the zoning code and a ground-truth response for each prompt. We then submitted the set of questions to each LLM and graded the responses they provided against the ground-truth answers for accuracy, relevance, consistency, ability to convey uncertainty, and use of the right pieces of the zoning code to answer questions.

We found that the LLMs performed worse for the questions asked by our nonexpert resident persona. Unless the prompt provided the model specific procedural knowledge, the responses were often generic, difficult to understand, and sometimes incorrect.

For example, when asked generally about a “zoning requirement checklist for multi-family home,” the Mistral model provided guidance like, “Ensure that yard dimensions and all other requirements for the district are met, excluding lot area or lot width,” without specifying what those requirements were.

This example illustrates the importance of bringing a user-centered approach and domain knowledge into evaluation. If we hadn’t worked with a subject matter expert to craft test prompts from a range of personas, our assessment of LLM performance at answering zoning questions may not have been as accurate or robust.

When you partner with the AI Studio, we work with you to conduct evaluations grounded in real-world user personas and ensure your tools effectively serve your target users.

Designing a repeatable evaluation framework for AI tools with experts in the loop

In 2026, we built an AI-ready knowledge base for Urban’s vast library of upward mobility data, tools, and technical assistance. We then created a prototype of a navigator tool that state and local leaders could use to query our library and retrieve answers about upward mobility data, potential policy solutions, and case studies.

We wanted to ensure the AI navigator tool would provide high-quality answers across the range of potential user questions. To assess this, we first needed help from domain experts.

We created an evaluation framework for the tools’ answers with six criteria (accuracy, completeness, relevancy, uncertainty, refusal, and bias) and specified how our experts should evaluate the tool’s performance on each criterion.

Next, we asked our experts to generate 32 realistic prompts. For each, they set requirements for a correct answer and identified relevant sources in our knowledge base. We ran the prompts and had the same experts compare the AI-generated responses against this ground-truth information.

We conducted multiple rounds of evaluations, identified improvement areas, and iterated on the navigator tool accordingly until it passed the litmus test for usability.

We could not have identified or implemented these improvements without the expertise of our human reviewers. This manual, time-intensive evaluation process was worth the trade-offs, as it helped ensure the AI navigator tool provided responses that were both accurate and aligned with the Upward Mobility Framework’s goal to increase mobility for all people, especially those historically excluded from prosperity.

Our technology and policy experts will work with your in-house experts to design and implement AI evaluations that accurately measure performance of even the most complex AI applications.