Maximizing the value of AI experimentation
Image by Cath Virginia for Bloomberg Philanthropies
Listen to This Article
Much of the conversation about artificial intelligence in government in recent years has focused on assistants that help civil servants draft documents, summarize information, and automate other forms of routine knowledge work. And those productivity gains are meaningful. Now, a growing number of local governments are also experimenting with AI tools designed around specific city problems: identifying potholes, mapping tree canopy, reducing traffic congestion, and generally helping leaders make more informed decisions about the physical world. In doing so, some of them are finding that when they approach these projects strategically, AI experimentation can help them not only learn whether a new tool works, but also identify the need for larger changes around underlying services.
That kind of learning is not unique to AI. Any serious data or technology initiative can reveal gaps in how a city understands a problem or organizes itself to address it. But AI projects can make those gaps especially visible because they require cities to define problems even more clearly, identify exactly which data matters most, make tacit knowledge more visible, and decide how the resulting outputs will shape real decisions.
The most valuable AI experiments, then, will be designed not just to build a new tool, but to help city leaders learn about the data, the people, and the processes behind key services—and how they can get better.
Here’s what that looks like in practice.
Getting clearer about the reality of frontline work.
One of the first things cities are learning as they develop AI tools geared at individual priorities is whether leaders and the staff delivering those services share the same understanding of the problem. Moving forward, that suggests cities will benefit from approaching AI experimentation as a way to get in sync with frontline teams.
For example, in Luján de Cuyo, Argentina, Mayor Esteban Allasino—whose team participated in the Bloomberg Philanthropies City Data Alliance—was determined to improve how the city repaired potholes. And the mayor and his team believed that likely meant tapping new technologies. But they also recognized early on that the issue was not only a technical one: Street maintenance staff were telling city leaders that their daily work was not fully understood. These civil servants were already taking steps to respond to the pothole problem, but the extent of their efforts, such as repairs of potholes that were not documented in citizen complaints, were not always visible to the mayor’s office.
As the mayor’s chief of staff, Matías Meric, recalls, “The team told us, ‘We do a lot of work every single day that you don't know about and you're not measuring.’”
So the city approached AI experimentation around potholes as a way both to improve services and to address that disconnect with teams in the field. What has emerged is a computer-vision tool that uses low-cost cameras mounted on vehicles to map potholes systematically, whether they’ve been reported or not. Now, by giving leaders a better sense of the scale of the pothole problem across the city, the project is helping the mayor’s office make the case to the city council for new funding to help reinforce street teams that might be stretched too thin.
Those teams now “feel that they have more support and that we have a shared understanding of the complexity of the job they’re doing,” Meric says.
The lesson here isn’t just that AI experiments produce impact. It’s that cities can use those experiments to measure work that had been difficult for leaders to see—and to engage more closely with the people responsible for delivering services every day.
Revealing the data cities actually need.
Rather than assuming they need perfect data before they can experiment with AI, cities can also look to the development of these tools as a strategic way to help determine where their data capacity is strong and where it needs to improve.
Take Guatemala City, another City Data Alliance participant. There, local leaders are fine-tuning an AI tool to support climate-focused tree planting by helping determine where new trees would create the most shade and best cool neighborhoods. But this experiment hasn’t been carried out in isolation; when city leaders began work on the tool, they did so in the context of a long-term effort under Mayor Ricardo Quiñónez Lemus to become a more data-driven organization. And throughout the process, they have constantly been on the lookout for ways to get there.
It didn’t take long before the city’s AI project surfaced a basic problem: The team did not have sufficient data to understand their existing tree canopy cover. “We had only a couple of people doing tree inventory for the whole city,” explains Carlos Alejandro Soberanis Toledo, an official in the mayor’s office. For a city trying to use tree planting to reduce heat and improve shade, that missing information mattered. Without it, leaders could not confidently identify where new trees would have the greatest impact.
In response, the city trained additional staff to help implement a more complete census of local tree species. “This AI project reinforced the fact that with the data that we have, we can do something good, but if we have better data and more data, we can do something better,” Soberanis Toledo adds.
The lesson for cities elsewhere? Approach AI experiments not just as a way to capitalize on strong data foundations, but as a way to build those foundations up in the first place.
Building expertise around the problem.
Cities are also finding that AI projects can help them bring together expertise that is often spread across different parts of government, and sometimes beyond it.
In Guatemala City, the mayor’s office has used the AI project to connect two departments—geographic information services and environmental services—that each held data and expertise the other needed. And in Luján de Cuyo, the mayor’s office and street maintenance team have used the pothole-detection project to build stronger relationships with academia and the local tech community, which helped accelerate the effort.
That kind of collaboration is not incidental. AI tools that are embedded in city service delivery require more than technical capacity. They require people who understand the service or place being modeled, people who understand the relevant data, and people who can help gauge whether the tool’s outputs are useful in practice. They also require leaders to be clear about what decision the tool is meant to support and who will be responsible for acting on what it reveals.
“Cities fine-tuning these tools are realizing they need more knowledge, not just to develop them but to interpret their results,” explains Coca Rivas, a design manager at Projects By If, a Bloomberg Philanthropies partner that supports leaders in the City Data Alliance. “And so they’re starting to link up with universities and with various other experts who they had access to but may never have asked for support in the past.”
The larger lesson is not that every city problem needs an AI solution. It is that cities will get more from AI when they treat it as a way to learn about a real service challenge, not simply as a tool to deploy. Designed well, an AI project can help leaders ask the questions that should sit at the center of any government innovation effort: Do we understand the problem? Are we listening to the people closest to the work? Do we have the data we need? Are the right departments and experts involved? And are we prepared to act on what we learn?
Used this way, AI experimentation may well produce a useful tool. But either way, it can give city leaders something just as valuable: a clearer view of what needs to change in order to deliver better results for residents.
