The scholars explore these topics in “How AI Sees the City: Urban Visual Intelligence,” published this month by Routledge. The authors are Duarte, a principal research scientist and associate director of the MIT Senseable City Lab; Mazzarello, a research scientist and lead of MIT Senseable City Lab global initiatives; Carlo Ratti, a professor of the practice and founder and director of the MIT Senseable City Lab; and Fan Zhang, an assistant professor at the Institute of Remote Sensing and GIS at Peking University.
“Great urbanists such as Kevin Lynch and Willian H. Whyte showed us the extraordinary value of ‘looking’ at the city,” Ratti says, referring to two prominent thinkers about city dynamics whose work is described in the book. “Today, visual AI gives us new ways to build on that tradition — allowing us to observe cities at a scale and with a level of detail that was previously impossible.”
New tool, long tradition
“How AI Sees the City” stems from the work of the MIT Senseable City Lab, founded in 2004, which uses data to better understand urban dynamics. As the authors discuss in the book, there is a long history of visual representations that shape the way we think about cities, from Romans building marble maps to the introduction of photography — which produced influential urban images about things like Hausmann’s reshaping of Paris or the crowding of tenements in New York City’s Lower East Side during the 19th century.
More recently, some scholars have used visual studies to better understand city form, including Lynch, a former MIT professor whose 1960 book, “The Image of the City,” influenced many scholars. Whyte, a sociologist famous for his book “The Organization Man,” then became an urbanist closely examining public spaces.
By explicitly placing AI in a continuum with these visual urban studies, the authors are making a point: Powerful as it might be, we can still think of AI primarily as a tool serving human purposes, as we seek to design and refine urban form.
“Kevin Lynch at MIT was only using paper and pen,” Duarte says. “We can now scale up what he was doing, with visual AI, while also looking at many different dimensions of cities.”
There are extensive possibilities for applying visual AI to urban planning, ranging from emissions to traffic flow, safety, better imagery of street-level activity and sidewalks, and much more. The book also examines, for instance, urban greenery. While satellite imagery can show us how much tree cover and green spaces cities have, near-ubiquitous images from phones and other sources can also reveal to what extent people glimpse greenery in everyday life, a factor in reported wellness.
“The real promise of visual AI is not simply that computers can look at millions of images,” Zhang says. “It is that we can connect what is visible in those images — streets, buildings, greenery, traffic, public space — with larger questions about how cities function and how people experience them.”
Better image recognition by AI even extends to urban interiors. By using images from 400,000 AirBnB listings across the world, one recent Senseable City study shows that, contrary to some claims, interior design styles are not becoming globally more homogeneous, but reflect significant geographic differences.
“No matter what it is, we can learn from what we can see and then use it as urban designers, planners, policymakers, and citizens,” Mazzarello says. “It can be our eyes, or cameras with computers, but in the end, it’s the same methodology, and now we are trying to optimize the ways we can use these tools.”
Promise and pitfalls
If the promise of visual AI for urban studies is vast, the pitfalls are concerning. In “How AI Sees the City,” the authors outline multiple potential problems with the technology, including the intrusiveness of widespread visual surveillance and the potential for bias being reinforced through AI systems.
The installation of ubiquitous cameras can quickly raise concerns about surveillance. London, an early adopter of CCTV, has about 210 cameras per square mile. But eight of the world’s 10 most camera-heavy cities are in China; Shanghai has over 5,000 cameras per square mile. Such surveillance practices have raised controversy in other parts of the world, with debates over the uses of traffic cameras bubbling up in the U.S. this year as well.
In evaluating the potential safety gains from intensive video recording, the authors write, “the benefits must be weighed against the significant erosion of personal freedom and the potential for abuse inherent in a system of constant monitoring.”
Meanwhile, AI systems can reinforce social biases as well, leading to the production of data that reinforce prior perceptions as much as underlying realities — about people, neighborhoods, and whole cities. If AI models are trained on majority population groups, they may not evaluate minority groups the same way.
“We need to teach AI to see, and depending on how you teach it, it will see what is embedded in the culture,” Duarte says. “AI is not neutral.”
Still, as Mazzarello adds, “our eyes are not neutral, either. Every tool has to be guided in the right way, and trained in the best way.”
Other scholars have praised “How AI Sees the City.” Michael Batty of University College London has called it a “fascinating book” that “shows how we are beginning to interpret the world of urban design, suggesting ways in which we might improve design using urban analytics, AI and large language models.”
Ultimately, though the authors think there is great value in deploying visual AI to learn more about our cities, how they function, and how they might be improved. With caution and independent thinking, progress is possible. Or, as they conclude in the book, “We should explore this wisely, critically, and creatively.”
