The project, led by Diag Davenport, an assistant professor of technology policy, governance and society at Berkeley who holds appointments in the Goldman School of Public Policy and the School of Information, has been described by its creator as a fundamentally new instrument for civic knowledge. “We’ve built a new kind of telescope — one pointed at governance itself,” Davenport said. “It could be a fundamental shift in the way people interact with the rules that govern their daily lives.” While online repositories have cataloged state and federal statutes for years, no comparable resource has ever existed specifically for local laws, from Alameda, California, to Zephyrhills, Florida. The database, published online in June, opens the door for others to generate custom databases and local chatbots capable of quickly comparing laws on topics ranging from home construction to dog leash requirements.

The origins of the project trace back roughly six years, when Davenport was studying computer algorithms, systemic bias and the criminal legal system. At some point, he wondered whether he could invert that research focus and instead study bias embedded in the actual text of the law itself. To do that rigorously, he needed to examine local laws across the country and use statistics to measure how they differed between places with high levels of historic discrimination and places without it. That ambition collided immediately with a practical obstacle: no database of all local laws existed, and the more he looked, the more he realized the material was scattered across a mishmash of proprietary company databases and convoluted local government websites.

The scale of the fragmentation alone was daunting. The United States contains more than 3,000 counties and another 6,000 or so local jurisdictions, each with the power to make its own laws and each storing those laws in its own way. “Of course, nothing’s ever as simple as you want it to be,” Davenport said. “Once you realize how fragmented it all is, it’s easy to understand why no one’s done the work.” The idea simmered for years, but it took on new life about a year ago, when recent technological advances finally allowed the team to stack one AI tool on top of another to do the heavy lifting that had previously been impossible.

The first stage was collection. Davenport and his colleague Denis Peskoff, a postdoctoral scholar at Berkeley, needed to gather municipal and county laws from thousands of government and third-party websites. They consulted with lawyers as they developed the collection process and deliberately designed it to meet the technical requirements imposed by the sites hosting the documents. They also worked with Joe Barrow, an AI researcher specializing in documents and datasets, and Christopher Vu, a Berkeley undergraduate studying computer and data science. That effort produced an archive of nearly 10,000 often unwieldy documents — roughly 7 million pages in total, many stored as blurry, poorly structured or otherwise inaccessible PDFs.

Turning that archive into usable data was the larger technical challenge, and it is where the project’s AI pipeline proved decisive. The team deployed a vision-language optical character recognition model called LightOnOCR to extract both text and structure from the documents. Processing millions of pages required a massive amount of computing power, but the result was a relatively standardized body of machine-readable text that is far easier, faster and less costly to search, analyze and feed into modern AI systems than the original PDFs ever were. From there, the researchers used models from OpenAI to tag and organize samples of the laws, distilling that labeling work into categories that could then be applied across the full corpus, transforming a chaotic pile of scanned documents into a structured, analyzable dataset.

The resulting database contains millions of unique ordinances covering every state in the union, and it enables something that was previously extremely difficult: searching across local laws at national scale to identify patterns in how communities regulate everything from housing and public space to business activity and everyday conduct. “I think it was actually impossible to do this work until six or 12 months ago,” Davenport said. “As far as I’m aware, we’ve done this at a larger scale than anyone else.” The project’s scientific credibility received a significant boost last week, when a paper describing the work was accepted to the Conference on Neural Information Processing Systems, one of the premier venues in machine learning and AI, where the team will present its findings in December.

The practical implications extend well beyond academia. Developers, for instance, have long faced soaring project costs driven partly by the difficulty of deciphering dense, inconsistent local regulations. Computer scientists and other technically minded users can now train chatbots on specific slices of the repository — one could, for example, be trained on Bay Area building codes and quickly report the different requirements for building an apartment complex in Berkeley compared with neighboring El Cerrito. That could allow developers to build more accurate cost estimates into project plans, or help a homeowner navigate the rulebook for constructing a backyard accessory dwelling unit. Davenport emphasized that any forthcoming tools built on the data must be free and publicly accessible, a condition he considers critical to the project’s mission.

Journalists, researchers and policymakers stand to benefit as well, gaining streamlined access to compare laws and saving countless hours that would otherwise be spent wrestling with hard-to-navigate documents. Users can compare regional differences in regulation, study how difficult laws are to understand in different areas, and even probe regional differences in enforcement. Some members of the public have already created searchable interfaces based on the team’s published work, an early sign of the ecosystem the researchers hope will grow around the dataset. In effect, the database converts the country’s fragmented legal patchwork into a natural laboratory for policy analysis.

“Effectively, what we have are thousands of experiments about how to regulate housing, sidewalks and helmets,” Davenport said. “The real promise here is making local government legible. Once people can actually see and compare the rules that shape everyday life, we can ask much better questions about which ones work, who they work for and what we might want to do differently.” For a nation whose daily governance has always been local, diffuse and largely opaque, the arrival of a single, searchable corpus of millions of ordinances may mark the moment when the rules that shape ordinary life finally come into focus.

Subject of Research: An AI-built open-access corpus of United States local ordinances for policy and legal research

Article Title: To decipher local laws, researchers used AI to build a database of millions of them

Article References: To decipher local laws, researchers used AI to build a database of millions of them. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: local laws, ordinances, artificial intelligence, machine learning, optical character recognition, open-access database, public policy, UC Berkeley, governance, natural language processing, municipal law, NeurIPS