Neural Code Search Brings Semantic Discovery to Wikipedia’s 2,500-Repository Empire

Behind every Wikipedia article lies a vast and sprawling software empire. The MediaWiki ecosystem, the technical backbone of Wikipedia and its sister projects, has grown over two decades into a decentralised collection of more than 2,500 code repositories scattered across forges such as Gerrit, GitHub, and GitLab. Now, a new open-source tool called MediaWiki Code2Code Search promises to make that hidden universe navigable, using neural embeddings to find code by meaning rather than by name, and its developers report that it decisively outperforms the keyword-based search the Wikimedia community has relied on since 2019.

The problem the tool addresses is one that any developer in a large codebase will recognise. Wikimedia’s existing Codesearch tool, powered by the Hound engine, indexes repositories using regular expressions and pattern matching. That works well when you know exactly what a function is called, but it fails when vocabulary diverges from intent. A volunteer searching for permission validation logic may never find a function named checkUserLoginStatus, because the words simply do not overlap. With two decades of evolving and inconsistent naming conventions across thousands of repositories, these lexical gaps hinder volunteer onboarding and slow down even the Foundation’s core developers.

Code2Code Search, developed by Francesco Tosoni and described in the journal SoftwareX, takes a fundamentally different approach. Instead of matching strings, it transforms code snippets into vector representations, high-dimensional numerical encodings of a snippet’s semantic content. Queries and indexed code are embedded independently into a shared vector space and compared by distance, following the bi-encoder retrieval paradigm familiar from modern text search. The result is intent-based retrieval: submit a code sketch, and the system returns the functions, classes, and templates whose computational purpose most closely matches, even when they are written in a different programming language or use entirely different identifiers.

The scale of the undertaking is considerable. The team enumerated 2,568 repositories, cloned the main branch of each, and submitted all of them to Software Heritage, the universal archive of source code, ensuring persistent, citable identifiers known as SWHIDs for every artefact. Using the Tree-sitter parsing library, the pipeline extracted structural entities, functions, types, and templates, across twelve programming languages, from PHP and JavaScript to Go, Rust, and Lua. After filtering out repositories containing only documentation or assets, the final index covers 2,242 active repositories and roughly 1.29 million code entities, about 81 percent of them functions.

The neural engine at the heart of the system is Qwen3-Embedding-0.6B, chosen for its permissively licensed, open-source status and its strong showing in benchmarks of code-to-code recall. Each snippet is encoded into a normalised 1024-dimensional vector, a process that takes about 31 minutes on a single NVIDIA H200 GPU. But the clever part of the architecture lies in what happens next. The system follows a build-heavy, serve-light design: the expensive vectorisation runs offline on GPU hardware, while the deployed service operates on Toolforge, Wikimedia’s shared, CPU-only cloud platform with a strict 6 GiB memory limit.

To squeeze a million-vector index into that budget, the team turned to FAISS, Facebook’s similarity-search library, and product quantisation. The index is compressed into an IVF-PQ structure with 128 sub-quantisers at 8 bits per sub-vector, shrinking the memory footprint from roughly 4.9 GiB for a flat float32 baseline to just 168.6 MiB, a reduction of 96.6 percent. On a commodity laptop with no discrete GPU, the system answers queries with a median latency of 1.85 seconds. On the live Toolforge deployment, where the embedding model runs on throttled shared CPUs, latency rises to an average of about 31 seconds, and the team has an infrastructure request pending with the Wikimedia Foundation to bring GPU capacity to the service.

Does semantic search actually beat keywords? To find out, the researchers built a 27-query benchmark spanning four categories: canonical textbook algorithms, name-obfuscated patterns that hide their intent behind unusual identifiers, cross-language tasks, and niche Wikimedia-specific patterns. Because no human gold standard exists for code search in this ecosystem, they adopted the increasingly popular LLM-as-judge paradigm, assembling a jury of three large language models from different families, Claude Opus 4.8, Google Gemini 3.8 Flash, and Moonshot Kimi K2. Each judge graded a blinded, anonymised pool of results from both systems, never seeing which system produced which ranking, and inter-judge agreement reached a Krippendorff’s alpha of 0.74.

The verdict was unambiguous. On the headline rank-aware metric, nDCG@10, Code2Code Search achieved a jury mean of 0.81 against 0.63 for a BM25 keyword baseline over the same 1.29 million entities, with the confidence interval on the improvement excluding zero under all three judges. The advantage was largest precisely where lexical matching is weakest: in the name-obfuscated category, the neural system scored 0.83 versus 0.53, and in cross-language retrieval 0.81 versus 0.62. In one striking example, a query for SHA-256 file hashing pulled the keyword engine toward irrelevant path utilities because of a shared token, while the neural system returned the actual hashing functions at the top ranks. The baseline did retain an edge in a few niche, domain-specific cases where distinctive vocabulary dominates, and the authors note that dense retrievers can be misled by textual similarity between adjacent but unrelated functions, motivating future hybrid ranking approaches.

Beyond raw performance, the project makes a deliberate statement about values. Unlike some other AI initiatives within Wikimedia, which have relied on proprietary embedding models and commercial vector databases, every component of Code2Code Search, including the Qwen3 model, is released under the Apache Software License 2.0, in keeping with the movement’s commitment to free software. The interface, built on Wikimedia’s Codex design system, is localised in 30 languages at the time of writing, 14 of them Indian, maintained by volunteer translators through translatewiki.net and redeployed automatically with each merge. By attaching SWHIDs to every indexed entity, the system also makes each retrieved function permanently citable in academic work, operationalising the UNESCO Recommendation on Open Science at the granularity of individual software artefacts.

The implications reach well beyond Wikipedia. For the 1.5 billion monthly visitors to Wikimedia projects, the tool is invisible infrastructure, but for the developer community it tackles the cold-start problem of navigating an unfamiliar 2,500-repository corpus, and its RESTful API opens the door to integration with IDE plugins, bots, automated code review, and retrieval-augmented code generation. Because only the repository-discovery script is ecosystem-specific, the entire pipeline, parsing, archival, vectorisation, indexing, and serving, can be retargeted to other forges, research institutions, or enterprises by swapping a single component. Future work will explore cross-encoder reranking for higher precision, incremental reindexing for near-real-time updates, and natural-language queries. For now, the tool stands as a demonstration that state-of-the-art neural retrieval can live entirely on open infrastructure, compressing a million snippets of humanity’s collaborative code into a space small enough to run on a shared virtual machine, and finding the right one in under two seconds.

Subject of Research: Neural code-to-code retrieval for semantic search across the MediaWiki open-source software ecosystem

Article Title: MediaWiki Code2Code Search: Neural retrieval for the semantic discovery of open-source software entities

Article References: Tosoni, F. (2026). MediaWiki Code2Code Search: Neural retrieval for the semantic discovery of open-source software entities. SoftwareX, 36, Article 103127. https://doi.org/10.1016/j.softx.2026.103127

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103127

Keywords: MediaWiki, code search, neural embeddings, Qwen3, FAISS, Software Heritage, Wikimedia, semantic retrieval, open-source software, Tree-sitter, BM25, LLM-as-judge