Heterogeneous Graphs Help AI Crack Geometry Problems

Solving a plane geometry problem is deceptively hard for a machine. A human student glances at a diagram, reads a few sentences of text, notices that two lines look parallel or that an angle is marked as a right angle, and then reasons toward an answer. For artificial intelligence systems, that seamless blend of looking, reading, and deducing has remained one of the most stubborn challenges in multimodal reasoning. A new study published in Applied Intelligence proposes a fresh way to attack the problem: instead of treating a geometric diagram as an ordinary image, the researchers model it as a heterogeneous graph, an explicit network of points, lines, circles, and the relationships among them.

The work, led by Haotian Zhang of China Mobile Group Henan and Beijing Normal University, together with Qi Jia of Yuntu Zhixing and colleagues at IEIT Systems, Southeast University, and Beijing Normal University, addresses three long-standing weaknesses in existing geometry problem solvers. According to the authors, current methods struggle to capture the structural information embedded in diagrams, to align textual symbols with the visual primitives they refer to, and to cope with the sparse distribution of geometric primitives in typical figures. Each of these gaps can cause a model to miss the very relationships a problem depends on.

The core idea of the new system is a graph construction method that converts every diagram into a heterogeneous graph. In such a graph, nodes represent different kinds of geometric entities, for example points, line segments, circles, and angles, while edges encode the diverse relations that connect them, such as incidence, parallelism, or containment. Because the node and edge types differ, the representation is called heterogeneous, and it preserves far more of the diagram’s logical structure than a flat grid of pixels. The model can then reason over explicit geometric relationships rather than having to infer them implicitly from raw visual features.

Building on this construction method, the team created the Geometric Relation Heterogeneous Graph dataset, abbreviated Geo-RHG. The dataset pairs geometric diagrams with their corresponding textual descriptions and, crucially, with heterogeneous graph representations of each diagram. This triple structure gives researchers a resource for training and evaluating models that must understand both what a diagram shows and what the accompanying words say. The authors state that the dataset and code will be released at the project’s GitHub repository upon publication, making the resource available to the wider research community.

A second pillar of the approach is a cross-modal pre-training strategy designed to bridge the gap between language and vision. Text in a geometry problem refers to visual objects: a phrase such as ‘AB is parallel to CD’ names specific segments in the figure. The pre-training procedure explicitly aligns textual symbols with their corresponding visual primitives, teaching the model which words point to which parts of the graph. This grounding step is intended to eliminate a common failure mode in which a model understands the sentence and sees the diagram but never connects the two.

The third component is a hierarchical fusion mechanism that injects structural cues into visual features during the modality fusion stage. Rather than simply concatenating text embeddings and image embeddings, the system lets the graph-derived structural information refine the visual representation before and during fusion. The authors report that this integration enhances the model’s capacity to reason over geometric content and supports more accurate problem solving in multimodal contexts, because the fused features carry both appearance and relational information.

Extensive experiments described in the paper show that the proposed approach achieves superior performance on standard geometry problem solving benchmarks, outperforming existing baselines in both accuracy and generalization. Generalization matters particularly in this field: a solver that merely memorizes the problem templates of a training set is of little use in classrooms or examination settings, where problems vary widely in wording and figure style. The combination of explicit graph structure and cross-modal alignment appears to give the model a more transferable understanding of geometric content.

The new study sits within a research lineage that stretches back decades. Early efforts at automated geometry theorem proving date to the 1960s, when researchers such as Gelernter and colleagues built the first geometry theorem machines. Modern data-driven approaches began with systems like GeoSolver, which combined text and diagram interpretation, and later work harvested axiomatic knowledge from textbooks. More recently, benchmarks such as GeoQA, UniGeo, and augmented geometric question answering datasets have fueled progress, while frameworks like Inter-GPS introduced formal languages and symbolic reasoning, and GeoDRL applied reinforcement learning to deductive reasoning. Explainable solvers and layout-aware neural models have pushed the field further, yet the authors argue that none of these fully resolves the structural modeling and alignment problems their graph-based approach targets.

Technically, the system draws on a broad set of tools from modern deep learning. Vision encoders of the kind introduced in residual networks and vision transformers provide the raw visual features, while object detection architectures in the Faster R-CNN tradition help localize geometric primitives. Language understanding builds on pretrained models in the BERT and RoBERTa family. Graph convolutional networks, which were developed for modeling relational data, supply the machinery for propagating information across the heterogeneous graph, and the transformer attention mechanism underlies the cross-modal fusion. The authors also cite visual question answering research, including graph-structured representations for VQA and conditioned graph structures, as conceptual ancestors of their approach, alongside classical sequence models such as long short-term memory networks that once dominated multimodal tasks.

Diagram parsing itself has advanced rapidly in recent years, with work on plane geometry diagram parsing and alignment-based relation learning for diagram parsing showing that the geometric content of figures can be extracted with increasing fidelity. The new system builds on this progress but goes a step further by making the relational structure the central object of representation rather than a byproduct of parsing. The authors suggest that this shift is what allows their model to handle the sparse, symbol-heavy nature of geometric figures, where a handful of lines and labels carry all the meaning and background pixels carry almost none.

The implications extend beyond geometry homework. Multimodal reasoning systems that must combine images and language appear in visual question answering, document understanding, scientific figure analysis, and educational technology. A method that explicitly structures visual content as a graph and aligns it with text could transfer to any domain where the image is not a photograph but a structured drawing: circuit diagrams, flowcharts, chemical structures, or architectural plans. The hierarchical fusion idea, in which structural cues refine visual features before fusion with language, may prove useful wherever relational structure is the key to understanding an image.

For the field of automatic problem solving, the study offers a concrete recipe: represent the diagram as a heterogeneous graph, pre-train the model to align symbols with primitives, and fuse modalities hierarchically with structure in mind. The strong benchmark results reported by the team suggest that explicit structure, long championed in symbolic AI, and large-scale neural learning are not rivals but complements. As the Geo-RHG dataset and code become publicly available, other researchers will be able to test that proposition, refine the graph construction methods, and push multimodal mathematical reasoning closer to the fluency of a well-trained human geometer.

The article, ‘A geometry problem solver based on heterogeneous graph,’ appears in Applied Intelligence, volume 56, article number 439, published on 22 September 2026. The work was received on 12 November 2025 and accepted on 7 September 2026, and its authors span industry and academia in China, reflecting the growing collaboration between technology companies and universities on foundational problems in artificial intelligence. With benchmarks falling and resources opening up, geometry problem solving is shaping up as one of the clearest windows into how machines can learn to see, read, and reason as one.

Subject of Research: A multimodal AI system that solves geometry problems by representing diagrams as heterogeneous graphs and aligning them with text.

Article Title: A geometry problem solver based on heterogeneous graph

Article References: Zhang, H., Jia, Q., Liu, L., Xu, C., Jin, L., Pan, Y., Chen, F., Wang, Y., & Liu, Y. (2026). A geometry problem solver based on heterogeneous graph. Applied Intelligence, 56(15), Article 439. https://doi.org/10.1007/s10489-026-07471-z

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07471-z

Keywords: geometry problem solving, heterogeneous graph, multimodal learning, cross-modal pre-training, Geo-RHG dataset, graph neural network, visual reasoning, mathematical reasoning, AI benchmark, feature representation, automatic problem solving, Applied Intelligence