Researchers introduced a novel approach for classifying biomedical relations by combining Medical Subject Headings (MeSH) keywords with machine learning models, achieving an accuracy of 70.78% across 195 distinct relation classes. The study, published in the Journal of Biomedical Semantics, demonstrates that leveraging the structured bibliographic metadata of PubMed publications can effectively categorize relationships between biological entities. This method addresses the limitations of traditional algorithms that rely heavily on large chunks of raw text, which often suffer from reduced generalization, precision, and reliability due to the complexity of natural language. By focusing on the distinctive characteristics of bibliographic metadata, the team developed a dataset and classification framework that offers a more robust pathway for biomedical ontology engineering.
The core of the research involves the creation of MeSH2Matrix, a dataset consisting of 46,469 biomedical relations curated from PubMed publications. This dataset is built upon the principle that the qualifiers of co-occurring MeSH keywords can outline the type of semantic relation between them. MeSH keywords are controlled vocabulary terms used to annotate scholarly publications, and they are further refined by predefined subheadings known as qualifiers. These qualifiers provide specific facets of a topic, such as therapeutic use or diagnosis, allowing for a more precise representation of the subject matter. The dataset includes a matrix that maps the associations between the qualifiers of subject MeSH keywords and those of object MeSH keywords, specifying the corresponding Wikidata relation type and the superclass of semantic relations for each entry.
To construct the dataset, the researchers extracted biomedical relations from Wikidata, an open and collaborative knowledge graph that links entities using semantic relations. They identified items in Wikidata with a MeSH Descriptor ID and found their relationships with other items that also had MeSH identifiers. For each subject-object pair, the team retrieved the associated MeSH qualifiers from PubMed using the NCBI Entrez API. They then calculated a matrix of correspondence for each pair, where each element represents the proportion of publications featuring specific subject and object qualifiers. This matrix encodes the nature of the semantic relation, providing a structured input for machine learning models to learn from.
The study evaluated three machine learning models: a Support Vector Machine (SVM), a dense model (D-Model), and a convolutional neural network (C-Net). The SVM was used as a baseline, transforming the 89 by 89 qualifier matrices into single feature vectors. The D-Model was a multi-layer perceptron designed to learn latent feature representations directly from the labeled training data. The C-Net was a convolutional neural network architecture that interpreted the feature matrices as spatially correlated features, utilizing convolution layers to capture dependencies within the data. The dataset was split into training, validation, and testing sets, with the training set comprising 33,457 samples, the validation set 13,012 samples, and the testing set 9,294 samples.
The best-performing model, the D-Model, achieved an accuracy of 70.78% for the 195-class classification task and 83.09% for the five-superclass classification task. The five superclasses were derived by grouping the 195 relation types into broader categories: non-biomedical non-symmetric, biomedical non-symmetric, non-biomedical symmetric, biomedical symmetric, and taxonomic. The performance on the five-class task was notably better than the 195-class task, suggesting that reducing the complexity of the classification problem by generalizing relation types improves model accuracy. The SVM performed less effectively than the neural network models, likely due to a bias towards taxonomic relations caused by class imbalance in the dataset.
Feature analysis using Integrated Gradients revealed that only a limited number of MeSH qualifiers heavily influence the classification outcome. The most impactful features varied by class and model architecture, with some features being positively correlated in one model and negatively correlated in another. For instance, features related to pharmacology and therapeutic use were significant for biomedical non-symmetric relations, while features like epidemiology and ethnology were predictive for non-biomedical symmetric relations. This analysis highlights that not all MeSH qualifiers contribute equally to the classification, and some can be eliminated without affecting overall accuracy. The findings suggest that the MeSH qualifiers are particularly effective for representing biomedical and symmetric relations, but less so for generic or non-symmetric ones.
The study also examined the effect of restricting the dataset to matrices based on 100 or more scholarly publications and to well-represented relation types. Eliminating uncommon semantic relations co-occurring in fewer than 100 PubMed records improved the accuracy and F1-Score of the classification, confirming the importance of the number of associations between terms in scholarly publications. However, removing minor relation types did not positively affect classification efficacy, indicating that the approach is robust to label noise. The results show that the MeSH2Matrix-based classification cannot achieve absolute accuracy even with more efficient machine learning algorithms, primarily due to the confusion between taxonomic and non-symmetric relation types and the limited ability of MeSH qualifiers to capture non-biomedical semantic relations.
Limitations of the approach include its lack of accuracy in classifying non-biomedical relations, such as subject has role and shares border with, which are not well-characterized by MeSH qualifiers. Additionally, Wikidata, used as a benchmark, can include inconsistencies due to its open and collaborative editing policy, which may introduce irrelevant biomedical relations. The study notes that a considerable rate of Wikidata relations are not available in PubMed records, which limits the generation of qualifier matrices for these relations. Despite these limitations, the approach provides a promising direction for improving biomedical relation classification by leveraging bibliometric-enhanced information retrieval.
The researchers propose future directions that include optimizing the approach by considering the semantic features of the subjects and objects and revising the MeSH2Matrix dataset. They also envision expanding the method into a tool for constructing and enriching biomedical knowledge graphs based on PubMed MeSH keywords. By coupling this method with knowledge-based approaches for biomedical entity recognition and relation extraction, the team aims to develop more robust systems for validating biomedical ontologies. The source code and dataset are publicly available to ensure reproducibility and to allow other researchers to explore and build upon the findings.
This study contributes to the field of biomedical informatics by demonstrating the value of structured bibliographic metadata in relation classification. By moving away from reliance on raw text and focusing on the controlled vocabulary of MeSH keywords, the researchers have developed a method that offers improved precision and reliability. The MeSH2Matrix dataset and the associated machine learning models provide a foundation for further research into the automatic classification of biomedical relations, with potential applications in organizing and extracting insights from the vast body of biomedical literature. The work underscores the importance of leveraging existing structured data resources to enhance the efficiency and accuracy of biomedical ontology engineering.
In detailing the mathematical construction of the correspondence matrices, the authors grounded the matrix values in publication co-occurrence proportions. To prevent request timeouts and rate limit errors from the NCBI PubMed application programming interface, the extraction pipeline restricted queries to the one hundred most relevant articles identified by the PubMed Best Match search algorithm. Within each matrix, an individual entry represents the count of retrieved records containing both specified qualifiers divided by the total number of papers containing both MeSH terms. For instance, the authors highlighted a pair examined on March 6, 2022, where Hepatitis C and Sofosbuvir co-occurred across thirty-two PubMed records. In fifteen of those publications, Hepatitis C was assigned the qualifier drug therapy while Sofosbuvir received the qualifier therapeutic use, producing an association value of approximately 0.469 within the resulting correspondence matrix.
The design of the neural architectures reflected differing strategies for ingesting the eighty-nine by eighty-nine dimension matrices, which represent seventy-six unique qualifiers alongside thirteen duplicate designations included across multiple contexts. The dense model featured an input layer with an output feature size of 3,960 connected via rectified linear units to a hidden layer of 1,980 nodes, whereas the convolutional network stacked four sequential blocks comprising two-dimensional convolutions, batch normalization, activation functions, and max-pooling operations before reaching two fully connected layers. When evaluating specific relation types, performance varied between the two deep learning systems. The dense model demonstrated superior F1-scores for relations such as diplomatic relation, has part, and cell component. In contrast, the convolutional network performed better on specific classifications including significant drug interaction and instance of. Both architectures achieved F1-scores close to or above 0.9 on well-represented relations, specifically drug or therapy used for treatment and medical condition treated, while recording comparatively weak scores on shares border with and part of.
To evaluate class imbalance across categories, the researchers analyzed the geometric and arithmetic means of class distributions alongside the ratio between them. Confusion matrix assessments demonstrated that notable misclassifications occurred between taxonomic statements and non-symmetric relations because both relation structures link concepts with differing domains and ranges. Furthermore, the heavy proportion of taxonomic instances in the dataset introduced a prediction bias that penalized the baseline support vector machine more severely than the neural network classifiers. To address this confusion, the authors proposed an architectural adjustment that splits the dataset into an array of symmetric features and a separate matrix of non-symmetric attributes, thereby training models separately and restricting symmetric features to the classification of taxonomic and symmetric relations.
Subject of Research: Biomedical relation classification using MeSH keywords and machine learning
Article Title: MeSH2Matrix: combining MeSH keywords and machine learning for biomedical relation classification based on PubMed
Article References: Turki, H., Dossou, B. F. P., Emezue, C. C., Owodunni, A. T., Hadj Taieb, M. A., Ben Aouicha, M., Ben Hassen, H., & Masmoudi, A. (2024). MeSH2Matrix: combining MeSH keywords and machine learning for biomedical relation classification based on PubMed. Journal of Biomedical Semantics, 15(1), Article 18. https://doi.org/10.1186/s13326-024-00319-w
Image Credits: AI Generated
DOI: 10.1186/s13326-024-00319-w
Keywords: MeSH keywords, biomedical relation classification, machine learning, PubMed, Wikidata, MeSH2Matrix, support vector machine, convolutional neural network, dense model, bibliographic metadata, semantic relations, biomedical ontology, feature analysis, integrated gradients, knowledge graph, class imbalance, qualifier matrices, biomedical informatics, relation extraction, superclass classification
Cite Scienmag News
Blake Davidson. (November 21, 2026). MeSH Keywords Improve Biomedical Relation Classification Accuracy. Scienmag. https://scienmag.com/mesh-keywords-improve-biomedical-relation-classification-accuracy/
Blake Davidson. "MeSH Keywords Improve Biomedical Relation Classification Accuracy." Scienmag, 21 November 2026, https://scienmag.com/mesh-keywords-improve-biomedical-relation-classification-accuracy/. Accessed 21 November 2026.
Blake Davidson. "MeSH Keywords Improve Biomedical Relation Classification Accuracy." Scienmag. November 21, 2026. https://scienmag.com/mesh-keywords-improve-biomedical-relation-classification-accuracy/








