The system, called CLASS—short for Clinical LLM Abstraction & Structuring System—is described in a new feasibility report published in the Journal of Medical Systems. Led by anesthesiologist and informatics researcher Frederick H. Kuo, the team built CLASS as a modular, Python-based pipeline that runs entirely within a secure institutional computing environment, keeping protected health information behind the hospital’s own walls rather than sending it to external services. That design choice reflects a growing consensus in medical informatics: the power of large language models can be harnessed for clinical data without compromising patient privacy, provided the infrastructure is configured correctly.

Technically, CLASS rests on three pillars. The first is a set of concept lists curated by subject matter experts—in this case, surgeons and informatics specialists who defined exactly which procedures and clinical details the system should look for. The second is a task-specific prompt suite, a collection of carefully worded instructions that steers the large language model toward consistent, clinically grounded interpretations of each note. The third is a schema-constrained output format, which forces the model to return its findings in a structured, predictable structure rather than free-flowing prose. The results are then exported to an interactive dashboard built for expert review, allowing clinicians to verify outputs, spot errors, and analyze the extracted data at scale.

One of the most innovative features of CLASS is its handling of the unknown. Rather than simply classifying notes against a fixed list of predefined concepts, the pipeline actively flags potential variants or entirely novel concepts that do not fit the existing schema, surfacing them for expert consideration. In specialized fields where terminology evolves quickly and procedures are often described in non-standard ways, this ability to propose expansions to the concept vocabulary could fundamentally change how clinical registries and research databases are built and maintained.

To test the system, the researchers applied CLASS to a retrospective corpus of pediatric esophageal airway treatment surgery, or EATS, operative notes at their single center. EATS is a demanding test case: these operative notes describe complex, highly individualized procedures in children with airway and esophageal abnormalities, and much of the procedural detail is not captured in standard billing codes. The team compared CLASS outputs against adjudication by an experienced surgeon on the twenty longest notes in the corpus, yielding 3,960 individual note-procedure pairs for evaluation.

The results were striking. Across those thousands of judgments, the observed concordance between the automated pipeline and the surgeon’s adjudication reached an F1 score of 0.9967—a near-perfect measure of precision and recall combined. In practical terms, the model almost never missed a procedure the surgeon identified, and almost never invented one that was not there. For a task as subtle as parsing operative prose about pediatric airway surgery, that level of agreement suggests that large language models, when properly constrained and prompted, can match expert-level abstraction performance in at least some specialized clinical domains.

The novelty-detection results were more nuanced, and arguably more interesting. CLASS proposed 28 candidate procedure variants or additions to the curated concept list, and the clinical team judged 18 of them—64.3 percent—to be genuinely useful. That is a meaningful yield: nearly two out of every three suggestions from the machine were worth a clinician’s time. At the same time, the surgeon identified 12 additional procedures that CLASS failed to surface, a reminder that the system works best as a collaborator rather than a replacement. The human expert still caught things the machine missed, and the machine still surfaced things the human might not have thought to codify.

The authors are careful to frame the study appropriately. This was an exploratory implementation at a single center, focused on a single surgical service, with evaluation limited to a small set of long notes. Generalizability to other institutions, other note types, and other clinical tasks remains unproven, and the team emphasizes that broader validation is needed before such pipelines could be trusted for high-stakes applications. The full production code and clinical data cannot be released publicly because they involve protected health information and institution-specific infrastructure, but the researchers have shared technical implementation details, template code, pseudocode, and de-identified prompt examples in the online supplementary materials, giving other informatics teams a practical roadmap for building similar systems.

Even with those caveats, the implications are considerable. Clinical research has long been throttled by the bottleneck of manual abstraction: cohort studies that could enroll thousands of patients are often limited to hundreds simply because there are only so many hours in a research coordinator’s day. Quality-improvement programs face the same constraint, unable to measure surgical outcomes comprehensively when the relevant data must be pulled by hand from narrative notes. If pipelines like CLASS can reliably convert unstructured text into analyzable data within standard institutional infrastructure, the effective sample sizes of clinical research could expand dramatically, and hospitals could monitor the quality of specialized care in near real time.

The study also highlights a shift in how medical informatics teams may work in the coming years. Instead of writing brittle rule-based extraction algorithms or training bespoke machine-learning models on small labeled datasets, teams can now curate expert concept lists, design prompts, and review machine-generated suggestions—a workflow in which clinicians define what matters and the model handles the linguistic heavy lifting. The CLASS experience suggests this human-machine partnership can work: the model performs the extraction with near-perfect fidelity, proposes useful vocabulary expansions, and leaves final judgment to the experts who bear clinical responsibility. As large language models continue to demonstrate their grasp of medical language, studies like this one offer a concrete, privacy-conscious template for turning the narrative richness of the medical record into structured knowledge—without a single chart being pulled by hand.

Subject of Research: A large language model pipeline for extracting structured data from unstructured clinical notes

Article Title: Exploratory Implementation and Feasibility Report of CLASS (Clinical LLM Abstraction & Structuring System), A Large Language Model Pipeline for Extracting Unstructured Data From Clinical Notes

Article References: Exploratory Implementation and Feasibility Report of CLASS (Clinical LLM Abstraction & Structuring System), A Large Language Model Pipeline for Extracting Unstructured Data From Clinical Notes. (n.d.). https://doi.org/10.1007/s10916-026-02462-6

Image Credits: AI Generated

DOI: 10.1007/s10916-026-02462-6

Keywords: large language models, clinical notes, natural language processing, electronic health records, medical informatics, data extraction, pediatric surgery, artificial intelligence, CLASS pipeline, clinical research, esophageal airway treatment surgery, Johns Hopkins

The study, conducted by Sungsu Oh and Hyunsu Lee of Pusan National University’s Department of Physiology together with neurosurgeons In Ho Han, Jae Il Lee, and Byung Kwan Choi of Pusan National University Hospital, was published in Medical & Biological Engineering & Computing. The work tackles a problem that has plagued health informatics for decades: electronic medical record systems are heterogeneous by design, shaped by local workflows, billing requirements, and clinical habits. When hospitals want to collaborate — whether for multi-site clinical trials, epidemiological surveillance, or artificial intelligence development — their data must first be normalized onto a common clinical ontology. SNOMED-CT, the Systematized Nomenclature of Medicine—Clinical Terms, is the most comprehensive such ontology in existence, but mapping free-text diagnoses to its precise concepts has traditionally required laborious manual coding or brittle rule-based systems.

The Korean team’s strategy begins with ClinicalBERT, a variant of the BERT transformer architecture that has been pretrained on clinical text, giving it an innate familiarity with the abbreviations, fragments, and idiosyncrasies of physician prose. Rather than treating diagnosis classification as a pure text-labeling task, the researchers framed it as a problem of geometry. They trained ClinicalBERT on electronic medical record data to produce latent representations — dense numerical vectors — for spans of text corresponding to diagnoses. In parallel, they encoded the fully specified names, or FSNs, of SNOMED-CT concepts: the ontology’s unambiguous, self-describing labels such as “myocardial infarction (disorder).” The goal of the fine-tuning stage was to pull these two families of vectors toward one another in the embedding space, so that the vector for a hospital’s idiosyncratic diagnosis text would land close to the vector for its correct SNOMED-CT concept.

The alignment was accomplished through mean squared error-based fine-tuning, a loss function that penalizes the squared distance between the EMR-derived diagnosis embeddings and their target SNOMED-CT FSN embeddings. This is a conceptually elegant move: instead of asking the model to memorize a lookup table of diagnoses, the researchers asked it to reshape its internal geometry so that semantic equivalence becomes spatial proximity. Once the alignment was complete, the resulting latent representations were fed into a downstream classification model that assigns each diagnosis span to one of 273 SNOMED-CT classes.

The performance numbers are striking. The proposed model achieved an accuracy of 0.934, a weighted F1-score of 0.923, and a macro-averaged F1-score of 0.823 across the 273 classes. The gap between the weighted and macro scores reflects a familiar reality of clinical data: some diagnostic classes appear far more frequently than others, and models naturally excel at common conditions while struggling with rare ones. A macro-averaged F1 above 0.82 nevertheless indicates that the model maintains strong performance across the board, not merely on the frequent diagnoses that dominate hospital records.

Perhaps the most thought-provoking finding, however, is what the fine-tuning did not do. Despite measurably improving semantic alignment in the embedding space, the fine-tuned model performed roughly on par with the base, unmodified ClinicalBERT when it came to the downstream classification task. Both remained competitive with dedicated biomedical entity-linking systems — SapBERT, which achieved an accuracy of 0.944 in the comparison, and BioSyn, at 0.931. This apparent paradox — better alignment, similar classification — is one of the study’s central insights. The researchers analyzed the latent representations directly, examining how diagnosis vectors were distributed before and after fine-tuning, and found that the fine-tuned model exhibited reduced similarity distances and more distinct class separation. Fine-tuning reduced the average Manhattan distance between paired representations by 36.2 percent and the average cosine distance by 53.1 percent, evidence that the embeddings had indeed converged toward their ontological targets.

In other words, the geometry of the latent space became cleaner and more semantically coherent, even if the final classification accuracy did not climb. The authors suggest that base ClinicalBERT, having been pretrained on large volumes of clinical text, already encodes much of the structure needed for discrimination, and that MSE-based alignment improves the interpretability and organization of the space without necessarily adding discriminative power. This trade-off between semantic alignment and discriminative performance is a subtle but important consideration for anyone designing medical NLP systems: a model whose embeddings cluster cleanly by concept may be more useful for retrieval, normalization, and cross-dataset integration even when its headline accuracy is unchanged.

The implications reach beyond a single hospital’s coding workflow. Medical concept normalization is a foundational enabling technology for the secondary use of health data — the practice of reusing clinical records for research, quality improvement, and public health. Studies have documented both the enormous opportunity and the serious technical and privacy barriers involved. Manual coding is expensive and inconsistent; rule-based systems fail when clinicians invent new abbreviations; and naive deep learning classifiers trained at one institution often collapse when deployed at another, where documentation habits differ. An embedding-based approach that explicitly anchors diagnoses to SNOMED-CT fully specified names offers a path toward models that transfer more gracefully, because the anchor points are institution-independent.

Privacy considerations loom large in this domain, and the Korean team’s approach has advantages here as well. Because the model operates on diagnosis spans rather than whole patient records, and because the data used to train it were de-identified retrospective records — a design approved by the Institutional Review Board of Pusan National University Hospital, with informed consent waived for the de-identified data — the pipeline is compatible with privacy-preserving deployment. The authors describe the method as having strong potential for scalable, privacy-preserving medical concept normalization in real-world clinical environments. Notably, the trained artifacts are embeddings and classifiers rather than raw patient data, meaning institutions could in principle share model components without sharing records — a property that aligns with the growing movement toward federated and privacy-conscious health AI.

The study also contributes to a rapidly maturing field. Automated medical coding has seen an explosion of deep learning approaches in recent years, with transformer models applied to ICD-10 and SNOMED-CT coding tasks across multiple languages and healthcare systems. Comparisons in this literature are notoriously difficult because datasets, class counts, and annotation standards vary widely; by benchmarking directly against SapBERT and BioSyn, two widely cited biomedical representation models, the Pusan National University team situates their result within a recognizable landscape. Their 0.934 accuracy sits within roughly one percentage point of SapBERT’s 0.944, achieved with a straightforward MSE alignment strategy rather than specialized contrastive pretraining.

There are caveats, as with any machine learning study in medicine. The model was trained and evaluated on data from a single hospital network, and while 273 SNOMED-CT classes is a substantial label space, it is a small fraction of the ontology’s several hundred thousand concepts. Rare diagnoses, ambiguous abbreviations, and multilingual clinical text all remain open challenges. The authors also emphasize that the observed tension between semantic alignment and classification performance deserves further study — future work may find alignment objectives that improve both properties simultaneously, or demonstrate that better-aligned embeddings pay off in downstream tasks beyond classification, such as cross-lingual normalization or retrieval-augmented clinical decision support.

Still, the work represents a meaningful step toward a long-promised vision: hospital data that can travel. When a diagnosis written in a Busan emergency department can be automatically and reliably expressed in the same standardized language as one written in a Baltimore clinic, the foundations for multi-institutional research, fairer AI training datasets, and population-scale health insights become dramatically more accessible. The Korean team’s results suggest that the key may lie not just in better classifiers, but in sculpting the hidden geometric spaces where machines understand medicine — one embedding at a time.

At the forefront is the Journal of Medical Internet Research (JMIR), JMIR Publications’ flagship journal, which continues to set the benchmark for quality and influence in digital health. Registering a JIF of 6.0, JMIR holds a distinguished Q1 ranking in both the "Health Care Sciences and Services" as well as the "Medical Informatics" categories. With over 50,000 citations within the 2024 impact factor tracking window, it stands as the most cited journal in these disciplines, underscoring its pivotal role in shaping the intersection of medicine and digital technology.

Two other journals within JMIR’s catalog have emerged as leaders in their fields, securing top rankings that affirm their scholarly prestige. JMIR Aging maintains its position as the highest-ranked journal in the field of Gerontology, boasting an impact factor of 4.8. Meanwhile, JMIR Medical Education has made a remarkable debut with an inaugural JIF of 12.5, earning the top spot in the “Education, Scientific Disciplines” category. Its focus on emerging technologies, including artificial intelligence and large language models, positions it at the cutting edge of medical education research in an era increasingly shaped by digital innovation.

The portfolio further exhibits outstanding growth among several journals that have seen a notable increase in citation metrics this year. JMIR mHealth and uHealth, with an impact factor of 6.2, has attained a ranking within the top ten journals dedicated to medical informatics and healthcare services, signaling its importance in mobile and wearable health technologies. Similarly, JMIR Mental Health has achieved a JIF of 5.8, ranking in the top 10% of psychiatry journals globally and marking its prominence in digital mental health interventions.

In addition to these standout journals, a cohort of JMIR’s specialized titles continues to contribute significantly to their respective fields, evidenced by upward trends in their impact factors. Fields such as public health surveillance, biomedical informatics, and human factors in healthcare technology are represented by journals that cluster in competitive quartiles, signaling consistent academic interest and validation of the research they publish.

JMIR Publications is also proud to announce the successful launch of six new journals receiving inaugural impact factors. Among these, JMIR Nursing impressed by entering the top decile of nursing journals immediately, with a JIF of 4.0. JMIR Rehabilitation and Assistive Technologies, focused on innovations in rehabilitation science and assistive devices, debuted with a solid impact factor of 3.0, indicating rapid recognition in the rehabilitation research community. Additional titles such as JMIR Diabetes and JMIR Cardio provide emerging platforms for targeted medical specialties within digital health.

The significance of these achievements extends beyond statistics, highlighting JMIR’s dedication to an open access model that promotes rapid and equitable dissemination of scientific knowledge. By ensuring that the latest digital health research is freely accessible, JMIR Publications cultivates a comprehensive ecosystem combining rigorous peer review, technological innovation, and author advocacy to amplify research impact worldwide.

According to Tiffany I Leung, MD, MPH, FAMIA, JMIR Publications’ scientific editorial director, “These impact factor results are testament to the high-quality, innovative scientific contributions from our authors. Our commitment to open access ensures that vital discoveries in digital health reach clinicians, policymakers, and researchers without barriers, ultimately advancing health and well-being globally.” Such a statement echoes the publisher’s vision to transcend traditional scholarly metrics while emphasizing the societal relevance of accessible research.

Complementing this vision, Gunther Eysenbach, MD, MPH, FACMI, founder and publisher of JMIR Publications, articulated the strategic ambition to develop leading journals covering technology-driven innovation across all medical disciplines. This holistic portfolio approach is crafted to align with the pervasive influence of digital health and artificial intelligence on modern medicine, ensuring that each specialty benefits from tailored, cutting-edge research dissemination.

Technical nuances behind the 2025 impact factor calculations reveal that certain journals, notably JMIR Medical Education and JMIR Rehabilitation and Assistive Technologies, experienced delays in indexing within the Clarivate Web of Science database. Despite this, official projections affirm their inclusion in forthcoming Journal Citation Reports updates, signifying confidence in the accuracy and validity of the metrics assigned. Such transparency around data curation enhances trust within the scholarly community.

The growing body of impact metrics places JMIR Publications firmly within a competitive landscape, alongside its recent success in Scopus CiteScore rankings, where 26 JMIR journals attained scores, and 12 were listed in the top quartile (Q1). This multi-metric approach underscores the publisher’s commitment to providing comprehensive, reliable, and nuanced indicators of journal performance and influence.

Importantly, JMIR Publications advocates for a broader conception of research impact, encouraging the integration of alternative metrics such as author satisfaction surveys, Altmetric scores, and other open science indicators alongside traditional impact factors. By endorsing guidelines such as the Declaration on Research Assessment (DORA), JMIR fosters a scholarly culture that balances quantitative citation data with qualitative and societal value assessments in journal evaluation processes.

As JMIR Publications continues to expand and refine its portfolio, its open access platform drives forward evidence-based digital health practices and innovations that hold promise to transform healthcare delivery globally. The 2025 impact factor release is a milestone that both celebrates past accomplishments and energizes future efforts to harness technology and scholarship for improved population health outcomes.

In essence, the new Journal Impact Factor results solidify JMIR Publications’ position as an influential force in the global digital health research ecosystem. Its journals offer researchers advanced avenues for thought leadership and collaboration, while providing practitioners and policymakers with timely access to groundbreaking studies in medical informatics, mental health, aging, and numerous other specialized domains.

The synergy between open access publication, rigorous peer review, and technological innovation positions JMIR Publications as a model for future scholarly communication, optimizing the translation of research into real-world health solutions. As digital health continues to evolve rapidly, JMIR’s growing portfolio will likely remain at the forefront of this dynamic field, championing transparency, accessibility, and impact for years to come.

Subject of Research: Digital Health, Medical Informatics, Open Access Publishing

Article Title: JMIR Publications Achieves Major Growth with 21 Journals Receiving 2025 Impact Factors, Demonstrating Open Access Success

News Publication Date: June 20, 2025

Web References:

• JMIR Publications: https://jmirpublications.com/
• Journal Citation Reports 2025 from Clarivate: https://clarivate.com/academia-government/scientific-and-academic-research/research-funding-analytics/journal-citation-reports/

Image Credits: JMIR Publications

Keywords: Digital Health, Medical Informatics, Open Access, Journal Impact Factor, Digital Medicine, Medical Education, Gerontology, Mobile Health, Mental Health, Rehabilitation Technologies, Nursing, Artificial Intelligence