Online consumer health forums provide an alternative source of medical information for internet users seeking guidance that static web articles cannot satisfy, yet these platforms face operational strain due to limited healthcare personnel and surging inquiry volumes. Focusing on Indonesian consumer health forums, the authors addressed three interconnected natural language processing tasks: sentence recognition, medical entity recognition, and keyphrase extraction. Developing these modules allows a semi-automatic system to identify critical components of questions, facilitate deeper semantic understanding, and support question reformulation with extracted key information. The authors expanded existing Indonesian datasets, established competitive transformer baselines, and evaluated multi-task learning architectures across pairwise and three-way configurations.
User inquiries posted on consumer health forums present natural language processing challenges because they routinely violate standard grammatical and orthographic rules. To parse these unstructured texts, the researchers formalized three sequence labeling tasks. Sentence recognition decomposes narrative submissions into segments and categorizes each as background context, question clauses, or ignorable material like greetings and courtesy expressions. Medical entity recognition detects four clinical categories: diseases, symptoms, pharmaceutical drugs, and therapeutic treatments. Extractive keyphrase extraction identifies concise word sequences that capture the essential focus of an inquiry, supplying structured terms that can guide document retrieval or match past questions in consumer health repositories.
A primary motivation for combining these modules is their documented interdependence within consumer health text. Statistical analysis of the training corpus revealed that tokens tagged as symptoms, drugs, or treatments never appeared within sentences designated as ignore-type text. For disease entities, tokens appearing within ignore sentences occurred exclusively in conventional preambles where users politely addressed a physician before stating their inquiry. A chi-square test confirmed a statistically significant relationship between the presence of medical entities or keyphrases and sentence category at a significance level of 0.05. Furthermore, prior research demonstrated that incorporating medical entities as predictive features improved keyphrase extraction performance. Because clinical entities often intersect with keyphrases and their presence indicates that a sentence represents background or an active inquiry rather than ignorable text, multi-task learning provides an effective framework to share representations across tasks.
Prior research on Indonesian biomedical question processing was constrained by small datasets and limited documentation, relying on cohorts of 192 instances for sentence recognition and 309 instances for entity and keyphrase extraction without public annotation guidelines or agreement metrics. To establish a standardized benchmark, the authors aligned 173 overlapping instances from existing corpora under refined guidelines and annotated 1,000 additional question records drawn from a collection by Hakim and colleagues. Annotation was conducted using Label Studio by two annotators per task over three months, requiring eighteen to twenty-five hours per annotator. The resulting unified corpus comprises 1,173 labeled data points divided into 773 training instances, 200 validation instances, and 200 testing instances. The training split contains 2,987 extractive keyphrases alongside annotated sentence and entity distributions, providing a substantial resource for Indonesian biomedical text mining.
To ensure data quality under resource constraints, annotation proceeded in two phases: paired annotation across 400 instances and single annotation across 300 instances per annotator. During the paired phase, annotators completed pilot and final rounds adhering to established biomedical annotation protocols. Analysis of pilot discrepancies revealed common oversights, such as unlabeled tokens in sentence recognition and omitted entities in medical entity recognition, prompting guideline revisions. Inter-annotator agreement was assessed using Cohen’s kappa and chunk-level F1-scores, which evaluate sequence labeling boundaries. Final chunk-level F1 agreement reached 88.61 percent for sentence recognition, 64.83 percent for medical entity recognition, and 35.01 percent for keyphrase extraction. Remaining discrepancies were resolved through rule-based adjudication prioritizing background and question types over ignore sentences and taking span intersections for entity boundaries, with author adjudication resolving persistent disagreements.
The authors formulated all three tasks as sequence labeling problems using the Begin-Inside-Outside tagging format, assigning specific prefix tags to designate chunk boundaries and categories for sentence types, entity classes, and keyphrases. To establish a baseline, the authors trained linear-chain Conditional Random Fields using handcrafted features adapted from prior Indonesian studies, including n-grams, part-of-speech tags, positional attributes, and medical dictionaries. For transformer models, input token sequences were bounded by classification and separation tokens, handling subtoken divisions by taking the last hidden state of the first subtoken. While generative large language models have advanced natural language processing, the authors prioritized BERT-based architectures due to their lower computational cost and documented advantages in fine-tuned sequence labeling scenarios. The resulting token representations were fed into output layers to generate probability distributions over the finite tag sets.
In single-task learning experiments, the authors benchmarked nine pretrained transformer encoders against the Conditional Random Field baseline across validation and test sets. Evaluated Indonesian models included IndoDistilBERT, IndoLEM Base, IndoNLU Base, IndoNLU Large, IndoBERTweet, and an IndoNLU variant fine-tuned on entity extraction, alongside multilingual encoders XLM-MLM, XLM-RoBERTa Base, and XLM-RoBERTa Large. All transformer models achieved higher F1-scores than Conditional Random Fields across all tasks. However, no single architecture dominated every task: XLM-RoBERTa Base attained the highest F1-score for sentence recognition, IndoNLU Large achieved top performance for medical entity recognition, and IndoLEM Base led in keyphrase extraction. Larger models did not consistently outperform smaller variants, nor was there a clear advantage distinguishing Indonesian language models from multilingual models.
To investigate multi-task learning, the authors adopted IndoNLU Large as the shared encoder because it achieved the highest average score across tasks in single-task experiments. The team designed parallel and hierarchical multi-task architectures for pairwise and three-way configurations. In parallel architectures, task-specific feedforward neural networks operate on shared representations from the encoder to predict probability distributions independently for each task. In hierarchical architectures, the output probability distribution of an upstream task is projected through a trainable label weight matrix into an embedding, which is concatenated with encoder outputs before feeding into the downstream task network. Pairwise models balanced objectives using a loss weighting hyperparameter alpha, while three-way architectures incorporated two weighting parameters to balance losses across sentence recognition, medical entity recognition, and keyphrase extraction during training on an NVIDIA DGX A100 GPU.
Pairwise multi-task learning experiments demonstrated that joint training can outperform single-task baselines, although performance dynamics varied between architectures. In parallel models, increasing the alpha weighting parameter consistently improved the performance of the primary task while diminishing results for the secondary task. In hierarchical models, this trade-off was less predictable, reflecting more complex feature interactions introduced by label embedding concatenation. The pairwise combination of medical entity recognition and keyphrase extraction achieved the highest scores for both constituent tasks. Under statistical testing, the hierarchical entity and keyphrase model with alpha set to 0.3 stochastically dominated the single-task baseline and other pairwise configurations for keyphrase extraction. For sentence recognition, a parallel model pairing entity recognition with sentence classification achieved the highest numerical score, though it lacked stochastic dominance over the single-task baseline.
In three-way multi-task learning trials, the investigators evaluated seven combinations of loss weights across parallel and hierarchical networks to simultaneously train all three modules. While certain three-way configurations improved entity and keyphrase extraction performance over single-task baselines, none surpassed the best pairwise model for sentence recognition. Across all three-way configurations, only a parallel model with alpha set to 0.4 and beta set to 0.2 achieved stochastically dominant performance for keyphrase extraction over baseline configurations. Adjusting loss weights across three tasks produced no consistent performance trajectory.
Detailed error analysis illuminated specific operational challenges faced by the models on consumer health inquiries. For sentence recognition, the best-performing parallel model correctly predicted both boundaries and types for 1,092 out of 1,509 gold standard sentences, achieving an exact match accuracy of 72.37 percent. The most frequent errors occurred when the model classified question or ignore tokens as background information, particularly when users described personal metrics like age or weight while asking questions on behalf of others. For medical entity and keyphrase extraction, over 60 percent of prediction errors resulted from incorrect span offsets, indicating difficulty in precisely delineating boundaries in informal text.
Subject of Research: Multi-task natural language processing models for sentence recognition, medical entity recognition, and keyphrase extraction in Indonesian consumer health forums.
Article Title: Sentences, entities, and keyphrases extraction from consumer health forums using multi-task learning
Article References: Naufal, T., Mahendra, R., & Wicaksono, A. F. (2025). Sentences, entities, and keyphrases extraction from consumer health forums using multi-task learning. Journal of Biomedical Semantics, 16(1), Article 8. https://doi.org/10.1186/s13326-025-00329-2
Image Credits: AI Generated
DOI: 10.1186/s13326-025-00329-2
Keywords: Multi-Task Learning, Consumer Health Forums, Sentence Recognition, Medical Entity Recognition, Keyphrase Extraction, Indonesian NLP, Sequence Labeling, Transformer Models, IndoNLU, XLM-RoBERTa, BIO Tagging, Conditional Random Fields, Almost Stochastic Order, Inter-Annotator Agreement, Clinical Text Mining, Label Studio, Health Question Answering, Biomedical Semantics
Cite Scienmag News
Daphne Blevins. (November 19, 2026). Multi-Task Transformers Advance Information Extraction From Online Health Forums. Scienmag. https://scienmag.com/multi-task-transformers-advance-information-extraction-from-online-health-forums/
Daphne Blevins. "Multi-Task Transformers Advance Information Extraction From Online Health Forums." Scienmag, 19 November 2026, https://scienmag.com/multi-task-transformers-advance-information-extraction-from-online-health-forums/. Accessed 19 November 2026.
Daphne Blevins. "Multi-Task Transformers Advance Information Extraction From Online Health Forums." Scienmag. November 19, 2026. https://scienmag.com/multi-task-transformers-advance-information-extraction-from-online-health-forums/








