Why AI Still Struggles to Answer Medical Questions in Bangla

Large language models have transformed medical question answering, but only for a privileged slice of the world’s languages. A new review published in Frontiers in Digital Health argues that Bangla, spoken by roughly 300 million people, remains critically underserved by the AI systems increasingly consulted for health advice. The study, led by researchers from BRAC University and partner institutions in Bangladesh, Malaysia, and the United States, analyzed 15 representative studies published between 2022 and 2026 to map the methodological landscape of medical question answering in Bangla and compare it against the mature English-language ecosystem.

The review’s central finding is stark: the performance gap between English and Bangla medical AI is not incidental but structural. English systems benefit from massive domain-specific pretraining corpora, standardized benchmarks such as MedQA and MedMCQA, and well-established evaluation practices. Bangla systems, by contrast, grapple with tiny datasets, heavy reliance on machine-translated English text, and no shared benchmarks at all. The authors describe this as an asymmetry in the maturity of the entire research ecosystem, not merely a difference in model capability.

To organize the field, the researchers built a taxonomy of eight methodological paradigms, classified by the mechanism that produces answers at inference time. These range from rule-based systems using hand-crafted lexicons, through supervised neural models and transformer fine-tuning, to prompt-based inference, parameter-efficient adaptation, retrieval-augmented generation, multi-task learning, and multimodal question answering. Where a study combined approaches, the authors assigned it to the paradigm its own evaluation credited with the principal performance gain.

The trajectory of Bangla medical NLP mirrors the broader evolution of the field. Early systems were rule-based and database-driven. One 2022 effort, XBLQPS, combined tokenization, root extraction, part-of-speech tagging, and a simplified LESK algorithm for word sense disambiguation to map Bangla health queries onto structured database lookups, achieving about 70 percent accuracy. Later work shifted to statistical and classical machine learning: researchers built datasets such as BnMedText and BnMedQA, applied TF-IDF and word embedding features, and trained classifiers including support vector machines and random forests, alongside a hybrid CNN-BiLSTM model called MeQANet.

Transformer architectures then raised the ceiling on contextual understanding. A team curating the BanglaCHQ-Summ dataset of 2,350 conversational health query pairs fine-tuned BanglaT5, mT5, and mBART for medical summarization, with BanglaT5 reaching a ROUGE-L score of 48.35 percent and a BERTScore of 89.91 percent. Notably, language-specific models consistently outperformed multilingual counterparts, a pattern repeated in named entity recognition, where BanglishBERT achieved the highest F1 on informal Bangla clinical text while zero-shot GPT-3.5 managed only 15.51 percent on the Bangla-HealthNER dataset.

Prompt-based methods have produced some of the most striking results. On a 659-sample Bangla medical dataset for specialty recommendation, zero-shot GPT-4-Turbo scored an F1 of just 38.2 percent, but mixed prompting strategies lifted the same model to 86.1 percent. The catch, the review notes, is that such gains depended on manual prompt engineering, and the dataset itself suffered from class imbalance and gender and organ biases. Parameter-efficient fine-tuning has also emerged as a practical pathway: applying LoRA to a Gemma-2B model demonstrated effective domain adaptation on a small 300-pair Bangla medical QA dataset, though the study was qualitative only and lacked baseline comparisons.

On the English side, the review documents far more sophisticated machinery. Multi-task learning with ClinicalBERT improved answer-extraction F1 by 2.2 percent and classification accuracy by 6.2 percent over single-task training on the emrQA dataset. Retrieval-augmented generation has become the dominant strategy for compensating for LLM knowledge gaps: the MKRAG framework combined two-stage retrieval with knowledge injection through in-context learning, improving Vicuna-7B accuracy by 4.08 percent without any fine-tuning, while the RAG2 system, which generates chain-of-thought rationales to guide retrieval and filters evidence with a perplexity-guided model, improved accuracy by up to 6.1 percent across MedQA, MedMCQA, and MMLU-Med. Crucially, the review emphasizes that retrieval quality matters more than quantity, since irrelevant context actively degrades accuracy.

Yet even English-language success comes with caveats that carry sobering implications for any language. One 2026 evaluation found ChatGPT-4 answered only 1 of 14 breast cancer exercise counseling questions reasonably accurately against clinical guidelines. Other work showed weak correlation between open-ended and closed-ended benchmark performance, and that selecting the lowest-uncertainty responses improved accuracy without architectural changes, suggesting that knowing when not to answer is as clinically valuable as answering correctly. Strong benchmark scores, the authors conclude, do not automatically translate into trustworthy responses in nuanced clinical contexts.

The structural constraints on Bangla medical QA are formidable. Existing corpora are tiny, such as the roughly 4,500 QA pairs in BnMedQA or the 659 patient-symptom pairs used in prompting studies, and many rely on machine translation from English, introducing semantic drift and cultural misalignment. Classifiers trained on translated data collapse on semantically broad categories like general medicine while performing reliably on specialties with distinctive vocabulary such as gynecology or ENT. No Bangla symptom-based diagnostic QA dataset exists, none include clinician-validated ground truth, and datasets remain almost entirely text-only despite the multimodal reality of clinical medicine. Bangla’s rich morphology, dialectal variation, and code-switching with English further strain models trained primarily on English corpora.

The review closes with a structured roadmap. The authors call for building the first clinician-validated Bangla symptom-based diagnostic QA dataset, developing expert-annotated corpora that capture dialectal variation and code-mixed text, pursuing domain-specific pretraining on native Bangla medical text with parameter-efficient methods such as LoRA and QLoRA, rebuilding retrieval pipelines around native-language knowledge bases, and establishing standardized benchmarks derived from sources excluded from model pretraining. Bias auditing and lightweight deployment strategies, they argue, should be prerequisites for any clinical deployment. The message is clear: the goal is not to replicate English models but to build the language-specific resources and evaluation infrastructure that make reliable, linguistically inclusive medical AI possible for the hundreds of millions who ask their health questions in Bangla.

Subject of Research: Large language model approaches to medical question answering in Bangla and cross-lingual comparison with English systems

Article Title: A mini-review of medical question answering for Bangla: the methodological landscape and cross-lingual perspectives

Article References: A mini-review of medical question answering for Bangla: the methodological landscape and cross-lingual perspectives. (n.d.). https://doi.org/10.3389/fdgth.2026.1856734

Image Credits: AI Generated

DOI: 10.3389/fdgth.2026.1856734

Keywords: Bangla NLP, medical question answering, large language models, retrieval-augmented generation, low-resource languages, healthcare AI, transformer fine-tuning, parameter-efficient fine-tuning, LoRA, prompt engineering, benchmarking, digital health