A research team has demonstrated that decoupling pre-trained language representations from downstream sequence encoding can yield high classification accuracy on noisy retail investor discussions while substantially lowering computational requirements. Published in the Journal of Electrical Systems and Information Technology, the investigation presents an ERNIE-Transformer framework designed to classify the sentiment polarity of Chinese financial forum texts into positive, neutral, and negative categories. Evaluated on an authentic corpus of 6,000 investor comments gathered from the Eastmoney Stock Forum, the proposed architecture recorded an overall classification accuracy of 92.75 percent, an analytical precision of 89.14 percent, a recall rate of 87.21 percent, and a macro-averaged F1-score of 88.12 percent. By freezing the underlying language model and updating only a lightweight task-adaptation module, the system shrank the volume of active training parameters from approximately 110 million down to 12.5 million, achieving an 88.6 percent reduction in trainable capacity without degrading predictive reliability.
Online investment communities have evolved into primary hubs where retail investors exchange market observations, debate corporate developments, and express real-time financial expectations. Platforms such as the Eastmoney Stock Forum produce massive streams of unstructured, user-generated commentary that reflect underlying market psychology and collective trading behaviors. Analyzing these communicative streams offers potential utility for tracking investor sentiment and evaluating short-term market dynamics. However, financial forum text presents unique computational obstacles that frequently confound conventional natural language processing techniques. Posts are typically brief, highly informal, and saturated with emerging Internet neologisms, colloquial idioms, and figurative rhetoric such as sarcasm, irony, and hyperbole. Furthermore, market-specific discussions rely heavily on specialized technical terminology and financial jargon, creating dense contextual nuances that generic text classification models struggle to disambiguate.
Previous approaches to sentiment detection historically advanced from dictionary matching and shallow machine learning classifiers to recurrent neural networks and transformer-based pre-trained language models. Traditional algorithms such as Support Vector Machines and Naive Bayes depend heavily on manual feature engineering, which limits their capacity to represent deep contextual dependencies in unstructured data. While deep recurrent networks and bidirectional architectures improve automated feature learning, standalone recurrent structures can encounter gradient degradation across extended sequences and incur computational redundancy when combined into complex multi-module hybrid frameworks. The emergence of general-purpose models like BERT introduced powerful contextual modeling, yet adapting these expansive architectures to niche financial corpora remains difficult. Full-parameter fine-tuning of massive models on modest datasets frequently induces severe statistical overfitting. Moreover, financial emotional polarity is intensely context-sensitive, where specific industry terms carry conflicting connotations depending on whether a market participant holds cash or active equities.
To overcome the dual challenge of domain-specific linguistic ambiguity and small-sample overfitting, the authors formulated a decoupled single-stream deep learning pipeline centered on the knowledge-enhanced ERNIE language model and a stacked Transformer encoder. The study established three primary empirical questions: whether knowledge-enhanced pre-training surpasses general-purpose BERT representations in financial text classification, whether a lightweight multi-layer Transformer classifier can curb overfitting in data-constrained settings, and whether parameter decoupling can slash computational overhead while preserving competitive analytical performance. Rather than subjecting the entire 110-million-parameter backbone to backpropagation, the researchers opted to freeze ERNIE completely, employing it exclusively as a static high-dimensional feature extractor. Task adaptation was delegated to a separate, compact Transformer encoder responsible for performing secondary discriminative feature extraction on frozen sentence vectors.
Constructing a dependable experimental foundation required assembling an authentic, rigorously annotated financial dataset. The research team gathered raw commentary from Eastmoney Guba focusing on the Hang Seng Pharmaceutical Exchange-Traded Fund and its constituent equities during a market fluctuation window spanning May 12, 2023, through March 20, 2026. Data hygiene protocols filtered out non-substantive digital artifacts, resulting in 6,000 cleaned comment records displaying an approximate categorical distribution of 35 percent positive, 30 percent neutral, and 35 percent negative remarks. Each database record retained minute-level posting timestamps, user view tallies, reply counts, and comment titles spanning between 0 and 44 characters. To establish ground truth labels across the corpus, the authors utilized the Chinese Financial Sentiment Dictionary, a domain-tailored lexicon containing 1,764 positive expressions and 2,816 negative terms, assigning polarities based on dominant emotional lexical counts while labeling mixed or term-deficient records as neutral.
Because automated dictionary labeling can introduce systematic errors, the researchers implemented validation protocols to verify annotation consistency across the collection. A random subset of 500 samples was independently inspected and re-annotated by two domain specialists in financial economics. Statistical verification revealed an agreement metric of kappa equals 0.742 between the automated dictionary assignments and human annotations, alongside an inter-annotator reliability coefficient of kappa equals 0.812 between the two human specialists, confirming acceptable label dependability. The cleaned raw strings then passed through a structured data preparation sequence designed to eliminate extraneous noise. Regular expressions removed punctuation marks, non-standard digital characters, and platform symbols, preserving only Chinese ideograms, English characters, and numerical digits. Missing fields were systematically standardized to empty strings to maintain structural uniformity across downstream processing tensors.
Lexical segmentation was conducted using the Jieba Chinese tokenization engine operating in its precise segmentation mode, converting continuous ideographic strings into structured arrays of individual words. To suppress statistical interference from high-frequency function words devoid of sentiment weight, the text was filtered against the Harbin Institute of Technology Stop Words List, which includes more than 1,200 functional lexical entries. This filtering suppressed non-informative grammatical particles such as structural markers while preserving semantically critical nouns, verbs, and financial modifiers. Subsequent sequence length distribution profiling across all 6,000 comment titles revealed that 99.6 percent of the tokenized samples contained 18 tokens or fewer. Based on this observation, the maximum sequence length was standardized to 18 units, truncating longer entries at the 18th token and appending special padding tokens to shorter strings to guarantee dimensionality alignment.
Semantic vectorization was carried out using Baidu ERNIE-Text-Embedding, converting the standardized text into 1,024-dimensional dense vectors processed in batches of 16 samples. During embedding generation, title strings were mapped into subword vocabularies bracketed by standard classification and boundary tokens, with the final hidden state of the classification token serving as a comprehensive sentence summary. The embedding architecture synthesized token vectors, learnable absolute position embeddings encoding sequence order, and sentence embeddings capturing discourse relationships. Unlike BERT, which relies predominantly on masking independent token characters, ERNIE integrates phrase-level and entity-level masking during initial pre-training. This architectural foundation enables the model to encode compound market terminology and metaphorical vernacular common in Chinese investment circles, including idioms referencing sudden price collapses, loss cutting, and position building that cannot be deciphered from isolated component characters.
The downstream classification framework comprises an input projection layer, a stacked Transformer feature adaptation layer, and a classification output tier. Because the 1,024-dimensional vector produced by the frozen ERNIE backbone exceeds the operating width of the lightweight classifier, the input projection layer applies a trainable linear matrix and bias term to project embeddings into the classifier hidden space. The Transformer feature adaptation layer receives this representation transformed into an explicit sequence shape of batch size by one by hidden dimension. Rather than performing token-level sequence tracking, the multi-layer stacked encoder applies multi-head self-attention, feed-forward sub-layers, residual connections, and layer normalization to execute non-linear feature reweighting directly on the sentence-level representation. An output fully connected layer subsequently projects the adapted features into three logit values, which a Softmax operator converts into discrete categorical probabilities.
Model hyperparameters were calibrated using a systematic grid search across 64 structural permutations encompassing attention head counts, Transformer layer quantities, hidden dimensions, dropout rates, learning rates, and batch sizes. Candidate configurations underwent five-fold cross-validation on an AMD Radeon Vega 8 graphics processor with 8 gigabytes of video memory, requiring approximately eight hours of computation. Training routines incorporated an early-stopping mechanism that halted updates whenever validation performance plateaued for 10 consecutive epochs, alongside a ReduceLROnPlateau scheduler that halved the learning rate if validation loss failed to drop over three consecutive cycles. The optimized pipeline was constructed in Python utilizing the PyTorch deep learning framework, implementing categorical cross-entropy loss and the Adam optimization algorithm under a fixed data allocation strategy distributing 60 percent to training, 20 percent to validation, and 20 percent to evaluation.
Empirical comparisons confirmed the statistical superiority of the ERNIE-Transformer pipeline over eight alternative baseline architectures combining BERT or ERNIE backbones with feedforward, Bi-GRU, CNN-BiLSTM, and Transformer classifiers. Across all network configurations, ERNIE representations consistently outperformed BERT equivalents. When paired with identical Transformer encoders, ERNIE delivered a 92.75 percent accuracy and an 88.12 percent F1-score, surpassing the BERT-Transformer baseline by 1.67 percentage points in accuracy, 3.35 points in precision, 3.01 points in recall, and 2.11 points in F1-score. When measured against full-parameter ERNIE fine-tuning, the decoupled model demonstrated an accuracy advantage of 0.17 percentage points, a precision gain of 0.70 points, and an F1 improvement of 0.20 points, with fully fine-tuned ERNIE maintaining a slight 0.25-point edge in recall. Bootstrap 95 percent confidence intervals affirmed that the decoupled architecture retained equivalent predictive stability to full fine-tuning while operating with only 11.4 percent of the trainable parameters.
Confusion matrix evaluations provided detailed insight into class-specific error distributions, showing that the decoupled ERNIE-Transformer configuration mitigated the pronounced class bias that impaired the fully fine-tuned baseline. Under full fine-tuning, the mismatch between 110 million trainable parameters and 6,000 training examples caused the network to memorize noise, distorting neutral sample identification. By freezing the pre-trained weights, the decoupled framework prevented catastrophic forgetting of broader linguistic patterns while enabling the multi-head attention mechanism to emphasize sentiment-salient signals. Despite these gains, the authors acknowledged key technical constraints, noting that the model functions as a black-box system without internal interpretability, which limits its transparency in risk-sensitive trading environments. Furthermore, because the dataset relied exclusively on textual comments, the model omitted multimodal cues such as emojis, images, and embedded videos, pointing to explainable machine learning frameworks and cross-modal alignment systems as necessary areas for future research.
Subject of Research: Sentiment classification of financial forum text using parameter-decoupled neural architectures
Article Title: Sentiment classification of stock forum text based on a parameter-decoupled ERNIE-Transformer architecture
Article References: Xiumei, L., Fei, C., Wenchao, L., & Kun, C. (2026). Sentiment classification of stock forum text based on a parameter-decoupled ERNIE-Transformer architecture. Journal of Electrical Systems and Information Technology, 13(1), Article 96. https://doi.org/10.1186/s43067-026-00399-x
Image Credits: AI Generated
DOI: 10.1186/s43067-026-00399-x
Keywords: financial sentiment analysis, ERNIE model, Transformer encoder, parameter decoupling, Eastmoney Stock Forum, natural language processing, text classification, sentence embeddings, feature extraction, overfitting mitigation, CFSD dictionary, Jieba tokenization, hyperparameter optimization, Hang Seng Pharmaceutical ETF, confusion matrix analysis, deep learning
Cite Scienmag News
Daphne Blevins. (November 20, 2026). Decoupled Language Model Classifies Stock Forum Sentiment With High Accuracy. Scienmag. https://scienmag.com/decoupled-language-model-classifies-stock-forum-sentiment-with-high-accuracy/
Daphne Blevins. "Decoupled Language Model Classifies Stock Forum Sentiment With High Accuracy." Scienmag, 20 November 2026, https://scienmag.com/decoupled-language-model-classifies-stock-forum-sentiment-with-high-accuracy/. Accessed 20 November 2026.
Daphne Blevins. "Decoupled Language Model Classifies Stock Forum Sentiment With High Accuracy." Scienmag. November 20, 2026. https://scienmag.com/decoupled-language-model-classifies-stock-forum-sentiment-with-high-accuracy/








