The new system, called JEPA-Mamba, is the work of Luyang Dai and Yuwen Li of the School of Instrument Science and Engineering at Southeast University. Its name compresses two of the most consequential ideas in contemporary machine learning. The first is the Joint-Embedding Predictive Architecture, or JEPA, a self-supervised learning strategy that shot to prominence when Meta AI researchers demonstrated it on natural images in 2023. The second is Mamba, a state-space-based sequence model introduced in late 2023 that promises to handle very long data streams with linear computational cost, sidestepping the quadratic bottleneck of conventional attention-based transformers. Dai and Li fused these ingredients with a custom attention module tailored to the peculiar anatomy of a twelve-lead ECG, and the resulting model outperformed established self-supervised baselines across a battery of diagnostic classification tasks.

To understand why this matters, it helps to grasp how JEPA differs from the self-supervised methods that preceded it. Early approaches to learning without labels, such as contrastive learning, worked by showing the network two augmented views of the same recording and teaching it to pull their representations together while pushing apart representations of different samples. This required careful engineering of the augmentations, and for medical signals it is far from obvious which distortions are safe: flip a T-wave or shift a baseline and you may destroy the very clinical information the model needs. JEPA takes a different path. Instead of comparing raw signals, it predicts representations of masked regions in embedding space. In JEPA-Mamba, the authors apply high-ratio contiguous-block masking, hiding large stretches of the waveform, and train an exponential moving average encoder alongside a narrow predictor network that must guess what the hidden segments mean. Because the model operates in abstract feature space rather than pixel-by-pixel reconstruction, it learns high-level diagnostic structure without any hand-crafted augmentation, a property the authors argue is crucial for medical time series.

The second pillar, the bidirectional Mamba encoder, addresses a fundamental tension in ECG analysis. A twelve-lead recording sampled over ten seconds, and often much longer in clinical practice, is a long sequence, and transformers that process it with full self-attention become computationally expensive as length grows. Recurrent networks process sequences cheaply but struggle to retain distant dependencies. Mamba, built on selective structured state spaces, scans a sequence with input-dependent dynamics that let it filter and remember information selectively, achieving linear-time processing that scales gracefully. Dai and Li run Mamba in both directions, so the model can integrate context from earlier and later parts of the cardiac cycle simultaneously. They enhance this backbone with rotary position embeddings, which encode temporal position in a way that respects relative offsets, and a register mechanism that helps the network organize global information, then couple the state-space modeling with convolutional waveform extraction that captures fine-grained morphological features of P waves, QRS complexes, and T waves.

Perhaps the most domain-aware component is the channel-temporal attention module, or CTAM. A standard clinical ECG is not one signal but twelve simultaneous recordings, each lead viewing the heart’s electrical activity from a different spatial angle, and different leads carry different diagnostic weight for different conditions. CTAM recalibrates the contributions of the twelve leads adaptively, borrowing ideas from squeeze-and-excitation and convolutional block attention designs, while also focusing temporal attention on the clinically salient segments of the recording. In other words, the model learns both which leads matter and which moments in time matter, rather than treating the full multi-lead tensor as a homogeneous block of numbers. Attention maps produced in the study suggest the trained model concentrates on the regions cardiologists would also scrutinize.

The evaluation was deliberately stringent. The team pretrained the encoder on MIMIC-IV-ECG, a large public dataset of diagnostic ECGs matched to clinical records, and then transferred the learned representations to PTB-XL, the largest publicly available twelve-lead ECG benchmark, using a strict patient-level split so that no patient appeared in both training and test sets. This cross-dataset setup is a demanding test of generalization, because the recordings come from different devices, patient populations, and labeling conventions. On the fully annotated PTB-XL tasks, JEPA-Mamba achieved macro-averaged area under the ROC curve of 0.9296 plus or minus 0.002 and 0.9306 plus or minus 0.002, scores that place it ahead of prior self-supervised approaches on most downstream tasks. Crucially, it beat the most direct JEPA-based competitor in the ECG domain across every task tested, and the improvements over external baselines on the super-class, subclass, and diagnostic-level tasks were statistically significant.

The authors did not rely on headline numbers alone. They conducted ablation studies to quantify the contribution of each architectural component, confirming with DeLong tests that the differences observed when removing the bidirectional design, the register mechanism, or the attention module were statistically meaningful rather than noise. Per-class performance analyses revealed where the model excelled and where it struggled, while t-SNE visualizations of the learned embeddings showed diagnostic categories separating into coherent clusters, evidence that the representations encode clinically relevant structure even before fine-tuning. Robustness experiments exposed the model to noise and baseline wander, two pervasive artifacts of real-world ECG acquisition, and demonstrated that its performance degrades gracefully, an important property for deployment outside pristine laboratory conditions.

The significance of this work extends beyond one benchmark. Self-supervised pretraining on large unlabeled archives addresses the labeled-data bottleneck that has slowed clinical AI adoption, and the cross-dataset results suggest the learned features are not merely tailored to one recording environment. Because Mamba operates in linear time, the architecture is well suited to the long, continuous streams produced by wearable monitors and intensive-care telemetry, where transformer-based models become impractical. The pipeline also requires no hand-crafted augmentations, removing a layer of human guesswork from the training recipe. Together these properties sketch a realistic path toward foundation-model-style ECG encoders that hospitals could adapt to their own populations with modest local fine-tuning.

Caveats remain, and the authors are careful to frame their findings as preliminary evidence rather than a finished clinical tool. The study used only public, de-identified datasets, so the model has not yet faced the messier distributions of routine hospital practice, and regulatory validation, prospective trials, and integration with clinical workflows all lie ahead. Nonetheless, the combination of JEPA-style predictive learning, bidirectional state-space modeling, and channel-temporal attention represents a coherent template for physiological signal representation, one that could plausibly extend to electroencephalography, arterial pressure waveforms, and other multichannel biosignals where labels are scarce and sequences are long.

What makes the story compelling is its timing. Self-supervised learning transformed computer vision and natural language processing over the past half-decade, and medicine has been waiting for an equivalent breakthrough that respects the constraints of clinical data. JEPA-Mamba shows that the joint-embedding predictive paradigm, when rebuilt around the physics and cardiology of the heartbeat rather than borrowed wholesale from images, can deliver state-of-the-art diagnostic representations without a single expert label during pretraining. If the approach generalizes as its cross-dataset results suggest, the machines that watch over our hearts may soon learn their craft the way residents once did: by observing vast numbers of patients, guided only by the signals themselves.

Subject of Research: Self-supervised deep learning for electrocardiogram representation and classification

Article Title: JEPA-Mamba: a self-supervised joint-embedding architecture with bidirectional Mamba for ECG representation learning

Article References: Dai, L., & Li, Y. (2026). JEPA-Mamba: a self-supervised joint-embedding architecture with bidirectional Mamba for ECG representation learning. Biomedical Engineering Letters. https://doi.org/10.1007/s13534-026-00620-w

Image Credits: AI Generated

DOI: 10.1007/s13534-026-00620-w

Keywords: electrocardiogram, self-supervised learning, JEPA, Mamba, state space models, attention mechanism, PTB-XL, MIMIC-IV-ECG, transfer learning, cardiovascular disease, biomedical engineering, ECG classification