When Numbers Meet Words: Hybrid Data Representations Push Wearable Wellness AI Forward

Smartwatches and fitness trackers have quietly become some of the most prolific data collectors in modern life, logging steps, heart rate, sleep duration, and calorie expenditure around the clock. Yet a persistent gap remains between the sheer volume of numbers these devices produce and the meaningful wellness insights users actually want: Am I stressed? Am I fatigued? Is my sleep quality slipping? A new study published in Machine Learning with Applications tackles this translation problem head-on, asking a deceptively simple question that turns out to be surprisingly difficult: how should raw physiological measurements be represented so that large language models can reason over them effectively?

The research, conducted by Cheng-Huan Yu and Mohammad Masum, introduces a framework called WellFusion that converts continuous wearable measurements into hybrid numerical–semantic representations. Instead of forcing a language model to process bare numbers alone or stripped-down category labels alone, WellFusion presents both simultaneously. A day’s activity might appear in a prompt as “Steps = 11,896 (Active)” or “Sleep = 387 (Fair)” — the raw value preserved alongside a human-readable descriptor. The authors contrast this with two alternatives: WellCat, which discards the numbers and keeps only categorical labels such as Sedentary, Active, or Highly Active, and WellNum, which keeps the raw values and drops the descriptors entirely.

The motivation stems from a fundamental mismatch in how foundation models process information. Large language models are trained on text and operate most naturally on language-compatible representations, yet wearable sensors emit continuous numerical observations with heterogeneous scales, units, and distributions. Categorical descriptors compress these measurements into a vocabulary the model can interpret semantically, but compression inevitably discards within-category differences — two people labeled “Active” may have step counts that differ by thousands, potentially reflecting meaningfully different physiological states. Preserving both channels, the authors hypothesized, might give the model the best of both worlds: fine-grained magnitude information plus contextual interpretation.

The study reformulates wellness prediction as a multi-target classification problem. Rather than predicting a single outcome, the model generates one structured output containing ordered labels — Below Normal, Normal, or Above Normal — for four self-reported wellness states simultaneously: stress, fatigue, sleep quality, and readiness. These labels are derived from the statistical distribution of training-participant scores, meaning “Normal” reflects position relative to the pooled training data rather than any clinical reference range. The authors are careful to note this is an operational modeling choice, not a validated clinical boundary.

The experimental foundation is the publicly available PMData dataset, comprising longitudinal records from 16 participants wearing Fitbit Versa 2 smartwatches in free-living conditions. After participant-level alignment and quality control, the final dataset contains 1,836 daily observations pairing passive wearable measurements with self-reported wellness assessments. Crucially, the study restricts model inputs to passively collected signals — steps, calories, sleep duration, and resting heart rate — deliberately excluding active self-reports to better reflect realistic deployment scenarios. Participants were partitioned exclusively into 10 for training, 2 for validation, and 4 held out for testing, with no participant appearing in more than one group.

The headline results come from a five-day evaluation using OpenAI’s GPT-4o-mini with full fine-tuning. Under the WellFusion representation, the model achieved target-level accuracies of 0.86 for fatigue, 0.83 for stress, 0.72 for sleep quality, and 0.69 for readiness, averaging 0.775 across the four targets. The categorical-only WellCat representation yielded consistently lower figures — 0.83, 0.74, 0.66, and 0.60 respectively, averaging 0.708. Every entry summarizes five repeated runs, and the hybrid representation outperformed the categorical-only version on all four targets in this descriptive comparison.

However, the picture becomes considerably more nuanced — and more honest — in the study’s additional three-day analyses. When the authors compared all three representations under a matched Mistral-7B configuration with LoRA adaptation, the numerical-only WellNum representation actually achieved the highest accuracy (0.596) and exact-profile match (0.129), outperforming WellFusion (0.380 accuracy) and WellCat (0.260). Yet WellFusion led on macro-F1, balanced accuracy, and quadratic weighted kappa, the ordinal-agreement metric. The representation ranking, in other words, depends on which metric and configuration you examine — a finding the authors present with unusual candor rather than burying.

Perhaps most striking is the performance of conventional baselines. A simple majority-class dummy classifier achieved the highest raw accuracy (0.677) among all listed three-day configurations, beating every trained model including the fine-tuned language models. XGBoost posted the best macro-F1 and balanced accuracy among conventional baselines. The authors explicitly state these results “do not support a general accuracy advantage for foundation models” — a refreshing admission in a field often eager to declare LLM supremacy. None of the tested three-day accuracy contrasts reached statistical significance after Holm adjustment, partly because with only four held-out participants, the minimum attainable p-value for the exact sign-permutation test was 0.125, making significance at the conventional 0.05 threshold mathematically unattainable.

The study also probes the fragility of its own categorical mappings. In a sensitivity analysis, shifting each numerical cut point upward by 10 percent caused category reassignment for 79.4 percent of resting heart rate values, 30.9 percent of calorie measurements, 28.9 percent of sleep durations, and 19.5 percent of step counts. This instability underscores that the descriptors — labels like “Normal” or “Excellent” — are operational tokens defining the model’s input vocabulary, not validated clinical states. The authors caution, for instance, that labeling sleep above 510 minutes as “Excellent” must not be read as a claim that longer sleep is always healthier.

On the adaptation side, the study compared Low-Rank Adaptation (LoRA), which trains small adapter parameters while freezing pretrained weights, against full-parameter fine-tuning. On the matched Mistral backbone, full fine-tuning produced higher accuracy and macro-F1 for both representations — WellFusion reached 0.560 accuracy with full fine-tuning versus 0.380 with LoRA — though the authors note that differing optimization schedules prevent attributing the gap to adaptation strategy alone. Zero-shot performance, without any fine-tuning, lagged substantially behind, with Mistral achieving only 0.22 to 0.26 accuracy across representations.

The authors are refreshingly explicit about limitations. The cohort is small — 16 participants, one fixed partition, no participant-level cross-validation. Wellness outcomes are self-reported rather than clinically measured. Complete-case filtering may introduce selection bias. The fatigue-directed SMOTE oversampling procedure inherits non-fatigue labels for synthetic instances without validating that assumption. Sleep duration, a predictor, may account for much of sleep-quality performance, but no feature-removal ablation was performed to disentangle this. The same-day design means results do not demonstrate prediction before measurements were available, and no claims are made about real-time deployment readiness, clinical utility, or computational efficiency.

What emerges is less a victory lap than a careful cartography of an emerging research frontier. The study demonstrates that representation design — how numbers become language — is a first-order modeling decision for LLM-based health analytics, not an afterthought. Hybrid representations that preserve both magnitude and meaning showed descriptive advantages in the five-day GPT-4o-mini comparison and on class-balanced metrics in three-day analyses, but numerical-only inputs won on raw accuracy in the matched LoRA setting. The authors frame their contribution as characterizing representation trade-offs within a fixed evaluation design, explicitly declining to claim statistical superiority, physiological mechanism, or clinical usefulness. In a domain where hype routinely outruns evidence, that restraint may be the study’s most valuable finding — a template for how foundation-model health research should communicate uncertainty while still advancing the field. Future work, they suggest, should prioritize participant-level cross-validation, joint-versus-independent prediction comparisons, sleep-duration removal ablations, and matched no-SMOTE training to establish which representation strategies genuinely generalize beyond a single small cohort.

Subject of Research: Using hybrid numerical and categorical representations of wearable sensor data for large language model-based multi-target wellness classification

Article Title: WellFusion: Hybrid numerical–semantic representations for multi-target wellness classification from wearable measurements

Article References: WellFusion: Hybrid numerical–semantic representations for multi-target wellness classification from wearable measurements. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: wearable devices, large language models, wellness prediction, WellFusion, PMData dataset, fine-tuning, LoRA, health informatics, machine learning, self-reported wellness, GPT-4o-mini, representation learning