Dynamic instance weighting boosts online learning for multi-cryptocurrency price and trend forecasting

Cryptocurrencies are famous for their volatility, and that volatility has long been both a lure and a nightmare for anyone trying to predict where prices will go next. Now, a team of researchers in Italy and Slovenia has developed a machine learning system that learns in real time from live market data, groups related cryptocurrencies together, and—perhaps most unusually—deliberately pays extra attention to the wildest price swings instead of discarding them as noise. The system, called LEMON (reaL-time wEighted Market predictiON), is described in an open-access paper published in Data Mining and Knowledge Discovery, and it outperformed state-of-the-art competitors on two separate forecasting tasks across 16 of the world’s best-known digital assets.

The central insight behind LEMON is that cryptocurrencies rarely move entirely on their own. Regulatory announcements, technological breakthroughs, macroeconomic shocks and social media sentiment tend to wash over the whole market at once, producing groups of coins that exhibit strikingly similar trends. Rather than treating each cryptocurrency as an isolated prediction problem, the researchers’ method first identifies which assets behave alike, then trains a shared model for each group so that the collective behavior of the cluster informs every individual forecast within it. This multi-target strategy allows the model to exploit temporal correlations that single-coin approaches simply cannot see.

Technically, the grouping phase relies on temporal clustering using the Dynamic Time Warping (DTW) distance measure, a technique well suited to comparing time series that may be offset in time or differ in amplitude and length. LEMON builds a square, symmetric matrix of pairwise DTW distances computed from the most recent window of hourly closing prices, a window set to three months in the team’s experiments, and then applies the K-Medoids clustering algorithm to partition the cryptocurrencies into groups. K-Medoids was chosen partly because it works directly from a precomputed distance matrix and partly because, unlike K-Means, its medoids are actual data points, which makes it computationally efficient. The clustering is refreshed periodically—every five months in the headline configuration—so the groups track the current market rather than fossilized patterns from the past.

Once the clusters are established, the heart of the system begins: continuous online learning over streaming data. Instead of the classic batch paradigm, where a model is trained once on historical data and then deployed, LEMON uses a prequential predict-update cycle. Every new instance arriving from the stream is first used to make a prediction with the current model, and then, once the true value is revealed, the same instance is fed back to update the model. This interleaving lets the system adapt continuously to shifting market conditions, a crucial property in an environment where trading patterns and trends can change within hours.

The genuinely novel ingredient, however, is the dynamic instance weighting scheme. Machine learning models usually treat all observations as equally important, and many traditional methods actively suppress outliers to improve robustness. In cryptocurrency markets, the researchers argue, that instinct is backwards: the extreme values are precisely the events that matter most, because they often mark the onset of abrupt trend changes that traders and analysts most need to capture. LEMON therefore assigns higher weights to instances showing large price variations and lower weights to ordinary ones near the median.

To decide which values count as “extreme,” the method builds on Tukey’s boxplot rule, which flags values outside the range defined by the first and third quartiles and the interquartile range. Because cryptocurrency price distributions are frequently skewed rather than symmetric, LEMON adopts the adjusted boxplot rule, which incorporates a robust skewness statistic known as the medcouple to correct the outlier cutoffs on either side of the distribution. The researchers then construct a continuous weighting function across the data range: the median value receives a baseline weight, values at the outlier cutoffs receive a much higher weight—in the main experiments, ten times higher—and a Piecewise Cubic Hermite Interpolating Polynomial (PCHIP) fills in the weights between boundaries, guaranteeing the monotonicity and convexity the scheme requires. As the function’s behavior adapts after each new batch of instances, the weighting curve visibly reshapes itself around the evolving distribution of price changes.

Operating this scheme on an unbounded stream poses a practical problem: quartiles and skewness statistics are normally computed from complete datasets, but a stream never ends and computer memory is finite. The team’s solution is the t-digest, a probabilistic data structure that estimates rank-based statistics such as quantiles with high accuracy using incremental computation. The t-digest partitions incoming data into small clusters represented by centroids and weights, updating them as each new instance arrives, and it is periodically re-initialized to reflect recent market conditions.

For the learning engine itself, LEMON builds on iSOUP-Tree, an instance-incremental, tree-based method designed for streaming data that attaches a perceptron or mean regressor to each leaf and can predict multiple targets at once. The researchers extended it with a weight-aware variance reduction (WAVR) heuristic for choosing tree splits, which normalizes variance reduction across all target cryptocurrencies in a cluster and accounts for each instance’s weight. They also redefined the method’s grace period—the minimum accumulated training weight a leaf must accumulate before a split is considered—so that nodes rich in highly weighted extreme instances are less prone to further splitting, curbing overfitting without sidelining those critical cases during tree construction.

The empirical evaluation drew on three years of hourly data, from January 2020 to December 2022, for sixteen cryptocurrencies including Bitcoin, Ethereum, Litecoin, Dogecoin, Monero and Ripple, sourced from Yahoo! Finance. Each dataset included standard price and volume features augmented with 48 engineered financial indicators—moving averages, Bollinger Bands, momentum oscillators, MACD variants, stochastic oscillators and volume-weighted average price, computed over multiple window lengths—plus a Fear and Greed index capturing market sentiment through volatility, momentum and social signals. The team tested LEMON on two tasks: predicting the percentage variation of the hourly closing price, evaluated by mean squared error, and classifying the next market trend as up, stable or down based on a sensitivity threshold of at least one percent hourly movement, evaluated by F1-score.

The results were decisive. On the regression task, the pretrained variant of LEMON, using the sum weight aggregation strategy, achieved the best mean squared error on 11 of the 16 cryptocurrencies and delivered error reductions ranging from 4 to 63 percent against the strongest online competitors, with even larger gains over batch baselines. On the classification task, the effect of weighting was especially visible: without instance weighting, LEMON ranked twelfth, but with weighting—particularly the sum strategy, which favors extreme values most strongly—it climbed to the top positions, reaching an average F1-score of 0.576 in its pretrained form and 0.515 in its purely online form, well ahead of all rivals. A sensitivity analysis varying the update interval and boundary weights confirmed that performance remains stable across reasonable parameter settings.

Efficiency numbers matter as much as accuracy for a system meant to run against live market feeds. Because LEMON rests on lightweight tree-based models rather than deep networks, its per-instance prediction and update times, along with its memory footprint, were substantially lower than those of the online LSTM and CARROT-based competitors, making it genuinely deployable in real-time settings. The authors caution, however, that the system still depends on periodic retraining at fixed intervals, so a sudden market crash occurring just after a retraining step could catch the model with stale cluster structures; a future concept-drift detector that triggers retraining on demand is the natural next step. They also note that translating predictive accuracy into economic utility—trading rules, fees, slippage—remains outside the current study’s scope. Still, the message of the work is clear: in a market defined by its extremes, the smartest models may be the ones that refuse to ignore them.