<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>data preprocessing &#8211; Science</title>
	<atom:link href="https://scienmag.com/tag/data-preprocessing/feed/" rel="self" type="application/rss+xml" />
	<link>https://scienmag.com</link>
	<description></description>
	<lastBuildDate>Sat, 17 Oct 2026 05:34:56 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg</url>
	<title>data preprocessing &#8211; Science</title>
	<link>https://scienmag.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">73899611</site>	<item>
		<title>Open-Source AI Tool Brings Machine Learning to the Hunt for Life Beyond Earth</title>
		<link>https://scienmag.com/open-source-ai-tool-brings-machine-learning-to-the-hunt-for-life-beyond-earth/</link>
		
		<dc:creator><![CDATA[Teresa Odom]]></dc:creator>
		<pubDate>Sat, 17 Oct 2026 05:34:56 +0000</pubDate>
				<category><![CDATA[Science News]]></category>
		<category><![CDATA[accessible AI for planetary science]]></category>
		<category><![CDATA[AI tools for extraterrestrial life detection]]></category>
		<category><![CDATA[AI-powered analysis of alien biosignatures]]></category>
		<category><![CDATA[astrobiology]]></category>
		<category><![CDATA[biosignatures]]></category>
		<category><![CDATA[data preprocessing]]></category>
		<category><![CDATA[democratizing access to AI in space exploration]]></category>
		<category><![CDATA[explainable AI]]></category>
		<category><![CDATA[graphical user interface]]></category>
		<category><![CDATA[Hugging Face]]></category>
		<category><![CDATA[Hugging Face AI hosting for space applications]]></category>
		<category><![CDATA[Machine learning]]></category>
		<category><![CDATA[machine learning in astrobiology research]]></category>
		<category><![CDATA[NASA]]></category>
		<category><![CDATA[NASA and Virginia Tech open-source]]></category>
		<category><![CDATA[open-source AI platform for space scientists]]></category>
		<category><![CDATA[Open-source machine learning for astrobiology]]></category>
		<category><![CDATA[open-source software]]></category>
		<category><![CDATA[rapid prototyping AI software for astrobiology]]></category>
		<category><![CDATA[scientific education]]></category>
		<category><![CDATA[supervised learning]]></category>
		<category><![CDATA[unsupervised learning]]></category>
		<category><![CDATA[user-friendly machine learning tools for educators and students]]></category>
		<category><![CDATA[web-based AI interface for space research]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=294214</guid>

					<description><![CDATA[Researchers at Virginia Tech and NASA Ames have launched AI², a free open-source web tool that lets astrobiologists train and interpret explainable machine learning models on tabular data without writing code.]]></description>
										<content:encoded><![CDATA[<p>The search for life beyond Earth has always been a discipline of extremes: vast telescopes peering at distant atmospheres, spacecraft drilling into alien analog sediments, and laboratories teasing chemical signatures from rocks billions of years old. What it has rarely been, until now, is accessible to the average researcher who wants to apply machine learning but lacks the programming chops to do so. That gap is what a team of scientists from Virginia Tech and NASA Ames Research Center set out to close with a new open-source tool called Artificial Intelligence for Astrobiology Investigations, or AI², described in a paper published in Frontiers in Astronomy and Space Sciences.</p>
<p>The software, built by Floyd Nichols and Grant Major of Virginia Tech&#8217;s Department of Geosciences and Ryan Felton of NASA Ames, is a web-based graphical user interface that lets educators, students, and researchers train and interpret machine learning models without writing a single line of code. It is hosted on Hugging Face, the popular open-source machine learning platform, and runs on two virtual CPUs with 16 gigabytes of RAM. The team describes it as an entry point for novices and a rapid prototyping tool for experts, a way to standardize the machine learning workflow across a community that has historically approached it in ad hoc ways.</p>
<p>The need for such a tool is not abstract. Within the last five years, machine learning has crept into nearly every corner of astrobiology: analyzing laboratory data on biosignatures, processing remote sensing and telescope observations, running simulations of planetary environments, and even enabling autonomous decision-making on space missions. NASA&#8217;s own strategic documents have repeatedly called for broader adoption. The agency&#8217;s Science Mission Directorate Strategy for Data Management and Computing recommended incentivizing and educating the community on AI and machine learning approaches, while the Origins, Worlds, and Life decadal survey explicitly endorsed machine learning for supporting autonomy on remote planetary missions. The NASA ROSES program has signaled the same need through dedicated solicitations for Solar System Science and Foundational Artificial Intelligence for the Moon and Mars.</p>
<p>Yet the barrier to entry remains steep. Machine learning algorithms, even those available as off-the-shelf functions in Python or R, demand a working knowledge of both the underlying mathematics and the programming language itself. Many astrobiologists are domain experts in geology, chemistry, or biology but have neither the time nor the institutional support to become competent programmers. This is precisely the gap AI² is designed to fill, offering a guided pipeline that walks users through the three canonical stages of a machine learning project: data preprocessing, exploratory data analysis with unsupervised learning, and supervised learning for prediction.</p>
<p>The preprocessing stage handles the unglamorous but critical work that determines whether any downstream model will be trustworthy. Users upload a CSV file, and the application immediately computes Pearson correlation coefficients to reveal pairwise linear relationships between variables and variance inflation factors to flag multicollinearity, a statistical pathology in which highly correlated predictors can silently degrade a model&#8217;s ability to generalize. The tool then offers options for handling missing values through removal or imputation using mean, median, or most-frequent strategies, encoding categorical variables into numerical form, and scaling features via min-max normalization or z-score standardization. These steps matter because many algorithms implicitly assume features are centered around zero with comparable variance; a single feature with orders of magnitude more spread can dominate the model and prevent it from learning anything useful from the others.</p>
<p>Once the data is clean, the unsupervised learning module offers dimensionality reduction and clustering techniques chosen specifically because they have appeared in prior astrobiology research. For dimensionality reduction, users can apply Principal Component Analysis, which captures maximum variance between features through linear transformation; Multidimensional Scaling, which instead preserves pairwise distances between data points; or t-Distributed Stochastic Neighbor Embedding, a nonlinear manifold learning method that preserves local structure and is particularly useful for visualizing high-dimensional data in two dimensions. For clustering, the options include K-means, which partitions data into k groups of equal variance around centroids; Gaussian Mixture Models, which identify clusters based on probabilistic Gaussian densities; and DBSCAN, which can detect clusters of arbitrary shape, size, and density without requiring the user to predefine how many groups exist.</p>
<p>The supervised learning module covers both classification and regression tasks. Classification options include Logistic Regression, k-Nearest Neighbors, Support Vector Classifier, and Random Forest, while regression offerings include Random Forest Regressor, Histogram Gradient Boosting, Support Vector Regressor, and Ridge Regression. After training, the interface produces standard evaluation metrics: accuracy, precision, recall, and F1 score for classification, and mean squared error, root mean squared error, mean absolute error, and mean absolute percentage error for regression. Cross-validation is incorporated to assess generalizability, and the tool generates interactive visualizations including confusion matrices for classification tasks and predicted-versus-actual crossplots for regression. Crucially, it also produces feature importance plots based on Shapley Additive Explanations, a technique that attributes each prediction to individual input features, keeping the entire workflow within the realm of explainable rather than black-box AI.</p>
<p>What distinguishes AI² from similar tools is its built-in statistical guardrails. The developers recognized that even experienced researchers sometimes select models inappropriately because they skip the diagnostic tests that reveal whether a dataset&#8217;s assumptions match a given algorithm&#8217;s requirements. The application therefore runs Henze-Zirkler&#8217;s test for multivariate normality, the Breusch-Pagan test for heteroscedasticity, and isolation forest analysis to quantify outlier rates, all under the hood. If a user selects a model that is statistically inappropriate for their data, the tool issues a warning and suggests a more robust alternative. For example, ridge regression, often chosen for its speed and resistance to multicollinearity, breaks down when data contains high outlier rates, heteroscedasticity, or non-normal distributions; in such cases the interface recommends tree-based models like random forest or histogram gradient boosting instead.</p>
<p>The team benchmarked the web-hosted version across datasets ranging from 100 to 10,000 observations with 10 features, sizes chosen to represent typical astrobiological datasets from biological measurements to stellar spectral catalogs. Most model scenarios completed in seconds to tens of seconds. The outlier was t-SNE with 10,000 observations, which averaged 194 seconds, a reflection of that algorithm&#8217;s inherent computational intensity rather than any flaw in the implementation.</p>
<p>The developers are candid about the limitations. The algorithm menu is deliberately restricted to methods most relevant to astrobiology, and neural networks are excluded for now because of their black-box nature, though simple artificial neural networks are planned for future versions. The team also acknowledges that large language models like ChatGPT and Claude can help novices write machine learning code, but argues that this path is fraught: LLMs require well-crafted prompts that themselves demand machine learning understanding, and studies have shown that novices using LLMs frequently make inappropriate model selections, misuse data, and skip systematic hyperparameter tuning, errors that then get baked into the training data of future models. Compared with existing tools like Google&#8217;s Teachable Machine, which handles only image and audio recognition, or Orange Data Mining, which requires moderate machine learning knowledge and lacks statistical assumption testing, AI² is purpose-built for tabular astrobiology data with guardrails baked in. Future plans include video tutorials and integration with the AI-Astrobiology portal at NASA Ames, a community resource that hosts seminars, literature reviews, and tutorial guides. The broader hope is that by lowering the technical barrier, the tool will help a generation of astrobiologists treat machine learning not as an arcane specialization but as a standard part of the scientific toolkit, one that complements rather than replaces human domain expertise in the ongoing search for life in the universe.</p>
<p><strong>Subject of Research:</strong> An open-source graphical user interface for applying explainable machine learning to astrobiology datasets</p>
<p><strong>Article Title:</strong> Artificial intelligence for astrobiology investigations (AI2): an open-source graphical user interface</p>
<p><strong>Article References:</strong> Nichols, F., Major, G., &amp; Felton, R. (2026). Artificial intelligence for astrobiology investigations (AI2): an open-source graphical user interface. <em>Frontiers in Astronomy and Space Sciences, 13</em>, Article 1933473. <a href="https://doi.org/10.3389/fspas.2026.1933473" rel="noopener noreferrer">https://doi.org/10.3389/fspas.2026.1933473</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.3389/fspas.2026.1933473" rel="noopener noreferrer">10.3389/fspas.2026.1933473</a></p>
<p><strong>Keywords:</strong> astrobiology, machine learning, open-source software, graphical user interface, NASA, biosignatures, data preprocessing, unsupervised learning, supervised learning, explainable AI, Hugging Face, scientific education</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">294214</post-id>	</item>
		<item>
		<title>Dueling Imputations: Deterministic Framework Sharpens AI Time-Series Forecasting</title>
		<link>https://scienmag.com/dueling-imputations-deterministic-framework-sharpens-ai-time-series-forecasting/</link>
		
		<dc:creator><![CDATA[Blake Davidson]]></dc:creator>
		<pubDate>Sun, 13 Sep 2026 02:45:58 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[AI forecasting]]></category>
		<category><![CDATA[AI robustness in incomplete data]]></category>
		<category><![CDATA[data imputation for predictive modeling]]></category>
		<category><![CDATA[data preprocessing]]></category>
		<category><![CDATA[deep learning]]></category>
		<category><![CDATA[deterministic data repair methods]]></category>
		<category><![CDATA[deterministic imputation]]></category>
		<category><![CDATA[evaluating imputation techniques]]></category>
		<category><![CDATA[forecasting performance-driven data filling]]></category>
		<category><![CDATA[forecasting reliability]]></category>
		<category><![CDATA[GRU]]></category>
		<category><![CDATA[innovative imputation frameworks]]></category>
		<category><![CDATA[LSTM]]></category>
		<category><![CDATA[Machine learning]]></category>
		<category><![CDATA[missing data]]></category>
		<category><![CDATA[missing data handling in AI]]></category>
		<category><![CDATA[PM2.5]]></category>
		<category><![CDATA[RNN]]></category>
		<category><![CDATA[sensor data imputation]]></category>
		<category><![CDATA[sensor data reliability]]></category>
		<category><![CDATA[sensor network data gaps]]></category>
		<category><![CDATA[time-series continuity restoration]]></category>
		<category><![CDATA[time-series forecasting accuracy]]></category>
		<category><![CDATA[time-series imputation]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=200972</guid>

					<description><![CDATA[Researchers have developed a deterministic duel-based imputation framework that selects missing-data replacements by directly testing candidates against forecasting performance, cutting forecasting error by up to 70 percent.]]></description>
										<content:encoded><![CDATA[<p>Missing data is one of the oldest and most stubborn enemies of artificial intelligence. In sensor networks that monitor air quality, energy grids, hospitals, and industrial plants, readings vanish for all sorts of mundane reasons: a sensor fails, a transmission drops, a report arrives late, or extreme weather interrupts collection. For time-series forecasting systems, those gaps are more than an inconvenience. They break the temporal continuity on which predictive models depend, and the way engineers fill them can quietly determine whether a forecast is trustworthy or wildly off the mark. A new study published in Discover Informatics argues that the fix lies not in smarter models but in smarter, and strikingly disciplined, data repair.</p>
<p>Researchers led by Agung Bella Putra Utama, Aji Prasetya Wibawa, and Anik Nur Handayani of Universitas Negeri Malang, together with Andrew Nafalski of Adelaide University, have introduced a deterministic duel-based imputation framework that puts forecasting performance itself in charge of deciding how missing values should be filled. The central insight is deceptively simple: most imputation methods are judged by how statistically similar their reconstructions are to the original data, not by whether they actually help a forecasting model make better predictions. A filled-in series can look numerically plausible on paper and still wreck a downstream forecast by smoothing away the very peaks and rhythms the model needs to learn.</p>
<p>The framework works like a structured tournament. For every missing observation, the system generates a set of candidate values using familiar baseline techniques, including mean, median, and mode substitution, K-Nearest Neighbors, Multiple Imputation by Chained Equations, and Last-Observation-Carried-Forward. Each candidate is then scored with a composite loss function that combines three complementary forecasting metrics: Mean Absolute Percentage Error, which captures proportional error; Root Mean Square Error, which penalizes large absolute deviations; and the coefficient of determination, or R-squared, which measures how much of the variance in the real series the reconstruction explains. The weights are set at 0.4, 0.4, and 0.2 respectively, so error-based criteria dominate while explanatory power still plays a supporting role.</p>
<p>What separates this approach from conventional candidate ranking is the way winners are chosen. Instead of collapsing every candidate into a single aggregated score and picking the top one, the framework stages deterministic pairwise duels. Candidates face each other one-on-one, and a sensitivity threshold of 0.001 prevents negligible score differences from flipping outcomes. After all comparisons, candidates are ranked by cumulative wins, and the top three advance to a final aggregation stage. Four aggregation modes are available: averaging the survivors for low-variance stability, taking the median for outlier robustness, taking the maximum to emphasize peak responsiveness, or a winner-take-all selection of the candidate with the lowest forecasting loss. Because every step follows fixed rules, the same input always produces the same output, a property the authors argue is essential for auditing, validation, and accountability in operational systems.</p>
<p>The team evaluated the framework on the Beijing PM2.5 dataset, a widely used benchmark of hourly meteorological and air-pollution measurements collected between 2010 and 2014. After cleaning and temporal alignment, 41,776 valid records remained out of 43,800, with roughly 2,068 missing values concentrated in the PM2.5 variable itself. Crucially, the missingness is not random: gaps cluster in cold seasons and during periods of rapid atmospheric change, exactly the conditions where forecasting matters most and where naive imputation does the most damage. The dataset includes dew point, temperature, pressure, cumulative wind speed, snowfall, rainfall, and temporal indicators as explanatory variables, with PM2.5 concentration serving as the prediction target.</p>
<p>The imputed datasets were then fed into four recurrent forecasting architectures trained under identical conditions: a vanilla Recurrent Neural Network, a Long Short-Term Memory network, a Bidirectional LSTM, and a Gated Recurrent Unit. Hyperparameters for all models were tuned with Particle Swarm Optimization, though the researchers are careful to note that this stochastic tuning happens entirely outside the imputation stage and does not compromise its determinism. The optimized configurations converged on surprisingly lightweight designs: three hidden layers of 24 neurons each, sigmoid activations, the Adam optimizer with mean squared error loss, a batch size of 32, 46 training epochs, and a dropout rate of 0.2.</p>
<p>The results are striking. Compared with conventional imputation methods, the duel-based framework reduced MAPE by up to 70 percent and RMSE by 12 to 15 percent, while maintaining R-squared values above 0.95 across all four architectures. The Mean aggregation variant delivered the strongest overall performance, preserving both proportional variation and the amplitude structure of the signal. Dropping missing observations entirely, by contrast, produced the worst forecasts, confirming that simply discarding incomplete rows destroys the temporal coherence recurrent models rely on. Even BRITS, a sophisticated deep-learning imputation method, showed less consistent gains, suggesting that reconstruction quality alone does not guarantee forecasting quality.</p>
<p>Statistical testing reinforced the case. Paired t-tests and Wilcoxon signed-rank tests comparing the proposed Mean variant against KNN, the strongest conventional baseline, produced p-values below 0.01 across all architectures, with each experiment repeated ten times under fixed random seeds. Visual analysis of predicted versus observed PM2.5 trajectories showed close alignment across smooth and volatile segments alike, with the Bi-LSTM achieving the highest R-squared values and the GRU delivering the most efficient runtime while matching LSTM accuracy. The framework&#8217;s computational overhead scales quadratically with the number of candidates but linearly with data size, and because the candidate set is small and fixed, runtimes remain predictable even as data volume grows.</p>
<p>To test generalizability, the researchers extended the evaluation beyond air quality to three additional domains: the Heart Disease dataset, which has weak temporal structure; the KEDS e-journal dataset, which shows irregular behavioral patterns and moderate sparsity; and the Sunspot dataset, which exhibits strong periodic behavior. The framework achieved lower MAPE and RMSE than baselines including drop-missing, mean imputation, KNN, and BRITS across all of them, with the largest gains appearing in datasets with strong sequential patterns such as Sunspot. The authors acknowledge limitations, including the quadratic cost of pairwise comparison for large candidate pools and the fixed forecasting horizons tested, and they point to cluster-based candidate reduction and online learning as future directions. But the broader message stands: imputation should not be a passive preprocessing chore. Treated as a forecasting-driven decision process, it becomes a strategic lever for building AI systems that are not only accurate but consistent, transparent, and worthy of trust.</p>
<p><strong>Subject of Research:</strong> A deterministic duel-based imputation framework that integrates forecasting performance metrics into missing-data selection for reliable AI-driven time-series forecasting.</p>
<p><strong>Article Title:</strong> A deterministic duel-based imputation framework for reliable AI-driven time-series forecasting</p>
<p><strong>Article References:</strong> Utama, A. B. P., Wibawa, A. P., Handayani, A. N., &amp; Nafalski, A. (2026). A deterministic duel-based imputation framework for reliable AI-driven time-series forecasting. <em>Discover Informatics, 1</em>(1), Article 3. <a href="https://doi.org/10.1007/s44564-026-00001-6" rel="noopener noreferrer">https://doi.org/10.1007/s44564-026-00001-6</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44564-026-00001-6" rel="noopener noreferrer">10.1007/s44564-026-00001-6</a></p>
<p><strong>Keywords:</strong> time-series imputation, missing data, AI forecasting, deterministic imputation, deep learning, LSTM, GRU, RNN, PM2.5, forecasting reliability, machine learning, data preprocessing</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">200972</post-id>	</item>
	</channel>
</rss>
