To overcome these experimental barriers, researchers have introduced an end-to-end computational pipeline that unifies four core image-processing operations: denoising, atomic localization, classification, and instance segmentation. Documented in the Journal of Analytical Science and Technology, the methodology integrates supervised neural networks with established signal-processing algorithms to process micrographs acquired under low-dose conditions. By coupling a UNet3+ convolutional network with the Segment Anything Model 2 foundation model and deterministic algorithms, the workflow extracts atomic coordinates, column identities, and individual column boundary masks. The authors demonstrated that their pipeline successfully processes simulated benchmarks and experimental micrographs across multiple distinct crystallographic systems without manual parameter readjustment between specimens.
The automated framework addresses fundamental physical and methodological constraints inherent to atomic-resolution transmission electron imaging. However, lowering the electron dose per pixel increases Poissonian shot noise across the recorded field of view. Simultaneously, non-dose-dependent distortions stemming from thermal sample drift, scan-coil instability, and line-dwell timing variations distort geometric fidelity. Conventional quantitative evaluations rely heavily on manual procedures, such as operator-dependent thresholding, subjective filtering adjustments, and iterative peak fitting. These fragmented steps introduce human bias, impede analytical reproducibility across laboratories, and create processing bottlenecks as automated microscopes generate massive multi-frame datasets and large-area maps. A standardized, robust analysis route is therefore essential to transform noisy raw micrographs into reproducible atomic-scale measurements.
To construct a reproducible bridge from noisy inputs to atomic-level descriptors, the workflow structures four distinct computational steps into an integrated sequence. The four tasks are designed to be complementary rather than redundant, each targeting a specific analytical dimension. Denoising restores degraded images by suppressing noise while maintaining lattice periodicities and contrast polarity. Atomic localization extracts two-dimensional coordinate positions of atomic centroids with sub-pixel precision, laying the foundation for drift compensation and interatomic distance calculations. Atomic classification subsequently evaluates local intensity profiles around detected centroids to assign categorical elemental or defect labels based on Z-contrast characteristics. Finally, atomic segmentation determines two-dimensional boundary masks for individual atomic columns, providing shape attributes, pixel-wise intensity distributions, and column boundary delineations. These area-resolved metrics furnish essential data for downstream strain mapping, local deformation quantification, and defect characterization that point coordinates alone cannot provide.
Because experimental microscopy acquisitions lack ground-truth references for pristine intensities, coordinate centroids, and column masks, network training utilized the open-source TEMImageNet benchmark. Built through physics-based forward modeling, TEMImageNet calculates atomic scattering potentials and simulates electron probe-sample interactions across varied crystal lattices. Crucially, the simulation incorporates microscope-specific scan distortions, realistic noise distributions, and background irregularities, effectively narrowing the domain gap between synthetic modeling and real experimental acquisitions. The benchmark provides paired annotations including noisy inputs, clean ground-truth images, atomic coordinates, Gaussian maps, and segmentation masks. The researchers trained UNet3+ networks on these paired sets while reserving 400 images as a dedicated validation split to evaluate model accuracy quantitatively. No supplemental data augmentations were applied beyond the realistic noise and distortion profiles present within the benchmark, ensuring an objective, reproducible training protocol.
The pipeline utilizes the UNet3+ architecture as a unified image-to-image translation backbone for both noise suppression and Gaussian likelihood prediction. Unlike classic U-Net models that connect encoder and decoder stages using simple horizontal skips, UNet3+ incorporates full-scale skip connections that combine multiscale feature maps at every decoder level. This architecture fuses fine-grained spatial details, such as high-frequency lattice fringes and column edges, with broader semantic context regarding lattice symmetry and atomic arrangements. The networks were optimized on 256-by-256 pixel image patches with a batch size of 10 using the AdamW optimizer with an initial learning rate of 0.001 and weight decay of 0.0001. Training spanned up to 100 epochs under a cosine annealing schedule with warm restarts and gradient clipping at a maximum norm of 5.0, minimizing an L1 loss function to avoid over-smoothing fine structural boundaries.
Quantitative benchmarking of the UNet3+ denoiser demonstrated substantial gains in signal fidelity and structural preservation across validation micrographs. The denoiser improved peak signal-to-noise ratio from an initial 15.45 decibels in noisy inputs to 38.28 decibels in restored outputs, while the structural similarity index measure rose from 0.369 to 0.957. In contrast, the conventional Block-matching and 3D filtering algorithm attained a peak signal-to-noise ratio of only 21.21 decibels and a structural similarity index of 0.513 on the same validation split. The physical benefit of this denoising was confirmed through line-profile analysis across periodic dumbbell structures.
Following image denoising, atomic column coordinates are determined through a four-stage localization sequence that combines Gaussian filtering, adaptive candidate identification, non-maximum suppression, and sub-pixel refinement. After applying a Gaussian filter with a standard deviation of 0.8 pixels, potential atomic centroids are detected within a 5-by-5 sliding window. A candidate is retained only when its prominence exceeds an adaptive threshold calculated from local standard deviation, governed by a baseline of 6.0, a floor of 1.5, and a scaling factor of 1.2. Redundant detections are removed via non-maximum suppression within a 4-pixel radius, followed by one-dimensional quadratic sub-pixel interpolation along each axis to estimate fractional peak coordinates bounded between negative 0.5 and positive 0.5 pixels. Finally, optional Otsu intensity thresholding eliminates false positive candidates caused by residual background noise, reducing localization error and narrowing coordinate variance.
Atomic classification assigns discrete identities to localized columns by analyzing local intensity profiles extracted from High-Angle Annular Dark-Field images, where intensity scales with atomic number. To prevent signal overlap between adjacent columns, the pipeline dynamically computes an adaptive sensing radius equal to one-third of the mean distance to an atom’s five nearest neighbors, constrained between 2 and 20 pixels. The local mean intensities within these circular masks are grouped using Density-Based Spatial Clustering of Applications with Noise, using an adaptive neighborhood radius defined as one-eighth of the total intensity range with a minimum cluster size of three. In cases where the number of atomic classes is unknown, an ensemble voting scheme determines cluster count by evaluating Sarle’s bimodality coefficient, silhouette scores from k-means clustering, and the Bayesian Information Criterion from Gaussian mixture models, assigning robust categorical labels across the lattice.
To delineate individual atomic column regions without merging closely spaced neighbors, the authors developed a two-stage instance segmentation strategy combining Gaussian probability maps with Segment Anything Model 2. First, a UNet3+ network predicts a Gaussian map that highlights atomic centers as probabilistic peaks against a near-zero background, effectively eliminating substrate noise and uneven background contrast. In the second stage, atomic coordinates extracted from the Gaussian map serve as positive point prompts for SAM2, which is employed as a pretrained foundation model without any fine-tuning on microscopy data. For each prompt, SAM2 generates candidate masks, and the mask exhibiting the highest predicted Intersection over Union is selected. Representing atomic columns as isolated probabilistic signals enables the foundation model to establish accurate instance boundaries and prevent the artificial merging of adjacent atoms in dense crystallographic regions.
Quantitative evaluation on the TEMImageNet validation set confirmed that providing localization-derived point prompts and Gaussian map inputs substantially enhances segmentation precision. Utilizing point prompts over Gaussian likelihood maps elevated the Dice similarity coefficient to 0.8638 and the Intersection over Union to 0.7760, compared to a Dice score of 0.4302 and Intersection over Union of 0.2925 when prompts were applied directly to denoised images. The near-zero background of the Gaussian map effectively prevented SAM2 from misinterpreting subtle background intensity variations as foreground structures. Furthermore, executing SAM2 in its unprompted automatic grid-sampling mode resulted in near-zero overlap, yielding Dice scores of approximately 0.0708 on denoised images and 0.0704 on Gaussian maps. These findings confirm that point prompts are indispensable for guiding vision foundation models at atomic scales, while intermediate Gaussian maps ensure clean separation between neighboring instances.
The practical utility of the complete analysis pipeline was validated on real experimental micrographs acquired using a monochromated, probe aberration-corrected JEOL JEM-ARM200F transmission electron microscope. The pipeline generated qualitatively consistent denoising, coordinate localization, intensity-based classification, and instance segmentation outputs across all three material systems, demonstrating robust cross-sample generalization. The authors noted that the workflow operates reliably when specimen Z-contrast is sufficiently separable and signal levels allow adaptive prominence thresholds to detect peaks accurately. By publishing fixed parameters, processing orders, and open-source models, the study provides a transparent, accessible baseline that enables researchers lacking deep-learning expertise to perform reproducible, automated atomic-scale characterizations of materials.
Subject of Research: Automated quantitative analysis of atomic-resolution scanning transmission electron microscopy images using deep learning
Article Title: Ai-assisted atomic-scale analysis in STEM images
Article References: Lee, S., Lee, T., Kim, M., Kwak, I. H., Lee, T. Y., Jang, J. H., Chung, H.-S., & Lee, S.-C. (2026). Ai-assisted atomic-scale analysis in STEM images. Journal of Analytical Science and Technology, 17(1), Article 35. https://doi.org/10.1186/s40543-026-00558-9
Image Credits: AI Generated
DOI: 10.1186/s40543-026-00558-9
Keywords: Scanning Transmission Electron Microscopy, Atomic-Scale Analysis, Deep Learning, UNet3+, Segment Anything Model 2, TEMImageNet, Image Denoising, Atomic Localization, Atomic Classification, Instance Segmentation, Gaussian Likelihood Maps, DBSCAN Clustering, Peak Signal-to-Noise Ratio, Sub-Pixel Interpolation, Low-Dose Microscopy, Z-Contrast Imaging, Structural Similarity Index
Cite Scienmag News
Blake Davidson. (November 18, 2026). Deep Learning Pipeline Automates Atomic Resolution Electron Microscopy Image Analysis. Scienmag. https://scienmag.com/deep-learning-pipeline-automates-atomic-resolution-electron-microscopy-image-analysis/
Blake Davidson. "Deep Learning Pipeline Automates Atomic Resolution Electron Microscopy Image Analysis." Scienmag, 18 November 2026, https://scienmag.com/deep-learning-pipeline-automates-atomic-resolution-electron-microscopy-image-analysis/. Accessed 18 November 2026.
Blake Davidson. "Deep Learning Pipeline Automates Atomic Resolution Electron Microscopy Image Analysis." Scienmag. November 18, 2026. https://scienmag.com/deep-learning-pipeline-automates-atomic-resolution-electron-microscopy-image-analysis/








