The foundation of the project is Mindat, the sprawling online mineral database that has quietly become the reference shelf of the entire mineralogy community. As of February 2025, Mindat cataloged 6,114 mineral species approved by the International Mineralogical Association, accompanied by nearly 1.41 million high-quality photographs and more than 140 distinct mineral attributes. That ocean of images has always been searchable by name or locality, but the relationships between the pictures and the text describing them remained largely unexploited. The researchers, led by Quanli Fu and Xiang Que of Fujian Agriculture and Forestry University, together with Xiaogang Ma of the University of Idaho and colleagues, saw in those unexplored connections an invitation: if every photograph is tied to a structured description of hardness, luster, cleavage and chemistry, could a machine learn to run the relationship in reverse?

Their answer builds on the OpenMindat project, an effort funded by the U.S. National Science Foundation to expose the Mindat database through a machine-readable interface that complies with the FAIR principles of findability, accessibility, interoperability and reusability. Earlier work in the OpenMindat ecosystem produced R and Python packages, a mobile application, and visualization frameworks for exploring associations among minerals, elements and localities. What had never been attempted, the authors note, was the integration of Mindat’s textual attribute data with its images to generate entirely new mineral pictures. Text-to-image synthesis, the technology behind tools like DALL-E and Midjourney, had already proven itself in medical imaging, remote sensing and design education; nobody had pointed it at the mineral kingdom.

The first task was data engineering, and it was substantial. The team pulled 10 to 20 images for each of the 6,114 IMA-approved mineral species, collecting every available photograph for species with fewer than ten, and ended up with a dataset of 43,745 images, which they have released publicly on Hugging Face. Each image was paired with a standardized textual description assembled from ten attributes drawn from four categories: chemical composition, physical characteristics, crystal system and optical behavior. The template reads like a telegraphic mineralogist’s shorthand: a mineral’s transparency, constituent elements, Mohs hardness range, luster type, streak color, crystal system, cleavage, fracture and optical type are stitched into a single sentence. Attributes with missing values are simply omitted, and the image-to-text pairing was verified by matching image folder names against mineral names returned by the Mindat API.

With the dataset in hand, the researchers fine-tuned Stable Diffusion, a prominent text-to-image model built on the Latent Diffusion framework. Stable Diffusion’s architecture has three moving parts that matter here: a variational autoencoder that compresses images into a compact latent space and reconstructs them, a U-Net that learns to strip Gaussian noise away step by step, and a CLIP text encoder that converts prompts into vector representations injected into the U-Net through cross-attention layers. During training, the model learns to predict the noise corrupting a latent image, conditioned on the text prompt, so that at generation time it can start from pure noise and denoise its way toward an image that matches the description. Fine-tuning concentrated on the U-Net, leaving the pre-trained VAE and text encoder frozen, since those components already carry strong general-purpose capabilities.

The team compared two fine-tuning strategies. Full fine-tuning updates every parameter of the U-Net, a comprehensive adjustment that in this study involved 859.52 million trainable parameters. Low-Rank Adaptation, or LoRA, takes a leaner path: instead of rewriting the weight matrices outright, it freezes them and learns small low-rank residual matrices whose product approximates the needed update, cutting the trainable parameter count to just 0.79 million. LoRA’s efficiency has made it the darling of the open-source AI community, but the mineral experiment delivered a clear verdict. Evaluated with the Fréchet Inception Distance, the Kernel Inception Distance and CLIPScore, the fully fine-tuned model outperformed both the LoRA variant and the untouched baseline, achieving an FID of 48.83 and a KID of 0.02, indicating generated images whose statistical distribution sits closest to that of real mineral photography.

The qualitative differences were just as telling. The baseline Stable Diffusion model, asked for a mineral, tended to produce geometric patterns with excessive specular reflections and a synthetic, manufactured sheen, while the Kandinsky model drifted toward an artistic style with morphologically concentrated structures. The fully fine-tuned model, by contrast, produced images with irregular granular textures, heterogeneous color distributions and distinct crystalline structures that echo real specimens. LoRA outputs improved on the baseline but remained unstable, sometimes over-smoothed and incompletely conditioned on the target attributes. The authors caution that CLIPScore, being trained on general-domain corpora, may understate the gains on specialized mineralogical vocabulary, a known limitation of automatic metrics when domain-specific terminology enters the picture.

One of the study’s most practical findings concerns prompt detail. When the researchers varied how many attributes appeared in a prompt, from a terse three-attribute description to a comprehensive one matching the training template, generation quality climbed steadily with specificity. Sparse prompts leave the model under-constrained, allowing attribute combinations that could correspond to many different minerals, while richer descriptions tighten the visual target. The team validated the approach against reality by querying Mindat’s own advanced search portal for images of chrysoberyl matching a given attribute combination and finding the AI-generated counterparts strikingly similar. They then pushed further, feeding the model attribute combinations belonging to no known IMA-approved species, generating portraits of hypothetical minerals that may exist in nature but remain unclassified, or may only form transiently under extreme temperatures and pressures.

To put the technology in users’ hands, the researchers built an interactive tool, available online, in which anyone can select mineral attributes, have them converted automatically into a prompt, adjust parameters such as random seed, image size, guidance ratio and inference steps, and receive a watermarked generated image. The watermark is a deliberate safeguard against misuse and protects the integrity of the underlying dataset. The authors envision the tool serving mineralogists seeking a visual reference for attribute combinations, enthusiasts exploring the diversity of the mineral world, and science educators making abstract properties like cleavage and streak tangible for students. They are candid about limitations: synthetic images cannot be verified as depictions of real entities, the uniform description template may constrain image-text alignment, and only ten of Mindat’s more than 140 attributes were used, partly due to the text encoder’s input limits.

The road ahead is ambitious. The team proposes supplementing fine-tuning with supervised fine-tuning on higher-quality pairs and with preference-based optimization methods such as direct preference optimization to sharpen alignment between images and descriptions. They also sketch the inverse problem: fine-tuning a generative image-to-text model so that a photograph of an unknown specimen yields a description of its compositional and optical attributes, bypassing traditional classification pipelines in favor of reasoning from visual cues. And they suggest benchmarking against newer generators beyond Stable Diffusion. If mineral formation is a story of temperature, pressure, redox state and fluid chemistry unfolding over billions of years, a model that has learned the visual grammar of that story may one day help predict what minerals on other planets look like, or what fleeting intermediate phases in a crystal’s birth might have been, long before any camera could capture them.

Subject of Research: Fine-tuning the Stable Diffusion text-to-image model with Mindat mineral data to generate mineral images from textual descriptions of attribute combinations.

Article Title: Fine-tune the stable diffusion model using mindat data to generate mineral images from textual descriptions of attribute combinations

Article References: Fu, Q., Que, X., Ma, X., Lin, M., Sun, S., & Chen, M. (2026). Fine-tune the stable diffusion model using mindat data to generate mineral images from textual descriptions of attribute combinations. Earth Science Informatics, 19(11), Article 190. https://doi.org/10.1007/s12145-026-02235-2

Image Credits: AI Generated

DOI: 10.1007/s12145-026-02235-2

Keywords: Mindat, Stable Diffusion, text-to-image generation, mineralogy, mineral informatics, fine-tuning, LoRA, OpenMindat, generative AI, crystallography, image-text dataset, Earth Science Informatics

The tool, called HGCTM-S—short for hierarchical geological context topic model with sparse deviations—was developed by Muhammad Atif Bilal of Jilin University’s College of Geoexploration Science and Technology and Kateryna Hlyniana of Jilin University’s School of Mathematics and the Institute of Mathematics of the National Academy of Sciences of Ukraine. Rather than trying to force every deposit into a rigid classification box, the model treats each copper deposit as a mixture of recurring “assemblage modes,” statistical themes that capture groups of minerals that tend to appear together. The approach borrows its core logic from topic modeling, a technique originally devised to discover latent themes in large collections of text, and adapts it to the language of rocks: instead of words in documents, the model reads mineral families in deposit records.

The scale of the analysis is considerable. The researchers drew on the global copper deposit dataset, an open-source compilation covering 1,335 deposits, and organized 1,205 distinct mineral species into 35 geologically defined families. Grouping species into families was a deliberate choice to counteract sparse and inconsistent documentation, since individual rare minerals appear in too few records to support robust statistics on their own. The hierarchical structure of the model allows geological context—information about where and in what kind of rocks a deposit sits—to inform how the assemblage modes are expressed, while a sparse-deviation component captures localized anomalies that depart from the broader patterns.

When the model was fitted to the full dataset, it recovered seven assemblage modes, a number derived from the data itself rather than imposed in advance. The most geologically meaningful of these were validated against independent deposit type labels. A copper–molybdenum mixed mode emerged as strongly enriched in porphyry deposits, the giant intrusion-related systems that supply much of the world’s copper, while a nickel–cobalt–arsenic mode aligned closely with magmatic sulfide deposits, which form when sulfide liquids segregate from cooling magmas. These correspondences matter because the model was never told which minerals should characterize which deposit types; the associations emerged from the raw mineralogical records alone, and the deposit type labels served only as an independent check.

Just as telling were the modes that did not correspond neatly to genetic classes. Several of the remaining themes represented shared sulfide backgrounds common to many deposit styles, secondary overprints imposed by later weathering and alteration, or residual components that likely reflect the idiosyncrasies of the dataset rather than genuine ore-forming processes. The authors are explicit on this point: HGCTM-S is a tool for comparing overlapping mineral assemblage components and their regional geological associations, not a universal deposit classifier or a regional predictor. That restraint is rare and refreshing in a field where machine learning results are sometimes oversold as oracle-like prediction engines.

One of the study’s central questions concerned the relative influence of regional lithology versus broad geological age on mineral assemblage composition. Across alternative priors and multiple lithology proxies, the model consistently found that lithology-associated effective deviations were larger than age-associated deviations, suggesting that the kinds of host and country rocks surrounding a deposit shape its mineralogy more powerfully than the era in which it formed. The magnitude of this effect varied, however, and the lithology proxies proved unreliable at reproducing finer details such as deposit scale or specific host rock types—a reminder that coarse global datasets can constrain broad patterns but stumble at deposit-level resolution.

The team also probed the stability of their results. Progressive initialization, a strategy in which model fits are seeded sequentially to encourage convergence toward consistent solutions, improved the aggregate stability of the recovered topics. Yet geographic performance remained heterogeneous: in some regions the model’s topic assignments added value beyond what geological context alone could provide, while in others they did not reliably outperform a baseline built purely from contextual information. This heterogeneity is itself informative, pointing to regions where mineralogical records are rich and internally consistent, and others where documentation gaps or sampling biases dominate the signal.

The significance of the work extends beyond copper. Mineral informatics, the emerging discipline that applies data science to mineralogical databases, has matured rapidly over the past decade, with network analyses and association-mining studies revealing deep structure in how minerals co-occur through Earth history. HGCTM-S adds a probabilistic, context-aware ingredient to that toolkit, one that explicitly models uncertainty and mixture rather than demanding clean categories from messy reality. For exploration geologists, the framework offers a way to compare deposits in terms of their full assemblage fingerprints, potentially highlighting overlooked analogs and guiding targeting in data-rich terranes.

The study is also a candid case study in the limits of big-data geoscience. Public mineral databases are treasures, but they are treasures assembled by many hands over many decades, with unequal documentation, variable data quality, and overlapping mineralogical signals baked in. By quantifying where the model succeeds and where it falls short, the authors provide a template for honest evaluation that the broader community can adopt. As the energy transition drives unprecedented demand for copper—and for the cobalt, nickel, and molybdenum that often accompany it—tools that can faithfully extract geological meaning from imperfect global datasets will only grow in value. HGCTM-S does not replace the trained eye of the field geologist, but it gives that eye a new way of seeing the planet’s copper endowment all at once, one statistical theme at a time.

Subject of Research: Statistical topic modeling of mineral assemblages in global copper deposit databases

Article Title: HGCTM-S: Modeling Regional Lithology-Associated Mineral Assemblage Modes in Global Copper Deposits

Article References: HGCTM-S: Modeling Regional Lithology-Associated Mineral Assemblage Modes in Global Copper Deposits. (n.d.). https://doi.org/10.1007/s11053-026-10767-z

Image Credits: AI Generated

DOI: 10.1007/s11053-026-10767-z

Keywords: copper deposits, mineral assemblages, topic modeling, machine learning, mineral informatics, porphyry deposits, magmatic sulfide deposits, lithology, geochemistry, mineral prospectivity, global copper dataset, Natural Resources Research