TPHT yields estimated 5.2-fold retained-output reduction in cloud simulation

A tested two-pass particle-tracking framework reconstructed the histories of selected cloud super-droplets in a three-dimensional marine stratocumulus simulation while producing an estimated 5.2-fold reduction in retained output compared with full-population backward tracking. The reduction applied after temporary discovery files were deleted and was specific to the broad radius and coalescence criteria used in the demonstration. Developed for SCALE-SDM version 5.2.6-2.3.1, the framework supports forward tracking, backward tracking and Two-Pass Hybrid Tracking, or TPHT. Its central capability is to preserve particle-level lineage and event records without requiring every possible super-droplet history to remain in the final archive.

The Super-Droplet Method represents groups of real cloud particles as computational super-droplets carrying properties such as radius, position and multiplicity. This particle-based approach can follow microphysical interactions without dividing droplets into fixed size bins, but retrieving the history of an individual super-droplet from a massively parallel simulation requires stable identifiers, links between saved records and efficient handling of files distributed among Message Passing Interface ranks. The new implementation was designed specifically for the SCALE regional model coupled to SDM and its large-eddy-simulation configuration, where the simulation domain is decomposed among parallel processes and particles can cross subdomain boundaries during integration.

Forward tracking assigns persistent identifiers to particles selected before or during a simulation and writes their later states. Backward tracking instead records, for each selected particle, the MPI rank and local record index where its predecessor can be found at the previous output time. Post-processing can then retrieve that predecessor through a direct constant-time lookup rather than scanning all super-droplet records at every output level. The lookup advantage applies to an individual predecessor query; the total reconstruction work still depends on the number of targets, saved times, requested variables and files read. Both workflows can use all valid super-droplets, a random sample, a height-and-radius-stratified sample or an externally supplied target list.

TPHT combines the two approaches for targets defined by predefined interest conditions evaluated during a discovery integration. The first pass writes identifiers when a super-droplet reaches a selected radius or participates in a recorded coalescence event. Because a particle can satisfy those conditions repeatedly, offline processing merges rank-local identifier streams, checks metadata for consistency with the parallel decomposition, removes duplicate identifier pairs and repartitions the result into compact target-list files. A second integration begins from the same physical and random-number state and writes predecessor links only for that deduplicated set. These lists contain identifiers rather than physical restart states, and the reconstruction pass aborts if required files are missing or inconsistent rather than silently switching to another selection method.

The researchers demonstrated TPHT with an idealized three-dimensional configuration based on the second research flight of the DYCOMS-II campaign, representing nocturnal marine stratocumulus off Southern California on 11 July 2001. The domain measured 2 km by 2 km by 1.5 km and used a 40 by 40 by 300 grid with spacing of 50 m by 50 m by 5 m. The model ran for 70 minutes, with the first 60 minutes treated as spin-up, and used 40 super-droplets per grid cell. During discovery, a super-droplet entered the interest stream upon reaching a radius of 15 micrometres or when its binary coalescence flag indicated participation in at least one recorded event during the current output interval. The reconstruction pass saved selected-particle states every 10 seconds and recorded coalescence-event information separately.

The broad discovery conditions produced 27,826,660,866 identifier records, demonstrating how temporary data can accumulate when the same particle repeatedly meets a condition. Offline deduplication reduced the stream to 3,554,256 unique target identities, estimated to be 19.3% of the valid super-droplet identities. All deduplicated targets were recovered in the backward output, with no additional targets and no mismatch in MPI-rank ownership. The raw interest-identifier files occupied 245 GiB, while the compact per-rank target lists required only 32.0 MiB. The backward selected-particle output occupied 69 GiB, compared with an estimated 356 GiB for full-population backward output under the same output assumptions.

Storage outcomes depended on whether the intermediate discovery files remained available. Retaining the raw streams produced 314 GiB of total TPHT working storage, only an estimated 1.1-fold reduction relative to the 356 GiB full-population estimate. Once the raw files were deleted after the target lists had been generated, the retained dataset consisted mainly of the 69 GiB backward output and the 32.0 MiB target lists, yielding the estimated 5.2-fold retained-output reduction. This factor was not a general performance constant: it depended on the selected criteria, output interval and variables written. Sparser target conditions could produce larger reductions, while broader criteria could reduce the advantage. The offline merge and deduplication stage was not separately timed, so the reported figures quantify storage and diagnostic workflow costs rather than a complete end-to-end optimum.

The two integrations also reproduced the same recorded coalescence history under the tested configuration. The audit compared 26,284 corresponding event-file pairs and 2,919,183 event rows from each run, examining event time, both pre-update radii, both multiplicities and the represented collision count. Every compared field agreed bitwise, with a maximum difference of zero. The two runs used the same executable, and all 256 corresponding boundary, initial-state, random-number-state and reference-state input-file pairs were byte-identical. This was a code-level verification of TPHT rather than proof that every later Eulerian or super-droplet state was identical, because the discovery pass did not retain those state histories. Matching the stochastic trajectory also required the same numerical settings, libraries, MPI decomposition, effective thread count, floating-point operation ordering and physical random-number call order.

Controlled computational tests indicated that diagnostic output writing was the main additional burden in sampled tracking configurations. In a 30-minute three-dimensional benchmark, forward and backward configurations using 5% stratified selection ran for 8,750–8,754 seconds, compared with 8,982 seconds for the no-tracking baseline. Because only one run was available for each configuration, the small differences could not be distinguished from scheduler and file-system variability and were not interpreted as speed-ups or resolved overheads. Peak resident memory remained around 649–655 MiB per rank. The sampled tracking cases produced about 121 MiB of diagnostic output, while coalescence logging alone produced about 4.37 MiB. Increasing the output interval reduced selected-output volume but increased the mean time per individual write because each operation contained more records.

Reconstructed histories demonstrated how selected particle-level information could be connected to simulated cloud fields without claiming a new mechanism of warm-rain formation. Among 3,554,256 targets, 2,279,068 were classified as reaching the 15-micrometre radius threshold and 1,676,150 carried the coalescence flag at least once. First recovered threshold crossings were concentrated mainly between 600 and 800 m, near the upper part of the simulated cloud layer, which extended approximately from 400 to 800 m. The median maximum reconstructed radius was only slightly above 15 micrometres, indicating that many targets were near threshold rather than substantially larger drops. Example histories included a target that exceeded 40 micrometres, evaporated near cloud base and was subsequently transported upward at a smaller radius, while another maintained nearly constant radius despite multiple linked coalescence records.

Several limitations constrain how the framework should be interpreted. TPHT can reconstruct only targets captured by its predefined interest criteria, rather than providing unrestricted access to every super-droplet history after both integrations finish. The binary coalescence flag indicates whether at least one recorded event occurred within an output interval; it is not an event counter, and linked event records are diagnostic links rather than complete causal process budgets. A short quasi-two-dimensional sampling verification showed that droplet-size-distribution errors generally decreased as sampling fractions increased and that stratified sampling reduced seed-to-seed variability for the tested diagnostics, but it did not establish representativeness for long-duration, three-dimensional simulations. Future extensions could include entrainment, mixing, sedimentation, supersaturation and ice-phase criteria, together with timed deduplication, compressed identifier streams and automatic cleanup of temporary files.

Subject of Research: Lagrangian particle tracking and output reduction in cloud microphysics simulations

Article Title: A Lagrangian particle tracking framework for the super-droplet method in SCALE-SDM 5.2.6-2.3.1: implementation and computational evaluation

Article References: Yin, C., Shima, S.-I., & Lu, C. (2026). A Lagrangian particle tracking framework for the super-droplet method in SCALE-SDM 5.2.6-2.3.1: implementation and computational evaluation. Geoscientific Model Development, 19(20), 9993-10022. https://doi.org/10.5194/gmd-19-9993-2026

Image Credits: AI Generated

DOI: 10.5194/gmd-19-9993-2026

Keywords: SCALE-SDM, super-droplets, Lagrangian tracking, cloud microphysics, TPHT, backward tracking, coalescence events, stratocumulus, parallel computing, diagnostic output, Lagrangian, particle