Federated and In-House Dashboards Complementary for Data Quality

Researchers from Maastricht University Medical Centre and Centre Léon Bérard have demonstrated that combining in-house and federated data quality dashboards offers the most effective approach for auditing electronic health records in privacy-preserving research environments. Published in Frontiers in Applied Mathematics and Statistics, the study addresses a critical gap in federated learning (FL) systems, where the inability to access individual-level data often limits the ability to detect data quality issues. By implementing two complementary dashboards, the team showed that while in-house tools provide granular, patient-level validation, federated dashboards are essential for identifying cross-institutional patterns and systematic biases that remain invisible to single-centre tools.

The research was driven by the increasing reliance on privacy-enhancing technologies in rare and heterogeneous disease research. Federated learning allows institutions to collaborate on fragmented datasets without sharing raw patient data, thereby adhering to strict data protection regulations. However, this privacy-by-design approach introduces a significant trade-off: without direct access to individual records, identifying outliers, filtering implausible values, and ensuring data integrity becomes significantly more challenging. The authors noted that traditional quality control frameworks, such as those defined by Kahn et al., are often designed for intra-clinic evaluation and require adaptation to function effectively in inter-clinic, federated contexts.

To evaluate these challenges, the team adapted a recognised data quality control framework using breast cancer records from Centre Léon Bérard, a comprehensive cancer centre in Lyon, France. The study focused on a cohort of 839 pathologically confirmed breast cancer patients aged 18 to 39 at diagnosis, who received surgical treatment between 1996 and 2024. The data was processed through an automated extract-transform-load pipeline into the OSIRIS Real World Data model. To simulate a real-world federated setting, the researchers created two distinct copies of the dataset: one for an in-house Microsoft SQL server and another split into two variants to simulate data from two independent centres, with controlled schema heterogeneities introduced.

The in-house dashboard was developed using Microsoft Power BI, leveraging its DAX language capabilities to perform aggregations and relational joins on the local database. This implementation enabled direct querying of the pre-processed dataset while maintaining full access to individual-level data. In contrast, the federated dashboard was constructed using Plotly Dash and integrated with the Vantage6 federated learning framework. Unlike typical federated analyses that aggregate results, this implementation retrieved station-specific information by dispatching descriptive statistics tasks to each data station, ensuring that no individual-level data left the local premises.

The data quality assessment framework was inspired by Kahn et al.’s guidelines, with dimensions of completeness, atemporal consistency, temporal consistency, and semantic consistency. For intra-clinic assessment, these dimensions validated logical coherence within the dataset, while for inter-clinic assessment, semantic consistency was introduced to evaluate semantic interoperability across independent data sources. Validation rules were grounded in the European Commission’s Joint Research Centre guidelines, which define clinically implausible combinations of demographic and tumour characteristics, such as age-morphology-topography and sex-topography inconsistencies.

Results indicated that the in-house dashboard provided superior granularity and reliability, allowing researchers to pinpoint individual-level inconsistencies and trace errors to specific patient identifiers. For example, the dashboard identified four implausible BMI values and four female patients diagnosed with epididymal cancer, a male-exclusive site, demonstrating its capacity for detailed local validation. The federated dashboard, while limited by privacy constraints that prevented individual-level traceability, enabled the detection of cross-institutional patterns. It revealed ecosystem-wide trends, such as non-independent and identically distributed value distributions between centres, which are critical for identifying systematic biases in federated settings.

Specifically, the federated dashboard highlighted semantic inconsistencies in neoplasm topography coding systems, where one organisation used ICD-O-3 codes and the other used ICD-10 codes. It also detected inter-clinic heterogeneity in the availability of laterality recordings, illustrating how undetected variations can introduce calculation errors in multi-centre research. Completeness assessments were comparable in both dashboards, with the in-house tool providing absolute counts of missing data and the federated tool displaying proportions, allowing for the assessment of whether missingness was variable-specific or institution-specific.

The study also evaluated the usability of these dashboards through two interest holder-driven use cases from the STRONG AYA consortium: assessing the feasibility of survival rate analyses and the proportion of patients receiving high-cost treatments. Both dashboards enabled the verification of data availability and completeness necessary for these analyses. The in-house dashboard offered unique advantages in linking high-cost treatments to patient-level variables and comorbidities, while the federated dashboard provided an overview of treatment distribution patterns across institutions, highlighting inter-site variations in the use of intensive therapies.

Limitations of the study include the context-specific nature of privacy requirements in federated learning, which can vary across collaborations. The authors noted that temporal data in the STRONG AYA infrastructure is converted to intervals, which blurs the distinction between atemporal and temporal plausibility, limiting the system’s ability to perform sophisticated temporal analysis. Additionally, while the federated dashboard’s schema-agnostic design allows for scalable deployment, it may require additional quality checks to ensure thoroughness, whereas in-house tools ensure thoroughness at the cost of scalability.

The findings confirm a complementary relationship between the two approaches. Federated dashboards provide scalable, collaborative oversight and reveal collaboration-wide patterns that remain invisible to intra-clinic tools, but they may produce imperfect results for specific quality dimensions. In-house approaches offer thorough validation and the ability to rectify erroneous data points incisively, but they lack the scalability and generalisability needed for multi-institutional research. The authors conclude that the optimal solution for federated learning ecosystems emerges from a combined model that accommodates both institution-specific needs and collaborative research requirements.

This integrated approach is particularly promising for dynamic ecosystems like STRONG AYA, where data structures are diverse and continuously evolving. By leveraging schema-agnostic approaches and common data models such as OSIRIS-RWD and OMOP, the combined framework establishes a robust foundation for data quality in evolving multi-institutional ecosystems. The study underscores the importance of balancing rigid standardisation with contextual adaptability to enable consistent, privacy-aware insights across fragmented healthcare data landscapes.

The study was conducted within the STRONG AYA initiative, a living federated learning infrastructure specifically designed for Adolescent and Young Adult cancer research. This consortium integrates historical registry data, electronic health records from participating institutions, and prospectively collected longitudinal research data. All data collated for STRONG AYA adheres to a consensual set of elements defined by the AYA Core Outcome Set, which specifies the variables to be included without prescribing a rigid data model. Given the diversity of data sources, the infrastructure ensures its data is Findable, Accessible, Interoperable, and Reusable, adopting semantic interoperability rather than enforcing a single model. Aligned with the Personal Health Train manifesto, the infrastructure prohibits individual-level data from leaving local premises, inherently preventing direct inspection and necessitating exploratory, aggregate-level quality analyses.

To facilitate semantic interoperability, the researchers utilized the Flyover tool to convert structured data into RDF triples, annotating them with standardised terminologies. This process involved storing the resulting graphs in a graph database for SPARQL querying, with a central JSON-LD file housing the mapping and defining the target terminologies. To complete the simulation, a local Vantage6 developer network was deployed, featuring a central server connecting to two data stations, each coupled with one of the graph database instances. The federated dashboard was integrated with Vantage6 through its Python client, dispatching distinct descriptive statistics tasks to each station. These tasks retrieved interquartile ranges, means, medians, and frequency counts, natively allowing for data stratification and inlier specification without disclosing individual-level information.

The validation rules employed in the study were grounded in the European Commission’s Joint Research Centre guidelines, which define clinically implausible combinations of demographic and tumour characteristics. These rules, distributed as CSV files, were directly integrated into the quality assessment tools and are designed for regular updates to reflect periodic revisions to published recommendations. The framework distinguished between atemporal consistency, which verifies whether data values are consistent with local or common knowledge in a non-temporal context, and temporal consistency, which verifies whether temporal distributions correspond to logical expectations. For inter-clinic assessment, the authors introduced semantic consistency to evaluate whether data were made semantically interoperable across independent sources, a critical dimension for evolving, heterogeneous federated ecosystems.

While the dashboards demonstrated complementary strengths, the authors acknowledged that standardisation presents ongoing challenges in federated learning ecosystems. The lack of semantic and syntactic standardisation across parties makes uniform deployment impractical without substantial resource investment. Adhering to data models such as OSIRIS-RWD or OMOP-CDM reduces these challenges, enabling in-depth quality evaluation solutions. However, enforcing such schemas places a substantial burden on non-adherent contributors and introduces rigidity, as mapping heterogeneous local systems to a shared model can introduce approximation and information loss. In contrast, the federated dashboard avoids these risks by revealing inter-clinic heterogeneities in terminologies and standards without imposing a common data model, though this flexibility requires re-ingestion after rectifications.

Future directions for this work include developing a complementary dashboard to address intra-clinic aspects that the federated approach cannot detect, establishing a uniform, scalable solution with different deployment characteristics. Current dashboards focus on structural dimensions such as completeness and consistency while neglecting metadata-level aspects like documentation, provenance, and representativeness. These dimensions are becoming increasingly important under the European Health Data Space, which requires datasets to be catalogued using standardised metadata. Incorporating such metrics would provide significant safeguards for downstream reuse while providing little extra concern in terms of privacy. Additionally, integrating machine or deep learning approaches could enable adaptive detection of population-specific anomalies, handling both structural diversity and subtle clinical patterns that traditional frameworks may struggle to identify.

Subject of Research: Complementary data quality assessment dashboards in federated learning consortia

Article Title: Data auditing and quality assurance in a federated learning consortium: getting the best of both worlds from cross-institutional and in-house data quality inspection

Article References: Hogenboom, J., Perez, N., Filori, Q., Sans, A., Lobo Gomes, A., Dekker, A., van der Graaf, W., Husson, O., Crochet, H., Wee, L., & Gouthamchand, V. (2026). Data auditing and quality assurance in a federated learning consortium: getting the best of both worlds from cross-institutional and in-house data quality inspection. Frontiers in Applied Mathematics and Statistics, 12, Article 1981705. https://doi.org/10.3389/fams.2026.1981705

Image Credits: AI Generated

DOI: 10.3389/fams.2026.1981705

Keywords: federated learning, data quality, electronic health records, privacy-enhancing technologies, cross-institutional collaboration, breast cancer, data auditing, semantic interoperability, Data, auditing, quality, assurance