Half a Billion Health Records, One Big Problem: Can the Data Be Trusted?

A health dataset can contain more than half a billion records and still be unsafe to compare. Hidden coding shifts, missing fields, truncated files and incompatible definitions can quietly turn impressive scale into misleading evidence, particularly when researchers assume that standardized columns mean standardized meaning.

In "Data Access and Quality Barriers in Large-Scale Administrative Health Data: A Reproducible, Information-Loss-Aware Harmonization Framework," published in Applied Sciences, authors Karol Wykrota and Justyna Kęczkowska test a harmonization framework across 561,966,231 hospital discharge records from Korea, Brazil, Mexico, Chile and New York State. The results show that trustworthy reuse depends not just on merging data, but on making every transformation, incompatibility and information loss visible enough to audit.

More Data Does Not Mean More Comparability

Administrative hospital records are attractive because they cover large populations and contain information on diagnoses, length of stay, costs and service use. However, these datasets are usually designed for billing, reporting or benefits administration, not for multinational research, which means they often reflect local rules, institutional practices and coding conventions.

The study identifies barriers across access, structure, integrity, semantics and value quality. Countries may use different ICD versions, different age groupings, different cost fields or different rules for anonymisation. Even technical details such as delimiters, character encoding and file structures can vary from year to year, creating hidden breaks in supposedly continuous datasets.

A shared column name does not necessarily mean a shared concept. The researchers explicitly distinguish fields that are genuinely comparable from those that need transformation and those that should not be pooled without an explicit override. In one example, a secondary-diagnosis field may represent a genuine comorbidity in one source and an external-cause code in another.

This approach is more cautious than many standard harmonisation exercises. Instead of forcing every dataset into superficial uniformity, it preserves visible boundaries around what can and cannot be meaningfully compared. It is critical for health policy, where false equivalence can turn a technical convenience into a misleading conclusion.

The Biggest Risk Is the Error You Never See

The framework was evaluated across 99 annual files spanning 2001 to 2025 and covering 561,966,231 harmonised records. The scale is impressive, but the more revealing findings came from the quality checks rather than the record count itself.

Two Korean files had record counts that strongly suggested source-level truncation. Under the study's assumptions, the likely missing volume across those years ranged from 21.8 million to 25 million records. The researchers do not claim to know the exact failure mechanism, but they show how continuity checks can flag defects that a routine read would simply accept.

A different problem emerged in Mexico, where one specialty-related field was almost entirely absent in a single vintage despite being close to complete in surrounding years. The framework also quarantined 378 malformed rows rather than trying to reconstruct them silently, preserving a clear audit trail of what was excluded and why.

Data cleaning often focuses on making irregularities disappear, but this study treats anomalies as information. A missing field, malformed row or implausible value may reveal a weakness in the source itself, and erasing that evidence can make downstream analysis look cleaner while becoming less trustworthy.

The framework achieved source-profile conformance for 82 of 99 files, full canonical conformance for 39, and complete idempotency across all 99 files. Record-weighted value-domain conformance reached 99.9933%, showing that transparent processing can operate at very large scale without sacrificing reproducibility.

Harmonisation Should Record What Gets Lost, Not Just What Gets Standardised

The study treats information loss as something that should be measured and reported. Instead of assuming that harmonisation improves data by definition, the framework records which fields are missing, which variables have reduced granularity, which records are unavailable, and which classification changes undermine comparability.

This is specifically important for diagnostic data. New York's public discharge files use grouped diagnosis categories rather than raw ICD codes, and the grouping system changes over time. The researchers therefore refuse to treat pre- and post-transition categories as automatically equivalent, because changes in classification can create artificial shifts that look like real changes in disease patterns.

To test whether the harmonised data could still support meaningful analysis, the researchers applied a common ischemic-stroke definition to Korea, Brazil, Mexico and Chile. The same query worked across all four sources retaining raw ICD-10 codes, demonstrating technical portability without separate scripts for each jurisdiction.

A technically consistent cohort does not automatically become a valid cross-country epidemiological comparison. Population denominators, age standardisation, institutional coverage and clinical validation are still required before researchers can compare rates or outcomes across systems.

Health Data Governance Needs Auditability as Much as Interoperability

Governments and health agencies increasingly invest in digital health infrastructure, but the value of those investments depends on whether secondary users can understand how records were created, transformed and constrained.

• For data custodians, the study strengthens the case for publishing stable documentation, version histories, schema definitions and access records alongside datasets. Making records available without explaining changes in coding, file structure or institutional scope may technically satisfy open-data goals while leaving researchers unable to use the information responsibly.
• For international organisations and development agencies, the framework offers a potentially useful middle ground. Comprehensive common data models such as OMOP remain powerful for federated and vocabulary-rich analysis, but they are resource-intensive. A lighter harmonisation layer could help lower-resource systems prepare data for comparative research without requiring immediate full-scale migration.

In the Global South, administrative health data can support stronger evidence on service use, system performance and public health, but technical capacity and financing remain uneven. A reproducible intermediate layer could reduce entry barriers while still forcing researchers to confront differences in coding, coverage and institutional meaning.

The study has some limits. Its five jurisdictions are case studies rather than a representative global sample, there is no stable patient identifier for longitudinal analysis, and independent replication of the crosswalks by a second analyst has not yet been completed. External comparison with provider-published aggregates was also possible for only one source.

Future work should extend the framework to more jurisdictions, incorporate additional outcomes such as mortality, test export into comprehensive common data models, and examine whether independent teams can reproduce the same transformations. The authors also argue that comparative research must add population denominators, standardisation and clinical validation before harmonised records can support robust cross-country inference.