Original Article

Data provenance, technical heterogeneity, and source confounding in a composite public brain MRI dataset

Download PDFDownload

Abstract

Background: MRI brain collections are often used to benchmark artificial intelligence; however, these derivatives may introduce signals from the source and preprocessing that correlate with diagnosis labels.

Objectives: To assess the data provenance, technical heterogeneity, and potential source-related confounding factors in a composite public brain MRI image dataset. It also seeks to examine whether tumor-class labels are influenced by non-anatomical image characteristics and how they threaten the validity of downstream AI model evaluations.

Methods: A file-level reproducibility audit of a composite Kaggle archive containing 6,892 JPEG MRI images labeled glioma, meningioma, pituitary tumor, or no tumor was performed. Images were assessed for clarity, exact clone, source indicators, color mode, spatial dimensions, predefined split, imaging plane, and pre-augmentation. Associations with labels were quantified using the chi-square test and Cramer's V. A transparent metadata-only rule was fitted to the predefined training partition and tested without reading pixels from the images.

Results: There were no byte-identical duplicates, and all images were readable. The archive contained a total of 5,611 images derived from BRISC, 1,281 legacies, 430 classes, 3,957 RGB files, 2,935 Gy files, and 171 explicitly augmented meningioma images. The resolution (512 × 512), mode of color (Cramer's V=0.582), source (Cramer's V=0.352), and pre-augmentation variable whose Cramers V were significantly associated (all p<0.001). In the untouched BRISC test subset, a rule using only color mode and resolution achieved 37.9% (95% CI 34.8-41.1%) accuracy and 45.2% balanced accuracy. For comparison, the majority baseline achieved 30.7% accuracy and 25.0% balanced-accuracy. Incorporating the source into the consolidated clean cohort enhanced the accuracy to 50.0%.

Conclusion: Technical metadata contained many label details. Therefore, random splitting at the image level or uncritical merging can inflate the apparent performance of the model. Clinical interpretation necessitates source-aware evaluation, patient- or series-level partitioning, metadata-only control, and external testing.

Keywords

Brain tumorMagnetic Resonance ImagingDataset auditSource confoundingShortcut learning

Corresponding Author

Dr. Ahed J Alkhatib

Department of Legal Medicine, Toxicology and Forensic Medicine, Jordan University of Science & Technology, Jordan (retired)

ajalkhatib@just.edu.jo

Article History

Received Date : 11 May 2026

Accepted Date : 08 June 2026

Published Date : 2026

Loading publication timeline...

WhatsApp Chat