CPDC: Confidence-agreement guided semantic-physical progressive data curation for multi-source diabetic retinopathy

Multi-source DR datasets carry label noise, class imbalance, and cross-device shift. Confident learning detects label issues from out-of-fold probabilities but inherits teacher miscalibration, and purely semantic signals cannot flag physically degraded images. We propose CPDC, a data curation framework that integrates a confidence-agreement mechanism, an autoencoder quality gate, and progressive tiering. The agreement mechanism derives cross-paradigm scores from a supervised and a self-supervised teacher. The quality gate screens images for physical degradation before any semantic assessment occurs. Progressive tiering then stratifies samples into three tiers. We evaluate CPDC on 53,149 fundus images from seven datasets with in-distribution and out-of-distribution test splits. Against empirical risk minimization, CPDC yields higher balanced accuracy (ID: 86.24% vs 82.44%, +3.80pp; OOD: 90.88% vs 87.40%, +3.48pp; both CIs exclude zero). Compared to the single-teacher baselines Cleanlab, AUM, and EL2N, CPDC achieves comparable discriminative accuracy (BA difference  ≤ 0.41pp, CI includes zero) while providing lower calibration error (teacher ECE: 4.8% vs 12.4%–12.6%; OOD NLL: 0.169 vs 0.176, 95% CI) and per-source retention varying from 32.5% to 66.7%. A teacher-matched control narrows this gap to within seed-level variation, so the agreement signal rather than the tiering rule drives it. Correction-enabled variants provide a controllable ID–OOD trade-off via τ_A: conservative relabeling (τ_A ≥ 0.85) maximizes ID performance (BA 86.78%), while aggressive relabeling (τ_A ≥ 0.70) improves OOD generalization (BA 91.25%). Code and reproducibility metadata are publicly available. Original images must be obtained from the source datasets, after which our curation framework reconstructs the exact curated sets.