Privacy Noise Damages Calibration in Federated Clinical Models

Abstract

Hospitals cannot pool patient records, and federated learning is widely proposed as the remedy. Most evidence comes from simulated federations cut from one dataset, which understates the difficulty because every shard shares one measurement process. We instead used a cardiology cohort of 920 patients collected independently at four hospitals in the United States, Hungary and Switzerland, where coronary disease prevalence ranges from 36.1 to 93.5 percent and missing data from 0.2 to 28.8 percent. Local training, pooled training, federated averaging, federated averaging with a proximal term, and federated averaging under record level differential privacy were compared over 25 repeated stratified runs, under one protocol that scores each hospital on its own held out records and a second that scores the shared model on a hospital it was never trained on. A membership inference attack measured leakage directly. Federated averaging recovered 87 percent of the benefit of pooling (area under the curve 0.798 against 0.800 pooled and 0.785 local), but that benefit was almost entirely confined to Long Beach VA, which gained 0.050 while no other site gained more than 0.001. The proximal term changed nothing. Privacy damaged calibration before ranking: at a budget of 8 the calibration error rose 14 percent while the area under the curve fell 1.2 percent. Privacy cost 1.6 to 2.7 times more on an unseen hospital, and federated training leaked less than local training (0.5048 against 0.5196). Federations should report per site outcomes and calibration, not averages and discrimination alone.