The Calibration Channel Determines the Bayes-Error Proxy: An Exact Law for Temperature-Induced Distortion
The soft label Bayes error estimator beta(z) = E[min(z, 1 z)] of Ishida et al. estimates the irreducible error of a binary task directly from probability valued labels. Recent work by Ushio et al. showed that this estimator is fragile when the probabilities are not the true posterior: even perfectly calibrated soft labels can yield a substantially inaccurate estimate, and they propose isotonic calibration as a con...