Research Seminar in Statistics A Variational Approach to Decomposing Proper Scoring Rules

Speaker: Eugene Berta, INRIA Paris Abstract: Proper scoring rules are fundamental for evaluating the quality of probabilistic forecasts. A canonical property of these scores is their decomposition into calibration and refinement errors. However, reliably estimating these two components is notoriously difficult in practice. Standard approaches are often tied to the specific setting of binary classification, and suffer from well-documented finite-sample pathologies that can lead to systematically over- or under-estimating calibration error. In this talk, I will present a variational formulation of the calibration-refinement decomposition that holds for any proper scoring rule and any forecasting space. This formulation gives a natural interpretation: refinement error is the best achievable r