Exact expressions for double descent and implicit regularization via surrogate random design

Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The authors provide the first exact non-asymptotic expressions for double descent of the minimum norm linear estimator, using a novel notion of surrogate random design that gives exact expressions for the mean squared error.