Abstract
Ann. Statist. Volume 46, Number 1 (2018), 30-59 We study the problem of aggregation of estimators when the estimators are not
independent of the data used for aggregation and no sample splitting is
allowed. If the estimators are deterministic vectors, it is well known that the
minimax rate of aggregation is of order $\log(M)$, where $M$ is the number of
estimators to aggregate. It is proved that for affine estimators, the minimax
rate of aggregation is unchanged: it is possible to handle the linear
dependence between the affine estimators and the data used for aggregation at
no extra cost. The minimax rate is not impacted either by the variance of the
affine estimators, or any other measure of their statistical complexity. The
minimax rate is attained with a penalized procedure over the convex hull of the
estimators, for a penalty that is inspired from the $Q$-aggregation procedure.
The results follow from the interplay between the penalty, strong convexity and
concentration.