Penalized spline regression using a mixed effects representation is one of the most popular nonparametric regression tools to estimate an unknown regression function <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$$f(\cdot )$$</EquationSource> </InlineEquation>. In this context testing for polynomial regression against a general alternative is equivalent to testing for a zero variance component. In this paper, we fill the gap between different published null distributions of the corresponding restricted likelihood ratio test under different assumptions. We show that: (1) the asymptotic scenario is determined by the choice of the penalty and not by the choice of the spline basis or number of knots; (2) non-standard asymptotic results correspond to common penalized spline penalties on derivatives of <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$$f(\cdot )$$</EquationSource> </InlineEquation>, which ensure good power properties; and (3) standard asymptotic results correspond to penalized spline penalties on <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$$f(\cdot )$$</EquationSource> </InlineEquation> itself, which lead to sizeable power losses under smooth alternatives. We provide simple and easy to use guidelines for the restricted likelihood ratio test in this context. Copyright Springer-Verlag Berlin Heidelberg 2013
