Testing dimensionality in the multivariate analysis of variance

In the multivariate one-way classification with fixed or random effects the between-group effects may be restricted to a lower dimensional space. The problem of testing the dimension of the effect space is treated. For the balanced random effect model the asymptotic null distribution of the likelihood ratio statistic is discussed; the asymptotic distribution is not chi-squared. For the unbalanced fixed or random effect model we suggest the use of a test statistic of the same form. The test statistic is shown to have the same asymptotic null distribution as that for the balanced random effect model. The result is extended to the fixed or random effect models with covariates. The use of the test is illustrated in an example from animal breeding. Some optimality properties of the tests and open problems are discussed as well.