Global permutation tests for multivariate ordinal data: alternatives, test statistics and the null dilemma

type="main" xml:id="rssc12070-abs-0001"> <title type="main">Summary</title> <p>We discuss two-sample global permutation tests for sets of multivariate ordinal data in possibly high dimensional set-ups, motivated by the analysis of data collected by means of the World Health Organization's ‘International classification of functioning, disability and health’. The tests do not require any modelling of the multivariate dependence structure. Specifically, we consider testing for marginal inhomogeneity and direction-independent marginal order. As opposed to max-T-tests, which are known to have good power against alternatives with few strong individual effects, the tests proposed have good power against alternatives with many weak individual effects. Permutation tests are valid only if the two multivariate distributions are identical under the null hypothesis. By means of simulations, we examine the practical effect of violations of this exchangeability condition. Our simulations suggest that theoretically invalid permutation tests can still be ‘practically valid’. In particular, they suggest that the degree of the permutation procedure's failure may be considered as a function of the difference in group-specific covariance matrices, the proportion between group sizes, the number of variables in the set, the test statistic used and the number of levels per variable.