Robustness and power of the t, permutation t and Wilcoxon tests

Data analysis is accomplished via parametric or nonparametric methods, depending on the data at hand. Authors have stated that parametric techniques are more robust with regard to Type I error and more powerful than nonparametric techniques. This statement is correct assuming that the numerous parametric requirements are verified, such as normality, homogeneity of variance and random selection and assignment among others. Nonparametric methods, however, are good alternatives to parametric methods as they are robust and powerful under non-normality. Although they have fewer assumptions compared to their parametric counterparts, they still have some requirements that need to be met, such as random selection and assignment. Permutation tests offer advantages compared to parametric tests as well, as they require fewer assumptions. They are distribution-free and exact statistics. It was found that they are robust with regard to Type I error and powerful. The problem resides in the fact that permutation tests maintain the Type I error to the nominal α, however, there is no evidence that they are more powerful than nonparametric tests. Monte Carlo simulations were used to investigate the Type I error and power of the Nest, permutation t-test and the Wilcoxon test for the normal, exponential (μ = σ = 1), Chi-square (df = 6) distributions and the Multimodal Lumpy, a real data set obtained from educational and psychological studies. It was found that, under normality, the t and permutation t-tests were robust with regard to Type I error compared to the Wilcoxon test. They were also slightly more powerful than the Wilcoxon test. However, under non-normality (especially as the departure from normality increases), the Wilcoxon test was, of course, robust with regard to Type I error and much more powerful than the t and permutation t-tests.