A novel class of interacting Markov chain Monte Carlo (MCMC) algorithms, hereby referred to as the Parallel Hierarchical Sampler (PHS), is developed and its mixing properties are assessed. PHS algorithms are modular MCMC samplers designed to produce reliable estimates for multi-modal and...