We introduce two online backtest overfitting tools: BODT simulates the overfitting of seasonal strategies (typical of technical analysis), and TMST simulates the overfitting of econometric strategies (typical of academic journals). We show that econometric methods lend themselves to extreme...

Empirical Finance is in crisis: Our most important "discovery" tool is historical simulation, and yet, most backtests published in leading Financial journals are flawed.The problem is well-known to professional organizations of Statisticians and Mathematicians, who have publicly criticized the...

We carry out several test cases to illustrate how the Probability of Backtest Overfitting (PBO) performs under different scenarios. We also assess the accuracy of PBO using two alternative approaches (Monte Carlo Methods and Extreme Value Theory).The paper "The Probability of Backtest...

In the field of mathematical finance, a “backtest” is the usage of historical market data to assess the performance of a proposed trading strategy. It is a relatively simple matter for a present-day computer system to explore thousands, millions or even billions of variations of a proposed...

Most papers in the financial literature estimate the p-value associated with an investment strategy, without reporting the power of the test used to make that discovery. In this paper we provide analytic estimates to Type I and Type II errors for the Sharpe ratios of investments, and derive...