The vast majority of investors wanting portfolio backtesting have good intentions. But even the well-intentioned investor may make some mistakes that can lead to dangerous consequences. These seven deadly sins of portfolio backtesting may cause misleading results and failure to achieve the expected results:
- Hindsight Bias: Did you really know that AAPL and GOOGL were going to be trillion-dollar companies? Investors fool themselves into believing that such inclusions are OK because they are obvious. But they are not. Having blindfolded or systematic selection processes protect against this bias.
- Survivorship bias: if you backtest stocks that exist no, the mere fact that they have not gone belly up yet creates a bias. The impact of this bias can vary greatly but for diversified strategy can be 1% to 3% overstated.
- A lack of diversity in the investments
- No optimization: failing to apply an intelligent weighting to the portfolio will likely result in several percentage points of underperformance.
- Cherry-picking the best of many backtests – who is to say that the results of the best backtest are more likely that the results of the worst ones? The more flawed and bias the backtest is the more this sin is dangerous. However, if applied to a credible portfolio backtesting process this could be ok.
- Buy and Hold backtests can be instructive of historical performance but have decreased usefulness to predict the future. Instead of backtesting a reproducible strategy it is merely seeing how a collection of assets would have done.
- A lack of diversification in the investment process. The more narrow the focus of the strategy the more time and backtested event needs to be to make the results credible. Diversification of the investment process is almost as important as diversification of the portfolio!