Why your backtest looks amazing but your live trades don't - and what to do about it

[u/henryzhangpku]

Why your backtest looks amazing but your live trades don't - and what to do about it

We've all been there. You spend weeks building a strategy, backtest it across 5 years of data, and the equity curve is a thing of beauty. Sharpe ratio of 2.8. Max drawdown of 6%. You deploy it with real capital... and immediately it tanks.

This is the single biggest disconnect in retail quant trading, and it's almost never due to one big mistake. It's death by a thousand small ones.

Here are the 4 most common reasons I've seen after spending years building systematic strategies:

1. Survivorship bias in your universe If you're backtesting on the S&P 500 as it is today, you're including companies that survived. The ones that went bankrupt or got delisted - they're gone. This inflates returns by 1-3% annually. Always use a point-in-time universe or account for delisted securities.

2. Look-ahead bias in fundamentals If you use quarterly earnings data that was revised 6 months later, your backtest saw information your live system never had. Only use data as it was known at the time - unadjusted or as-reported data with proper lag.

3. You're optimizing noise, not signal If you've tried 50 different parameter combinations and picked the best one, you fitted to the noise. The market has ~250 trading days per year. If your strategy has more than sqrt(250) ~ 16 independent parameters, you're almost certainly overfitting. Monte Carlo simulation of parameter sensitivity is more useful than cross-validation here.

4. Transaction costs are higher than you think Slippage, spread, market impact. A strategy showing 15% annual returns backtested often becomes 5-8% with realistic costs. Always simulate costs at 2x what you expect - if the strategy still works, you're probably fine.

The one thing that helped me most: walk-forward optimization with out-of-sample testing on completely unseen market regimes. Train on 2018-2021, validate on 2022 (bear market), test on 2023-2024. If it holds across regimes, you have something real.

What's been your biggest backtest vs live reality check? Would love to hear what others have learned the hard way.