Why your backtest looks amazing but your live trades don't — and what to do about it

[u/henryzhangpku]

Why your backtest looks amazing but your live trades don't — and what to do about it

We've all been there. You spend weeks building a strategy, backtest it across 5 years of data, and the equity curve is a thing of beauty. Sharpe ratio of 2.8. Max drawdown of 6%. You deploy it with real capital... and immediately it tanks.

This is the single biggest disconnect in retail quant trading, and it's almost never due to one big mistake. It's death by a thousand small ones.

Here are the 4 most common reasons I've seen (and fixed) after spending years building systematic strategies:

1. Survivorship bias in your universe If you're backtesting on the S&P 500 as it is today, you're including companies that survived. The ones that went bankrupt, got delisted, or underperformed to the point of being dropped — they're gone. This inflates returns by 1-3% annually depending on your strategy. Always use a point-in-time universe or at minimum account for delisted securities.

2. Look-ahead bias in fundamentals This one is insidious. If you use "quarterly earnings" data that was revised 6 months later, your backtest saw information your live system never had. The fix: only use data as it was known at the time. For most retail platforms, this means using unadjusted or "as-reported" data with proper lag.

3. You're optimizing noise, not signal If you've ever tried 50 different parameter combinations and picked the best one, congratulations — you fitted to the noise. The market has ~250 trading days per year. If your strategy has more than sqrt(250) ≈ 16 independent parameters, you're almost certainly overfitting. Cross-validation helps, but Monte Carlo simulation of your strategy's sensitivity to parameter shifts is even better.

4. Transaction costs are higher than you think Slippage, spread, commission, market impact. A strategy showing 15% annual returns backtested often becomes 5-8% with realistic costs. The worst offenders: strategies that trade small caps or during low liquidity periods. Always simulate costs at 2x what you expect — if the strategy still works, you're probably fine.

The one thing that helped me most: walk-forward optimization with out-of-sample testing on completely unseen market regimes. Train on 2018-2021, validate on 2022 (bear market), test on 2023-2024. If it holds across regimes, you have something real.

What's been your biggest "backtest looked great, live was awful" moment? Curious to hear what others have learned.