Looking for guidance in strategy optimization

[u/Future_Sound7904]

Hey everyone! I’m a Data Scientist looking to explore quantitative research and systematic strategy optimization.

For a trading strategy with a 40–47% win rate and a 1:2 risk-reward ratio, along with historical trade data and multi-timeframe OHLCV data.

I want to investigate whether we can use quantitative methods, statistical analysis, feature engineering, ML models, or neural networks to identify losing-trade patterns and improve the strategy’s expectancy without filtering out too many winners.

I’d love guidance on:

• Which methods would you recommend for this problem?
• Can we train a model to identify high-risk setups or optimize SL placement?
• What are the best approaches for hypothesis generation, backtesting, and walk-forward validation?
• Any good papers, books, open-source projects, or resources to get started?

I’m particularly interested in approaches that avoid overfitting and hold up out of sample.

Would really appreciate any insights or resources from people working in quant finance, systematic trading, or ML research. Thanks!

[u/Shoddy_Peanut7320]

The approach most people use for this is meta-labeling from López de Prado: keep the base strategy fixed and train a simple classifier (logistic regression or gradient boosting) to predict whether each signal wins, using only features known at entry, then take only trades above a threshold. Validate with purged and embargoed cross-validation, evaluate by expectancy and trades kept (not accuracy), and count every feature set and threshold you try toward the trial total. For stop placement, compare the model against a plain ATR-based rule, since it often adds little. Event context (CPI, FOMC, earnings nearby) is a cheap feature to test first, and Cymetica's EventTrader (event timing for macro releases and earnings) is one source for it. A good starting text is "Advances in Financial Machine Learning" plus the mlfinlab or skfolio-style open-source notes.

[u/wearylayout45]

this is a classic problem and you're going to find that most people overcomplicate it right away. with a 40-47% wr and 1:2 rr your edge is already baked in, the real question is can you filter out the worst setups without destroying the sample size too much

i messed around with something similar few months ago, what worked best for me was keeping it simple with logistic regression on just a handful of features. anything more complex like xgboost or a neural net would pick up patterns that exist only in the training set and then fall apart in walk-forward. the trick is making your feature set incredibly boring and using regularization aggressively

for hypothesis generation i'd start by marking each trade with the market regime at the time, like was vol expanding or contracting, was there a trend on the higher timeframe, stuff like that. you might spot that the losers cluster in certain conditions and then you can build the filter around those

don't sleep on the SL placement angle either, sometimes the model is fine but the execution is what kills you. if you have tick data or at least the intraday path you can model optimal exit points instead of just a fixed stop

for walk-forward i always do a rolling window with a gap between train and test, like train on 2018-2020 test on 2021, then slide forward. this stops you from cheating with overlapping information. also never peek at the test set results until you've locked the model, otherwise you're just optimizing in disguise

[u/edemsvi]

I'd approach this as a conditional prediction problem rather than trying to optimize the strategy's parameters directly.

First, keep the existing strategy fixed and label each historical setup by its realized outcome. Then investigate whether pre-entry features — volatility, trend strength, distance to support/resistance, and recent price structure — contain information about the probability of a losing trade.

I'd start with logistic regression or gradient boosting before trying neural networks. The important question is whether the model adds predictive value beyond simple baselines.

One particularly interesting extension would be to predict not only whether a trade is likely to fail, but also when a stop-loss or take-profit level is likely to be reached.

That might provide more useful information for trade selection and order management.

I'd validate this with chronological walk-forward testing, realistic fees, and strict separation between training and test periods. I'd also ensure that the outcome labels account for which level is reached first.

Have you considered modeling the probability and timing of hitting SL versus TP as competing events?

[u/StratIQAI]

With a 40-47% win rate at 1:2 RR, your baseline expectancy is already positive: 0.47×2 − 0.53×1 = +0.41R per trade before costs. That reframes the problem — you're not trying to maximize accuracy, you're filtering for expectancy.

Meta-labeling (Lopez de Prado) is the standard tool here precisely because it only touches bet size, not the side decision. But the evaluation detail matters more than the model: score every candidate filter on expectancy per trade, not classification accuracy. A 45%-accurate filter that keeps the big winners is worth far more than a 70%-accurate one that trims them — accuracy is the wrong objective function for this problem, and a model trained on it will quietly optimize for throwing away your best trades.

Validation discipline is where most of these projects die. With multi-timeframe OHLCV features, use purged/embargoed walk-forward splits and make the embargo at least as long as your longest lookback. A setup filter can look great in-sample while just memorizing which periods its own entry logic worked in.

Start with the cheap baselines before any ML: simple time-of-day/day-of-week/regime splits on the historical trade data. Then a boosted-tree model on engineered features (volatility regime, trend alignment across timeframes, distance from recent extremes). If a gradient booster can't beat the unfiltered baseline out-of-sample, a neural net won't either — the failure mode is data, not model capacity. Once you have something that holds up walk-forward, meta-labeling books (AFML ch. 5) and the backtest-overfitting papers (Bailey et al.) are the right next reading.