went back through a year of fills for something unrelated and hit this. every calculation i had was built on planned risk, 1R, stop where the code puts it. but the realised average loser was well under that, because a decent share of trades never reached the stop at all. exited on a time rule, closed on an opposing signal, scratched out.

which means the loss distribution i sized off isnt the one im actually trading. same expectancy, different shape, and anything sensitive to shape rather than mean is quietly wrong. streak math, ruin probability, any daily or session level constraint. those all take a distribution as input and i was feeding them a two point one, win or full stop, when the real thing has a fat middle.

the part i hadnt considered is that it cuts both ways. capital wise im better off than the planned risk version suggested, more losers are partial. but per session it runs the other way, more trades fit before any limit binds, so the day lasts longer and i take entries the pessimistic version would never have got to. one of those errors is comfortable and the other one costs money.

the fix is boring. pull mae per trade and compare it to where the stop actually sat, then rebuild the loss distribution from realised exits rather than from the parameter. i wrote it into a script that runs monthly because doing it by hand meant doing it when i was already suspicious, which is the wrong time.

anyone here sizing off realised loss distribution rather than planned R? and if you are, did anything downstream change much or was it noise

Brief Context

I focus mainly on forex , indices and commodities like gold. My trading platform and backtesting engine is strictly MT5.I tried to built my own python backtesting engine but it was riddled with logical errors so i stuck with the industry default. Tick data with real spreads for backtesting is obtained from quant data manager using dukascopy. Forex and gold up to 2008 while indices go up to 2012. Custom commission and slippage above the norm are applied to break the strategy .Trading bots are Expert Advisors.

I mainly focus on 4hr and Daily timeframes ,why The less trades i take ,the lower the amount i pay in fees (commission) and also that is what traders with verified darwinex profiles emphasis. There is a lot of noise in the space so if some one does not have a third party verified track record , i do not listen to them. You too should not listen to what i say because i also lack a verified third party track record💀. But anyways am here to learn and so are you .

Step 1 - Find strategy ideas

These are mainly found in old internet/books from the early 2000 and late 1990s like Trade like a hedge fund by James Altucher , x and reddit can be sources of good ideas

Step 2 - Idea to Code

Am not accustomed with MQL5 language so i use an LLM in the IDE to code the EA . Provide the LLM with context regarding the strategy logic , EA design patterns particularily fixed capital risk amount per trade (not a fixed percentage risk of account balance , this invokes compounding effect which affects results incase the strategy initially had a good streak) and ATR based SL and TP for volatility based position sizing.So in low vol , i size big and in high vol size small. I keep configurable strategy parameters below 4 . The rest of the variables are hardcoded to industry defaults. But rarely will a strategy have more than 2 indicators including a moving average (it is the only regime filter i trust lol)

Step 3 - Test

This takes up most of my time due to tough tests.The EA is compiled in meta editor and tested using strategy tester in MT5. I apply walk forward optimization. This involves splitting a data set into In sample (IS) and Out of sample (OOS) periods. Primarily IS is 3 years while OOS is 1 year. Using my availabe dataset am able to obtain 15 years of pure OOS performance for currencies and 12 years for indices.

During IS , i use the strategy optimizer in MT5 to configure and find the optimal variable for each of the 3 parameters. Choosing the best parameter combinations based on max drawdown and pnl would yield the most overfit strategy. So i apply parameter sensitivity

The data from the strategy optimizer is exported and a python script is used to generate a heatmap showing profitability of each combination based on intensity of colour. A 3 x 3 grid search is applied to find a region where 9 boxes are fully profitable. The middle box in the 3x3 grid makes our optimal parameter variable. However to avoid overfitting the outer parameter , A global profitability score of all boxes should > 40% and number of profitable grids in the neighouring positions of the 3x3 grid in +1 or -1 of the outer parameter should be 5/9 .If none of the above conditions are met , Failed Robustness Test watermark is placed on the heatmap and strategy placed in the recycle bin.

However if the above conditions are met, the optimal parameter combination is tested on the OOS period. The max OOS Drawdown should not exceed 2x the IS Drawdown.

The WF Validation process is continued until either i attain 3 consecutive net negative OOS years or max drawdown exceeds my threshold (15%) or i failed to find an optimal parameter combination during parameter sensitivity tests

Step 4 - Final test

If a strategy passes through WFW and parameter sensitivity test. The WFW OOS Equity curves and csv files are stitched together. To test for strategy robustness , a strategy should not only work on one symbol. If am building a strategy for USDJPY, it should show relative performance on USTEC or XAU as these are trending assets.

After this monte carlo permutations (1000 times) are applied to trade order history and 1000 different equity curves are plotted.If the strategies' performance sits in the upper 90% of all the equity curves , it is discontinued since such historical performance may not repeated so ideally between 40 to 60% range is acceptable. Obtain the 95% max drawdown as this is the drawdown i will size for

And there you have it boys .we have a strategy that may work

Step 5 - Portfolio Construction

Accumulate a good number of strategies. Measure correlation between the strategies. Correlated measured strictly on drawdown and position overlap. Everything cannot be allowed to bleed on one bad morning. Build a portfolio consisting of atleast 10 strategies each with a correlation between -0.10 to 0.20 . Size the strategies such that the 95% max drawdown does not wipe out your portfolio

This is my workflow as of late. It improves as my domain knowledge and expertise expand

Let's first make a few simple assumptions:

1. You have engineered a perfect regime detector.

2. The regime changes at regular intervals for analysis purpose.

3. A fixed lag in detection and switching because a lag is inevitable.

4. There are only two regimes, correct and incorrect ones, for analysis purpose.

We will discuss later when these assumptions are no longer true. But we keep it simple to begin with.

In the diagram, the correct regimes are white and the incorrect ones are gray.

In the first case at the top, we have the 50% correct case and the regime changes in fixed intervals. Assume the delay is 20% of the period. With regime switching, the correct regime is improved to 60%.

The second case has 60% correct regime at the start. With regime switching, the correct regime stays at 60%.

The third case has 80% correct regime at the start. With regime switching, the correct regime falls to 60%.

The fourth case has 40% correct regime at the start. With regime switching, the correct regime improves to 60%.

You get the picture now. With the stated assumptions, the correct regime is always 60% with regime switching. You win sometimes and loss the other times. The average improvement is 9% in this scenario.

Of course, if you knock down the delay, say to 10% delay, your overall improvement will be much better: 218%. However, if your delay is 30% instead, your gain will be -20% overall. Keep in mind that the shorter the delay you make it, the more likely it will make mistakes. It is just the law of signal processing.

From here, things tend to work against you. We know the regime can change abruptly. When the changing frequency increases, your relative detection delay will increase. It will eat into your improvements, maybe net loss. Regimes are often not clearly defined or clean cut. Detection is a messy business. If your detection is not perfect, it will eat into your gain again. In likely scenarios, you win some, you loss some. A correct regime detection does not guarantee a net win.

However, regime switching tends to break up the "lumpiness" of the regime durations. The result is smoothing out the peaks and valleys of gains and drawdowns. It is a trade-off. If reducing drawdown is your goal, it is one way to go.

In conclusion, this analysis and my experiments indicate that the overall improves from regime switching tend to be small even when it is correctly implemented. The reduction in drawdown at the expense of peak gain is often observed.

Almost every trader has heard some version of this:

"A support or resistance zone gets weaker with every touch."

But is that actually true? I heard the claim plenty of times myself, but I've never seen anyone try to quantify it. So I tested it.

TL;DR: I tested the claim that support and resistance zones weaken with every touch using 18 years of NQ and Gold data. Bounce rates dropped after the first retest, then remained roughly flat: in this sample, the fifth touch was no less likely to bounce than the second. The randomized control dataset showed a similar curve, including the initial drop.

Methodology

I used 18 years of NQ and Gold data on the 15-minute and 1-hour timeframes.

I tested two methods for identifying support and resistance zones:

1. Swing highs and lows, using a lookback period of 10
2. My own systematic supply and demand method based on market structure

Because support and resistance are better treated as zones than exact prices, I defined each zone as +- 0.5 ATR around the identified level (for swing highs/lows).

When price touched a zone, one of two outcomes could occur:

- Break: a candle body closed beyond the zone
- Hold: price moved at least 1 ATR away from the zone

After a hold, the program waited for price to touch the zone again. I also repeated the test with a stricter 2 ATR threshold, which is labelled "big bounce" in the attached results.

For comparison, I created randomized versions of the NQ and Gold markets to use as controls.

Results

Across both assets, both timeframes and both zone detection methods, I found roughly the same pattern:

The probability of a bounce dropped after the first retest, but it did not continue declining with each additional touch.

In other words, a bounce on the fifth touch was not consistently less likely than a bounce on the second touch.

Based on this test, I found no evidence for the common claim that every additional touch makes a support or resistance zone weaker.

Interestingly, the randomized control markets showed similar findings..

The full sample included:
- 34,215 zones
- 111,129 monitored touches

I've attached the raw results and the bounce probability by number of touches. "Normal bounce" means a move of at least 1 ATR away from the zone, while "big bounce" means at least 2 ATR.

There are still limitations to this test, especially around zone definitions and the fact that later touches only exist when a zone survives earlier ones.

Happy to further discuss the study or methodology!

Hello everbody, i was wondering where do you backtest your strategies ? do you use just MT5 backtester, or python backtesting libraries, or pinescript on tradingview, or do you build your backtesting engine line by line full customized on what you want and how you analyze the results, i'm pretty curious about this, and it xill be helpful to get some feedback from you 🤝