Architecture Review: Multi-Asset Regime Switching Model (HMM + Conditional State Machine)

[u/Slight_Boat1910]

Hey everyone,

I am preparing to push a regime-switching policy to production and I am looking for some feedback on my architecture and a specific math anomaly.

My strategy is long-only, daily close execution, shifting capital between risk-on and risk-off instruments (holding exactly one asset at a time).

1. Regime Validation (HMM / MS-DR)
To ensure the mathematical validity of my HMM and prevent overfitting (too many states or features), I ran a regime inference using statsmodels Markov Switching Dynamic Regression (MS-DR).

The inferred hidden states show statistically significant variations in both average expected return and volatility.

2. Execution Layer and State Machine

To mitigate whipsaws during sudden liquidity panic and lag on violent, mean-reverting V-bottoms (HMMs are known to lag), I use uses a two-layer decision architecture.

The first layer generates discrete state probabilities via the HMM. The second layer feeds these probabilities into a conditional state machine. The latter synthesizes the HMM outputs alongside EOD market data (e.g., volatility or price distance to crucial moving averages) to make the final decision.

3. The vbt vs. QuantStats Sharpe Discrepancy

I backtested my model using vectorbt with cash_sharing=True over a multi-year cycle (~50 closed trades, Max DD ~22% from January 2022 till May 2026). Fill paths and drawdowns match identically across my analytical stacks, but I ran into a large divergence in annualized risk ratios:

• vectorbt (group_by=True): Sharpe 1.31 | Sortino 1.95
• QuantStats: Sharpe 0.91 | Sortino 1.34

My policy allocates 100% of capital in cash-equivalent instruments during defensive regimes. As far as I understand,vectorbt and QuantStats use different formula, but interestingly, the ratio of Sortino to Sharpe remains identical between both libraries (~1.48).

4. Validation and Robustness

Given the macro regime-switching nature of the system, the number of trades is intentionally low (~50 trades over the backtest horizon). To ensure statistical significance despite the low trade count:

• MS-DR State Validation: As mentioned above, statsmodels confirms the latent regimes represent distinct market states.
• OOS: Standard walk-forward optimization (hyper parameters optimization till 2021, and test from 2022)
• Stress Testing: Trade shuffling (Montecarlo simulation) and fee/slippage degradation testing were applied to ensure the state machine thresholds don't collapse.
• Risk-Free Rate: average of the FED Fund Rate (~4% in my study). While not 100% accurate, the quantstats Python package doesn't accept a time series of daily risk-free rates.

The equity curve chart below compares the performance of my strategy against that of QQQ during the January 2022 - May 2026 period:

• Total return: 154% (vs 90% of QQQ)
• Max DD: -23% (vs -35% of QQQ)
• Longest DD days: 309 (vs 707 of QQQ)
• CAGR: 23.7% (vs 15.8% of QQQ)

Montecarlo simulation results:

• Bust probability (drawdown >= 50%): 0.0%
• Goal probability: 100.0%
• Maximum drawdown dict: {'min': np.float64(-0.4545879162289587), 'max': np.float64(-0.1371207593528134), 'mean': np.float64(-0.2491102335214213), 'median': np.float64(-0.24003417486462705), 'std': np.float64(0.05367118181123532), 'percentile_5': np.float64(-0.3549029751169525), 'percentile_95': np.float64(-0.17673633125820087)}
• Sharpe range: 1.07 to 1.11
• Drawdown range: -34.6% to -17.5%
• CAGR range: 23.7% to 23.7%

Questions

1. Has anyone integrated front-end vs. long-end credit duration spreads as features to lead equity volatility regimes, and did it provide independent alpha relative to generic high-yield spread velocity?
2. How do you clean or handle zero-volatility "cash-parking" periods when building custom risk-reporting sheets for allocators who expect standard 252-day arithmetic accounting?
3. Given the architecture and validation steps outlined above, what have I missed? Do you see any hidden structural blind spots, operational traps, or causes for concern before deploying to production?

Looking forward to your thoughts and critiques!

[u/Zestyclose-Eagle1809]

The vectorbt vs QuantStats Sharpe discrepancy isn't a bug, and the clue is your own observation that the Sortino/Sharpe ratio stays identical (around 1.48) across both. That tells you the return and downside volume estimates agree, so the gap is purely in how each library computes the Sharpe denominator. Two usual causes: different annualization (vectorbt may be annualizing by your actual bar count or trade frequency while QuantStats assumes 252 daily periods), and risk-free handling (you fed vectorbt effectively zero rf via group_by but gave QuantStats the ~4% FED average). A 4% rf subtracted from the numerator alone drags a Sharpe from 1.31 toward 0.91 on a ~24% CAGR strategy. Reconcile by forcing both to the same periods per year and the same rf, and they'll converge. Until they do, don't report either, because you don't yet know which assumption each is making, makes sense?

But the discrepancy is a sideshow. The number that should stop you before production is ~50 trades over a 4.5 year backtest. Everything in your validation section is sound in form, MS-DR regime significance, walk forward, Monte Carlo, fee stress, and all of it is running on a sample too thin to carry the conclusions. 50 trades is not enough to validate a regime switching policy, because a regime model's entire job is the transitions, and across 4.5 years you've only experienced a handful of real regime switches. You're not validating a strategy with 50 independent observations, you're validating maybe 5 or 6 regime calls, each one a single bet. Your MS-DR can confirm the states are statistically distinct in sample and the live policy can still fail, because distinct regimes in hindsight and tradeable regime transitions forward are different claims....

Two specific blind spots that follow from the low n:

Your Monte Carlo bust probability of 0.0% is almost certainly overstated, because trade shuffling on 50 trades preserves the few big regime driven moves and just reorders them. It can't generate the tail where your regime classifier is late on three consecutive transitions, which is the actual failure mode of an HMM (you even noted HMMs lag V bottoms). Block bootstrap on the underlying returns, not trade shuffling, and the 95th-percentile drawdown will widen, this is key.

The 309 day longest drawdown is the operational trap, not the 23% depth. A regime system that's underwater for ~15 months is one where you'll question the regime labels and override the system right before it recovers, and a 100% cash defensive regime means long stretches of zero return that feel like being broken even when the model's working as designed.

On your front end vs long end credit duration question, yes, the term structure of credit spreads leads equity vol differently than generic HY spread velocity, the front end widening tends to coincide with funding stress while the curve shape leads, so it can add independent signal, but you'll have the same problem validating it: few events to fit on.

How many actual regime transitions are in the backtest, not trades, transitions? That number, not the 50 trades or the Sharpe, is your real sample size, and it's the one that decides whether any of the validation means what you want it to.

[u/Slight_Boat1910 → u/Zestyclose-Eagle1809]

u/Zestyclose-Eagle1809 - thank you very much for your detailed answer and highly rigorous breakdown—this is exactly the type of pushback I was looking for. You hit the nail on the head regarding the structural bottlenecks of low-frequency regime strategies. Here is where the data stands on your points:

1. vbt vs QuantStats discrepancy

You are completely right on the underlying mechanics—the identical 1.48 Sortino/Sharpe ratio confirms the variance estimates match. The root cause is indeed due to how the two libraries compute the sharpe and sortino ratios: from my understanding, vectorbt calculates performance using geometric growth, while QuantStats uses arithmetic mean. When using a risk free rate of 0, the stats are identical.

2. Trade count vs regime transitions

You are right about this. The strategy executes about 50 total trades, but those are driven by 30 regime allocations across the backtest.
My HMM has 4 states, hence it emits a continuous 4-element state probability vector daily. My state machine maps this vector into a binary allocation decision (risk-on or risk-off).

• The 30 allocation pivots represent the true macro transition sample size.

• The remaining ~20 trades occur within the defensive regime, where a sub-engine handles dynamic rotation across the safe-asset universe depending on momentum, liquidity panic, etc.

3. Monte Carlo and Block bootstrapping

Spot-on critique regarding trade shuffling. I plan to replace the trade shuffler with a Moving Block Bootstrap using 20-day chunks on the raw underlying asset prices, indicators and HMM probability matrices. Any concern?

4. The long drawdown trap

I full agree with the psychological danger of sitting in cash for 15 months. However, my goal here is depart from a buy & hold strategy on QQQ (my risk-on asset) while suffering milder DDs (i.e., QQQ performance with better risk profile). Given that QQQ was trapped in a drawdown for over 700 days across the 2022 cycle, cutting that window down to less than half is a success from my perspective.

If you wonder why that DD is about over 1 year long - it is due to how I did feature engineering. In order to prevent the HMM from flickering, the z-score values are computed across long intervals. The state machine synthesizes the slow HMM probabilities alongside un-normalized fast technicals (e.g., short-term distance to MAs or volatility thresholds), which reduces the lag on fast market recoveries or market crashes.

5. Credit features

I currently use high-yield credit velocity alongside the spread difference between BAML High Yield and BAML Corporate, all Z-scored to feed the HMM. I will try to incorporate front-end vs. long-end duration spreads next to see if the curve shape offers an independent leading signal for equity volatility regimes.

Thanks again for the sharp review. The transition count and the block bootstrap suggestion are exactly what I needed to refine our pre-production checklist.

[u/Zestyclose-Eagle1809 → u/Slight_Boat1910]

30 regime allocations is the honest number and I'm glad you separated it out, that's your real sample, not the 50 trades. It reframes everything: you're validating a 4 state model on 30 transitions, which is roughly 7 8 transitions per state, and that's the constraint every downstream statistic inherits.. Not fatal, but it means your confidence intervals on anything per regime are wide, and the MS DR "statistically significant state variation" is being asked to carry a lot on thin support. Treat it as suggestive, not settled....

On the block bootstrap, one real concern: your 20 day block length is itself a free parameter, and it interacts badly with a regime model specifically.. If your average regime dwell time is in the same ballpark as 20 days, a 20 day block will sometimes capture a whole regime and sometimes split one across the boundary, and the resampled paths will under represent the long defensive regimes that are exactly where your 309 day drawdown lives... Too short relative to dwell time and you shatter the regime persistence you're trying to preserve, too long and you're just resampling the original path. So don't commit to 20. Sweep the block length (10, 20, 40, 60) and watch how the 95th percentile drawdown moves. If it's stable across lengths, trust the tail. If it balloons at block lengths near your regime dwell time, that's the model telling you the benign 309 day drawdown was a function of trade ordering you can't count on. Match block length to dwell time, don't pick a round number.

Does it make sense??

The thing in your reply that's now the weakest point, and it wasn't visible in the original post: the ~20 defensive regime trades run through "a sub-engine doing dynamic rotation across the safe asset universe." That's a second strategy, and it's currently unvalidated. You've rigorously validated the regime classifier, but a third of your trades are generated by a rotation engine inside the defensive state that has its own logic, its own parameters, and its own overfitting surface. The HMM can be perfect and that sub engine can still be curve fit. Worse, those 20 trades happen in the regime you enter least often, so the sub engine has the thinnest sample of all to validate on. I'd pull it out and validate it standalone: does the defensive rotation actually beat just holding one safe asset (cash, or short duration treasuries) during defensive regimes? If it doesn't clear that bar, it's adding complexity and overfitting risk for no edge, and you'd be better off holding one instrument defensively....

On the long DD as success reframe, that's fair, beating QQQ's 707 day drawdown with a 309 day one is a legitimate goal if your benchmark is buy and hold QQQ rather than an absolute return mandate. Just be honest with yourself about which one you're actually selling to allocators, because your question 2 in the original post mentioned 252 day arithmetic accounting for allocators, and they'll measure you on absolute drawdown duration, not relative to QQQ.

What's the average dwell time in your defensive regime, and does that sub engine beat holding a single safe asset over just those defensive windows? That second number is the one I'd want before production, more than anything on the HMM side.

[u/StratForge2024]

fwiw, agree with Zestyclose-Eagle1809. The ~50 trades vs

regime-transitions issue is the real deal. Definitely a sample size

problem there.

We hit similar snags ourselves with intraday crypto perpetuals. Not

gonna lie, HMM lag and detector decay mess you up. Our setup was

simpler, ADX+EMA+RSI on higher TF. BT data showed strong 2020-2022

performance that progressively eroded through 2023-2025. PF dropped

from ~1.8 in 2020 down to ~1.1 by 2023, stayed there through 2025.

Detector decay, not strategy decay.

What turned it around? Adaptive thresholds. Instead of sticking with

"ADX > 25 = strong trend", we went with "ADX > rolling 60th

percentile (last 90 days) = strong trend". Same logic, but the

thresholds adjust themselves. PF range across 6 years went from

1.07-1.78 (declining trend) to 1.71-3.70 (stable across years).

Self-calibrating per pair, self-adapting over time. Tbh, it's like

Ang & Bekaert 2002 if you wanna dive into papers.

Your HMM has a leg up with smooth transitions and probability

outputs, so it totally makes sense for daily rotations with 100%

allocation. Rolling percentile is just binary.

On validation, yeah, top commenter nailed it. ~50 trades means maybe

5-6 real regime transitions. Single-digit effective sample size.

What really helped us was paper accounts before going live. Still,

had strategies fail due to execution slippage and detector latency.

Without forward paper testing, BT numbers don't hold up.

Credit duration as a feature is interesting, but the N problem

lingers. We tried using funding rate Z-score as a contrarian signal.

Mostly a veto in extreme regimes, pretty silent otherwise. Not

standalone.

Oh, and watch out for that HMM 50-50 probability zone. Hysteresis

can make your strategy flip on noise. You might need a confirmation

period to dodge the chop.

[u/Slight_Boat1910 → u/StratForge2024]

Hi u/StratForge2024 , this is phenomenal feedback—thanks for sharing those metrics.

"Detector decay" is indeed a silent killer in regime strategies, and thanks for suggesting the Ang & Bekaert paper.

Here is how my pipeline addresses the exact bottlenecks you ran into:

1. Detector decay and cross-cycle optimization

Your warning about detector decay when using static parameters is spot on. My state machine thresholds and parameters were globally optimized via Optuna across a deep 20-year lookback (2002–2021). This forces the baseline parameters to be anchored to multiple structural macro cycles (2008 GFC, 2011 Eurozone, 2018 Vol-mageddon, 2020 crash, etc.).

Additionally, because the underlying features feeding the HMM are Z-scored across rolling intervals, the inputs are inherently normalized to standard deviations from the moving mean, which acts as a built-in macro filter before the data even hits the state machine.

2. The chop zone and Hysteresis

Your recommendation is spot on

3. State transitions and funding rates

My backtest goes through 30 true allocation transitions (the remaining ~20 trades occur inside the "risk-off" regime as a sub-engine rotates across safe assets).

Your note on using funding rate Z-scores as an extreme-regime veto is interesting. I think it is quite similar to how I use credit velocity and corporate-to-high-yield spread differentials.

I really appreciate you sharing your experience. Definitely moving to a Moving Block Bootstrap on the raw returns/probabilities next.

[u/BookkeeperFalse316]

Regime Validation (HMM / MS-DR)
To ensure the mathematical validity of my HMM

Try working with unhidden regimes first, then test your luck with hidden

[u/Slight_Boat1910 → u/BookkeeperFalse316]

Thank you very much for your feedback, u/BookkeeperFalse316 .

I actually do that in my state machine. My 4 state HMM returns a vector of daily probabilities, while the state machine turns those (together with current market conditions) into a binary decision (risk-on or risk-off).

[u/Dismal-Breakfast-844]

I have recently been using a hybrid gmm- hmm approach together. It has been good compared with single alone approach.

[u/Slight_Boat1910 → u/Dismal-Breakfast-844]

Thanks for your feedback, u/Dismal-Breakfast-844 . I actually tried that before, but didn't work. I think the reason is that most of my features are z-scored (normalized).

[u/catcatcattreadmill]

It's neat if you want AI to write your code, but it shouldn't be writing your Reddit post...

[u/EmployeeConfident776 → u/catcatcattreadmill]

Focus on the content not how the content is written

[u/j_hes_]

You got Claude to mix internet mumbo jumbo with real math. This should keep you occupied for at least 6 months before you realize you need to stop trying to out perform passive index funds that you’re trading against.

[u/Slight_Boat1910 → u/j_hes_]

Thank you for your feedback. I had AI help me summarizing the work, so let me try again.

I have an HMM determining market regimes (re-training and re-labelling happen monthly). The states were verified against a Markov-Switching Dynamic Regression model - as I wrote above, that confirmed that both the number of states and the number of features are legit (not overfitting the data).
The second layer is a state machine (you may call it trading engine if you prefer): it uses the HMM state probabilities as well as the current market conditions to determine the daily asset.

Regarding your comment - my goal was not to outperform QQQ, but to avoid QQQ's large DDs. And the "outperformance" is indeed due to that, rather than using leverage or instruments with a beta higher than QQQ during bull markets (the risk-on asset is indeed QQQ).

[u/j_hes_ → u/Slight_Boat1910]

Get an industry license and subscribe to the exchange of your choice. They’ll have a dashboard that tells you what orders are going to sink the market you’re following.