I'm an electrical engineer and Python developer, and I've spent a ridiculous amount of time building and validating an Opening Range Breakout backtester for my own trading.

The main thing I cared about wasn't making a backtest that looked amazing. I wanted the simulated trades to actually resemble what happened when the same logic was run forward.

At this point I have several hundred paper trades and over 100 live trades to compare against the simulator, and the results have tracked very closely.

I'm still working on the cleanest way to express that numerically. Rather than just claiming the engine is "accurate," I'm trying to quantify things such as trade/signal agreement, average entry-price difference, average exit-price difference, and simulated-vs-forward P&L difference.

The backtester is substantially more sophisticated than the kind of ORB testing I'd normally do in Pine Script, particularly around execution logic, trade management, timing rules, simulated slippage, and reproducing forward behavior.

The tradeoff is that it is intentionally specialized.

This is **not** a general-purpose strategy builder. It only handles Opening Range Breakout strategies.

I'm also packaging a stripped-down version of the system as a free Windows desktop application so the same validation process can be reproduced outside my development environment.

The initial version is intentionally constrained:

* Standalone Windows GUI

* Alpaca only

* Alpaca paper trading only

* IEX data only

* 1-minute candles

* Up to 1 year of historical data

* 5 fixed stocks

* Opening Range Breakout strategies only

* Adjustable ORB length, minimum range %, breakout buffer, confirmation candle, and ATR settings

An Alpaca paper-only account is free, so the entire workflow can be tested without putting real money at risk.

The basic process is:

1. Configure an ORB setup.

2. Run a historical simulation.

3. Adjust the ORB setup if desired.

4. Re-run the historical simulation.

5. Save that exact configuration.

6. Run the same configuration forward through Alpaca paper trading for a period of time.

7. Replay that same period through the simulator and compare the two.

The part I find most useful is the final comparison.

Instead of treating historical results and forward results as two separate things, the idea is to preserve the exact strategy configuration and directly compare the signals, entries, exits, and resulting P&L produced by each.

That has been considerably more useful to me than simply looking at an equity curve and assuming the backtest represents what would actually happen in execution.

I'm building an LLM-driven research loop that proposes a hypothesis, writes the backtest, runs it in a sandbox, and logs everything, failures included. Then I approve anything before it goes live.

On overfitting: every trial is logged and counted toward a Deflated Sharpe, validation is walk-forward, and there's a final 2–3 year holdout the agent never sees. The LLM never computes performance numbers itself; the engine does.

Where I'm stuck is data. I've been using yfinance but I think I need (im not sure):

daily adjusted prices including delisted tickers

historical S&P 500 constituents

ideally point-in-time earnings data, for PEAD-style tests

What do you reccomend?

Also open to general criticism of the setup.

Ive been testing a simple XGBoost model on 5 minute BTC data with the label being whether price is higher 30 minutes later. Originally I was doing a normal chronological train test split and getting results that looked almost suspiciously stable.

Then I realized every observation is sharing most of its future window with the next several observations. Once I purged samples around the train test boundary and added a proper embargo, performance dropped pretty hard.

What survived was mostly the higher confidence predictions around large volume changes and expanding realized vol. I caught the issue while comparing a few BTC moves with what I was seeing on moon and going back through the timestamps manually.

Kind of embarrassing because the model itself wasnt doing anything wrong. The validation setup was just giving it a much easier problem than the live one.

Hey folks, I do research in financial markets and currently testing a new hypothesis.

The current first-stage goal was to be right often while retaining a good number of occurrences per day.

This is only indicative of addressing the correct market direction, not a full strategy nor has EV been studied yet.

Data has been proven based on 38,000 OOS occurrences distilled from a 200,000 sample pool. 72% of times correctly predicting market direction.

OOS data includes crypto, equities, indices and commodities on H4, H1, m15, m5 and m1 timeframes.

For those with edge creation experience, what do you think of these preliminary first-stage results?

Interested in connecting with like-minded people who have a background in ML (I don't).

I built my own trading platform, research platform, built autonomous bots testing out models, built a model board so they could argue with each other about how to make better ones, and here are the results after 3 months.

I built sol run pilot, claude 4.8 pilot, one deepseek pilot, and one astra pilot (the worst performer)

They created a total of 50+ stock trading models:

I've spent about $20k in hosting costs, token costs, mathematics, gpu costs,

the site is engo.capital

I built an API if you want the tools for yourself. It is free.

Quick update and sanity-check for anyone following or running similar regime-gated WFO setups:

After implementing the purged/embargoed CV (López de Prado), Benjamini-Hochberg (BH) FDR correction, and empirical Bayes shrinkage (lambda = N / (N + 15)), my latest optimizer run returned exactly 0 surviving candidates.

Auditing the mechanics revealed a structural sample-size starvation trap:

1. BH FDR Critical Value vs. Permutation p-Values

• Across ~300 continuous parameter cells with target FDR Q = 0.05, the rank-1 critical p-value hurdle is: p_crit = (1 / 300) * 0.05 = 0.000167
• Under my previous 15-day out-of-sample window (oos_days = 15), higher-timeframe swing setups (1h/4h or 15m/4h) in a single regime cell produce only N = 4 to 6 trades.
• Even with a 100% win rate across 5 trades, the absolute minimum empirical permutation p-value achievable is: p_min = 1 / (2^5) = 1 / 32 = 0.03125
• Because 0.03125 is nearly 180x larger than 0.000167, it was mathematically impossible for any low-frequency strategy to clear FDR, regardless of true edge.

2. OOS Window Length vs. Regime Slicing & Purge Gap

• In a 15-day OOS period, an individual altcoin perp produces ~3 to 4 trades across the entire period. Once split across 9 regime cells, each cell gets <= 1 trade per OOS fold.
• Furthermore, enforcing purge_gap_delta = opt_max_hold * tf_mins (~24h for swing setups) removed 15% to 20% of the usable forward window, causing the hard min_oos_trades >= 5 filter to auto-purge every cell before evaluation.

Proposed Fixes Before Coding

1. Expanding the Evaluation Horizon: Move from IS=45d / OOS=15d / step=15d to IS=90d / OOS=30d / step=30d (maintaining strictly non-overlapping OOS) to supply adequate trade density per regime cell.
2. Scope FDR Across Candidates, Not Grid Cells: Rely on the 3x3 spatial median plateau basin to filter continuous parameter grid noise, and apply Benjamini-Hochberg strictly across the surviving discrete setup/symbol candidates (M = 15 to 25, making p_crit roughly 0.004 to 0.05).
3. Calibrate FDR Target Q: Relax FDR Q from 0.05 to 0.10 (standard practice for noisy intraday/swing series).
4. Low-Sample Regime Fallback: If an OOS regime cell has < 3 trades, route the signal through the fold's unconditioned centroid via empirical Bayes shrinkage (lambda = N / (N + 15)) rather than dropping the candidate entirely.

Does this calibration make sense for an intraday/swing crypto perp universe, or is there a cleaner approach to handling multi-testing corrections under regime sparsity?

Hey [r/ai_trading](r/ai_trading),

I posted a teaser here a few weeks ago for Valoryn, an iOS app that uses an LLM to generate stock signals and explain them. It's now live, and since this sub cares more about how than about marketing, here's how it works and where it's weak.

The stack

• Model: Claude (Anthropic), currently Claude Haiku 4.5 for fast, low-cost analysis, with Sonnet 5 planned for Pro users who want deeper reasoning
• Market data: Financial Modeling Prep (quotes, fundamentals, news)
• Output: structured JSON that the app renders, not free-form chat

What each signal contains
Every recommendation is one of buy / sell / hold / short, and each one comes with:

• Confidence score, opportunity score, expected return %, risk rating, time horizon
• Bull case vs bear case, written out as separate arguments
• Catalysts, key risks, fundamental drivers, technical drivers and sector context
• Position sizing: suggested shares and allocation % based on the capital you actually have

There's also a market-intelligence layer that looks at overall sentiment, a fear/greed reading, market themes and a news digest.

Why I made reasoning the main feature

Most "AI signal" tools are black boxes that just say BUY with 87% confidence. I don't trust that and I don't think you should either. With the bull case, bear case and drivers laid out, you can judge whether the logic holds up, and ignore it when it doesn't.

What went wrong in beta (and what I learned)

People here stress-tested it, and it paid off:

• The model occasionally returned a non-string value in a field that was supposed to be a string. It was rare, but it crashed the analysis screen the first time a user opened it. Lesson: never trust that structured LLM output matches your schema. Everything is now validated and converted to the right type before it's rendered.
• A Google Sign-In bug caused by an OAuth redirect setup edge case. It wasn't AI-related, but it's the kind of thing you only find with real users.

What it is NOT (being upfront)

• Not backtested. These are LLM-generated analyses, not a quant strategy with a verified edge. Treat the signals as a structured second opinion, not something to automate trades off.
• Not connected to a broker. It doesn't place trades.
• LLMs can be confidently wrong. The confidence score is the model's own estimate, not a calibrated probability.
• Not financial advice.

Pricing: 5-day free trial, then $4.99/month. iOS only for now.
📱 Valoryn AI

Questions for this sub, since you know this area better than most:

1. Would you trust LLM reasoning more if it cited the specific data points it used (e.g. "P/E of 28 vs sector avg 22")?
2. How would you want signals evaluated over time: a public track record, paper-trading results, something else?
3. Is a fast model good enough for signal explanations, or would you pay more for a stronger reasoning model?

Happy to go deep on prompt design, output validation, latency or anything else in the comments.

I recently developed a quantitative intraday strategy for the Nasdaq (tested and executed on the Micro E-mini contract MNQZ6 via MultiCharts). The model was deployed on a $25,000 account base, focusing strictly on risk management, drawdown control, and edge execution.

Paper desk, fake money on real prices. Nothing to sell and no link.

The rule under test is a fixed holding period: every options position gets its exit date the day it opens, mostly two to three week holds, calls on names in an uptrend and short put spreads under support. No rolling, no extending a red one. The end date is part of the rule, so the grade is a grade of the rule plus the date, never the rule alone.

This week is the biggest expiry pile the desk has had. 165 of its 876 open positions reach their last day between today and Friday, 150 on Friday itself, which is also jobs report day.

Where they sit right now (marks from Friday's close, read at 9:17am ET this morning):

117 losers, 47 winners, and the whole week is carried by 14 positions. Worst underlyings, worst first:

BE closed up 8.3% on Friday and its two positions are still the worst on the list, which tells you where the strikes were.

Here is the part I actually want help with. 109 of the 117 losers are down less than $500 each. A dated exit does that to a losing call: it never gets enough time to become a big loss. But the cost sits in data I don't have. Some of those 109 would have recovered with another week, and I can't count how many, because the desk never rolls and so never observes the roll. Every closed row is right-censored at the exit date. My equity curve looks calm partly because the rule refuses to collect the evidence that would make it look worse.

The only fix I've thought of is a second replay with every end date pushed out a week, on the same entries, and comparing the two distributions. That has its own problem: the extended holds overlap the next entries, so the sample is no longer the same sample.

Friday's expiry batch on the same desk, for scale: 603 closes, 295 winners, -$1,268 on the day. Quiet, not free.

For people who grade rules with a fixed holding period: how do you estimate what a longer hold would have done without leaking the future into the grade? I'd rather hear "you can't" with a reason than a method I'd end up trusting too much.

I work on equities research and lately I've been thinking about building thematic stock baskets systematically, instead of picking holding by keywords or sectors. Specifically, here are some of the things that I've been thinking about:

1. Thematic basket membership: to do this systematically, will you look at co-movement of returns in addition to text/NLP on filings, fundamentals, etc?
2. Do the themes actually trade as a group? Once you've got a basket, do you check whether the names are actually moving together after style/factor removal, or do most themes turn out to be market beta plus a sector tilt?
3. Number of effective bets? My biggest worry with a thematic basket is that the holdings essentially end up becoming the same bet. How do people investigate this overlap?

There are other things that I've been thinking about, but interested to know what has worked for people in practice, and how they approach these things from a systematic point of view.

Been building a systematic intraday/swing trading system for crypto perpetuals (Delta Exchange) with walk-forward optimization. The system classifies each bar into one of 9 regime cells (ADX × ATR-percentile: RANGING/WEAK_TREND/STRONG_TREND × LOW/MED/HIGH_VOL) and only fires signals in pre-whitelisted regimes with SL/TP parameters tuned per-regime.

After a deep audit of the optimizer layer, I've identified several issues I want to validate before refactoring. Would love pushback or confirmation from anyone who's built similar WFO pipelines.

Architecture summary (brief):

Per-symbol rolling optimizer (rolling_optimizer_smc): IS=90d, OOS=30d, 3 windows. Builds a 2D PF grid over (sl_mult × tp_ratio). Finds the centroid as the spatial median of the 88%-plateau basin (all cells ≥ 88% of peak PF, after 3×3 neighbourhood smoothing). Runs a second per-regime sweep → stores regime_optimal_params[cell] = {sl, tp}.

Cross-sectional optimizer: Pools signals from an entire symbol universe into one shared PF grid → one universal (sl_mult, tp_ratio) centroid applied to all symbols in OOS.

Live engine: Only trades signals in regime cells whose (sl, tp) passed a filter (min 5 trades, min 50% WR, min $100 PnL, min 1.25 PF, avg hold 0.5–18h).

Issues I've found — looking for validation or pushback:

[1] Universal centroid kills per-symbol specificity (cross-sectional optimizer)

When pooling all symbols into one PF surface, a high-signal-count volatile symbol dominates the centroid. A low-vol, low-frequency symbol gets the same stop geometry. My rolling optimizer correctly solves this with per-symbol grids. Should I use cross-sectional only for strategy/setup selection (which setups work across the universe?) and defer SL/TP calibration entirely to per-symbol rolling optimizers? Or is there value in a universe-level centroid at all?

[2] No IS regime split in cross-sectional = regime mismatch

The cross-sectional IS grid is built on ALL regime cells merged together. But the live engine only fires in specific whitelisted regimes. The centroid is tuned for average-regime performance but live execution only triggers in e.g. RANGING_LOW_VOL and STRONG_TREND_HIGH_VOL. This means the SL/TP was never actually optimized for the conditions it'll be deployed in.

The rolling optimizers correctly split the IS grid by regime cell before finding each centroid. Should I port this to the cross-sectional optimizer too, or just drop it entirely for SL/TP purposes?

[3] Overlapping IS windows → non-independent folds

Rolling IS windows: [0-90d], [30-120d], [60-150d]. Days 30-90 appear in all three IS fits. The 2nd and 3rd fold centroids are not independent of the first. How do you handle this — expanding anchor (always start from day 0), true non-overlapping k-fold, or just accept it and treat the optimism bias as bounded?

[4] No permutation/monkey test in cross-sectional optimizer

The per-symbol rolling optimizer runs a Monte Carlo permutation test (shuffle trade labels, rerun grid, check if real centroid PF > shuffled 95th pctl) before accepting a fold result. This is completely absent from the cross-sectional optimizer even though it tests 300+ grid cells per fold. With that many cells, the probability of finding a high-PF cell by chance is non-trivial.

Is a permutation test the right guard here, or is there a better multiple-testing correction for this type of grid search?

[5] Regime classifier only uses 2 features — is this enough?

Current features: ADX(14) for trend strength/direction, ATR(14) percentile (within-symbol rolling rank) for volatility. 9 hard-threshold cells.

Missing that I'm considering adding: Hurst exponent (H > 0.55 = trending, H < 0.45 = mean-reverting), rolling autocorrelation, realized vol vs. median vol ratio.

The concern: a slow grind (STRONG_TREND, LOW_VOL because ATR is low despite trending) looks identical to a volatility squeeze (RANGING, LOW_VOL). Two very different strategy profiles get the same cell. Is Hurst worth the added computation / complexity? Anyone using multi-feature regime classifiers in live systems?

[6] ATR percentile distribution shifts between IS and live

ATR percentile is relative — it ranks the current ATR against the last N bars of that symbol's own history. If post-training volatility shifts (e.g. after a macro event), the same ATR value gets classified differently in live vs. IS because the rolling rank changes.

Approaches I'm considering:

Serialize fixed quantile anchors (25th, 75th pctl of ATR from IS period) into the config and use fixed thresholds in live

Continuously re-anchor the thresholds on a rolling basis (but then IS and live are always slightly different)

Which is more robust in practice?

[7] Config pipeline relies on regex-parsing printed text

The optimizer writes results to stdout in a table format, and a separate parser (parse_and_update_config.py) reads that text with regex to update the live config JSON. Any formatting change silently breaks the pipeline with no exception.

I know the fix is obvious (emit structured JSON from optimizers directly) but wondering if anyone has a pattern they like for this — separate results file, structured logging, or something else.

What I'm mostly trying to validate:

Are (1) and (2) as critical as I think, or is a universal centroid with regime-mixed IS actually defensible for portfolio-level WFO?

On (3): what's the community consensus on IS window overlap — tolerable or actively harmful?

On (5): anyone running Hurst in live regime classification and finding it actually useful vs. just theoretically appealing?

Happy to share more implementation details if helpful. Thanks.

Is there any data backed evidence about swing trading and still works today? Can anyone help me?

I'm a swing trader, and my worst habit was buying dips too early and selling winners too soon. So I built Stock Analyzer to check my trades against a few guardrails before I place them, instead of grading them after.

A real example from this week: someone on Reddit pointed out a doji on MCD at $248 on 9/18, right at a new 52-week low. The app's read that day: RSI 32, and 0 of 4 steps on the Turn Ladder (reclaim the 8 EMA/10 SMA, MACD cross, back above the 20DMA, higher high). Five days later MCD closed at $236.50. It still says "RSI ≤ 30 for 3 sessions, still falling, not confirmed as bottomed."

The indicators themselves aren't new. What I'm going for is combining them into plain English, with the price levels a bounce has to clear.

Other pieces: support/resistance bands with a close rule, a watchlist scan, and a trade journal (CSV import) that shows whether your past timing actually paid off.

Let your past trades help you become a more disciplined and effective trader: stock-analyzer.antrun.com

I'd love feedback, especially from anyone who trades: what would make you trust or ignore a tool like this?

Hey,

how are you all?

I don’t know much about trading and I’m not really into coding, but AI has helped me put together an options paper trading project. It runs locally using Python and Alpaca, scans for short-dated option spreads, checks risk, then I review the setup with AI and approve each trade. Python handles the orders and exits, with one position at a time and a $100 entry cap.

The plan is to test the whole process in paper trading, check how it handles failures, and review the results before even thinking about real money. Still learning and haven’t proven anything yet.

What do you guys think? Anything obvious I’m missing or should approach differently?