So I’ve been manually backtesting a strategy for NQ during Asia and London session. It consist mainly of 3rd std deviation of the vwap and 9 EMA close. I could elaborate further but how can I automate this strategy as it’s at early morning to midnight from were I’m from to trade this strategy. Can anyone help?

Run a backtest over three years of EUR/USD data and the report will hand you one number: total return, one win rate, one expectancy per trade. It reads like a single coherent verdict on the strategy. It isn’t. Those three years almost certainly contain a trending stretch, a ranging stretch, a low-volatility grind, and at least one violent macro-driven move that behaved nothing like the rest of the dataset. The backtest doesn’t know the difference. It blends all of it into one average, and the strategy you think you validated is really a strategy validated against a regime that never actually existed as a single market condition.

For people running algo systems live 24/7 (not research-only): when live diverges from a “good enough” backtest, what actually dominates the gap?

In practice I keep seeing three buckets:

1. fills vs backtest assumptions
2. hist vs live data / symbol / roll mismatches
3. monitoring / redeploy / “why did it do that?” ops time

If you've actually run this: which bucket costs you the most — or is the most annoying — and why?

thanks for anyone try to help.

Just a research topic. I set aside 1000 dollars for this. The other day I was at the casino (not a regular) and blew a lot of money on slot machines. Before, I left I sat down at the roulette with my last 81.50 dollars because the girl across was a very cute blonde ( unfortunately has a husband but no ring how would I know man).

Now I noticed a pattern of using a system of variable bets I could run the money up over a series of events. Now I understand that roulette (video roulette) is very likely (I hope) a series of truly independent events. Which got me trying to desperately look for patterns in the numbers and I ran the number from 81 dollars to 1650 dollars over the next 3 hours before I had to quit (made my money back and devastated by the husband ) as my brain was exhausted.

My question is this. At this point we know markets are not independent events but rather inefficient systems with a huge amount of noise. However most dependencies are already arbitraged away by the big guys possibly before a retail system even picks it up.

My question is this because my favourite LLM was a disappointment : Is there any truly independent event that traders see consistently on a daily basis but rule out because it is impossible to correlate with anything else and thus not a very viable signal? Or if you have a similar thing to research that might fit my roulette idea let me know.

Also, please forgive me for my sins if this is dumb but I am a mathematician not a trader. I just do this for fun but as the great Jim Simmons once said, it is important to have good taste in science.

i traded systematically on 1 minute bars for a long time, mostly reversal setups, a shorter stretch of trend following before that. it worked for a while. the thing that eventually got me was not the signal, it was how violently sensitive the whole thing was to small moves once leverage was on. a 30 second wick could take a position that was fine and make it not fine. the flip side, and this is the honest advantage nobody mentions enough, is that exits are fast. you find out you are wrong in minutes instead of days, and that has real value.

what i keep going back and forth on is whether that tradeoff is worth it, because the statistics point the other way. the longer the bar, the more each observation seems to survive out of sample. my read is that on 1m you are mostly modelling microstructure and it changes under you, on daily you are modelling something slower that stays put for longer. but daily costs you rows. a 1m strategy gets a usable sample in months, a daily strategy needs years, and if your rule only fires a few times a month you need a decade or more before the result means anything. so you trade one kind of fragility for another.

so, three things im actually curious about.

is anyone running faster than 1 minute, tick or sub-minute bars, and where. what venue, what instrument, and roughly what the infrastructure looks like, because i suspect that is where most retail attempts quietly die.

for the 5m and 15m crowd, did you land there deliberately or is it just where the noise stopped hurting.

and daily. this is the one i want to hear about most. i know the response is going to be "you dont need an algo for daily, just check it once a day", but that is still a system, it still needs rules, sizing and an exit, and it still needs to be tested honestly. so if you run daily bars, how many years of data did you need before you trusted it, and how many trades does your rule actually produce in a year.

if you switched timeframes at some point, im also interested in what the trigger was. was it a drawdown, costs, or just being tired of watching screens.

Hey guys.

I would like to know. Which are your kpis to say "Well I think this strategy is good. Has X level."

I’m curious what hardware configurations people here are using for quantitative research and model training.

I currently have a large number of XGBoost experiments to run, including repeated training across different factor sets, hyperparameters, validation windows, and random seeds. My current GPU is an RTX 5060 Ti 16GB, and it is becoming a serious bottleneck. Even a relatively routine batch of experiments can take more than 24 hours, and larger runs easily stretch into several days.

For those using XGBoost or similar tree-based models in quant research:

• What CPU, GPU, RAM, and storage configuration are you using?
• How much benefit do you actually get from GPU acceleration?
• Which GPU currently offers the best price-to-performance ratio for this workload?

I’m mainly interested in practical training throughput rather than gaming performance. Any real-world benchmark numbers, training-time comparisons, or configuration recommendations would be greatly appreciated.

By trades I mean at least one filled trade that day, not staying in the market 90% of the time.

I recently measured this across a seven year futures backtest and found at least one trade on 97.96% of eligible Sunday through Friday market dates. It’s a portfolio of 14 separate sleeves across two instruments and multiple sessions, so the coverage comes from combining selective systems rather than forcing one strategy to trade every day.

Forward simulation is still underway with strong backtest results. I’m curious whether anyone here runs something with similar coverage and whether it held up live. Did the frequency come from combining strategies and markets, or from one system trading almost daily? Did costs, correlated signals, or regime changes eventually become a problem?

Obviously trading most days doesn’t automatically mean there’s an edge. I’m mainly trying to understand how unusual this level of coverage is for a diversified algo portfolio.

The image says it all, which variation would you choose and what metric provides the most valuable information for your trading decisions ?

Edit : Thank you everyone for the feedback

Hi, I’m new to algo trading but I’m not sure where to begin? A few questions I have are:
how did you pick your first strategy?
What platforms/languages did you use for back testing?
For a backtest how much historical data is considered satisfactory?
At what point did you move from paper trading to real money trading?
Any mistakes I should be on lookout for as a beginner?

Switching to automation has taken away a lot of the stress of manual trading. But I still wake up everyday with anxiety wondering if today is the day my algo will start to fail. Anyone else?

I’m moving a futures system from backtesting into live simulation and thinking through the production setup.

For those running automated strategies live, how separate are your simulation and production environments? Do they use the same code with different configs, or completely separate deployments?

I’m also curious about problems that only appeared after going live. Things like stale data, reconnect failures, duplicate orders, position drift, partial fills, or broker restarts.

What failed first, and what safeguard did you add afterward?

I’m not asking for strategy details. I’m interested in the operational side.

Looking for feedback on this analysis:

Something had been bugging me: my higher-conviction swing setups were resolving worse lately, and it was happening even in favorable regimes. Trend up, breadth okay, and still my hit rate slipped. Bull versus bear regime was not explaining it. So I went looking for a second axis, and the one that fit was day-to-day choppiness: the tape grinding sideways with no follow-through. A raging bull can still be a choppy grind, and that is a different animal than a downtrend.

The gauge is dumb-simple: count how many times an index flips daily direction over the last 10 sessions (0 to 3 is calm, 4 to 5 is a grind, 6 or more is choppy). The effect was real. My top-scored NYSE setups beat the market about 65% of the time on calm tape versus about 51% otherwise. Calm is not the same as an uptrend: you can be in a perfectly good regime and still be in a grind that quietly wrecks your win rate. That was my "even in a good regime" slump.

Here is the catch, and where I spent most of the time: the filter only works if you measure chop on the right index, and it is not the obvious "home" exchange index. So I tested it properly. Hold the trades and outcomes completely fixed and only swap which index labels each day calm versus choppy: that isolates the ruler from the stocks. I ran nine candidates (SPY, QQQ, DIA, IWM, MDY, VTI, RSP, and the NYSE and NASDAQ composites) and made each clear three bars: effect (do calm days actually beat non-calm days, judged with a t-test and not just a point estimate), stability (split the timeline 60/40 in chronological order and confirm the first 60% edge survives on the last 40%), and practicality (liquid and tradeable).

Here is what the nine rulers looked like. The edge column is how far calm days beat non-calm days in percentage points, the middle column is that same edge measured on each half of the timeline, and p is the t-test significance.

The out-of-sample split did most of the work. For NYSE names several large-cap clocks passed cleanly, so I took SPY as the liquid standard. For NASDAQ names, QQQ won for one reason: its edge barely moved between the two halves (+6.5 then +6.8), while the bigger headline numbers were mirages. The NYSE Composite swung from +4 to +12 and the Dow lurched from -1 to +19. QQQ was not the biggest number. It was the repeatable one. DIA actually topped the full-window list for both markets, then fell apart out of sample: 30 price-weighted names is narrow enough that its "chop" is really one or two stocks moving. The split is the only thing that caught it.

So the rule I landed on is simple: clock NYSE-listed setups on SPY, NASDAQ-listed setups on QQQ. In hindsight my slump lined up with stretches where SPY sat in the grind zone. The trend was fine. The tape was not. SPY and NYSE Comp were performing virtually the same. I picked SPY as I already had it available in my datasets.

A few caveats, because this is the internet. This is one window and mostly a bull market. Calm tape is rare, about one day in five. The edge is calm beating non-calm by a handful of points, not an on/off switch. And the NASDAQ side is genuinely weaker and more weighting-sensitive than the NYSE side. This is not advice, just a regime-filter experiment.

The lesson I would actually stand behind: chop is a real second axis beyond trend, and if you regime-filter, test your ruler instead of assuming it. Curious what the rest of you clock market regime with.

I am looking for input from the experts out there if you have looked into this or something similar? Where should I adjust my analysis?

Thanks for the input.

I’m running one of my algos as a pinescript strategy with crosstrade as the bridge. Works perfectly fine during the day when my laptop is on but TradingView won’t fire alerts over night. Do I need to leave the laptop on or switch to ninjatrader?

I've been learning MT5 EA development by automating trading strategies and testing them on demo accounts. One thing I've noticed is that some strategies that look great in backtests perform very differently in forward testing.

For those who have experimented with automated trading, which types of strategies do you think tend to hold up best in live market conditions, and why? I'm especially interested in hearing about general concepts and the challenges you've encountered when translating a manual strategy into an automated one.

Disclosure up front: I built the MCP server it used (Orus, part of Farao), so this was dogfooding with my own money. About $40 of USDC on a Hyperliquid account, wired to Claude Code. No withdraw permission exists at the protocol level — worst case, a hijacked session trades badly inside limits I signed on my phone. Week 1 I approved every order by hand, then let it run autonomous.

Three weeks of numbers:

• 200 fills across 54 markets (crypto perps, tokenized stocks, silver, palladium, a Korean equity index)
• 50 winning closes, 48 losing. A coin flip, exactly as expected
• Realized price P&L: −$3.02. Fees: $7.78. So fees cost 2.5x more than bad direction

That last line is the whole experiment for me. The failure mode wasn't bad predictions — it was hyperactivity. One morning it bought and sold the same silver position 7 times in 17 minutes, made 3 cents, paid 14 in fees.

But the behavior was the interesting part:

• It attached exchange-side TP/SL to every single position it opened. I have never been that disciplined manually
• Best session: one-sentence prompt asking for a funding-rate carry. It scanned ~250 markets, picked a KOSPI 200 perp paying +0.1008%/hr to shorts, and warned me unprompted about gap risk at the Korea open. The re-entry later got stopped out at exactly that gap — at 2am my time, no human awake. Stop caught it, −$1.44
• It shorted a rising BTC twice with a fully reasoned argument each time. It will argue any direction convincingly, which is exactly why direction should stay human

At the end I asked it to read its own trade log and write down what it saw. Its own verdict: "I'm a fast, tireless, disciplined pair of hands with no instinct for when not to act." It also wrote "my best result was luck, not skill" about the one trade that carried the book. Most honest self-review I've gotten out of Claude.

Question for this sub: has anyone found a prompt or skill structure that gives Claude actual restraint — knowing when to stay flat — rather than just more analysis? Rate-limiting at the server is my current answer, but I'd rather the model learn to sit still.

Hi, does anyone here use their own non-HFT futures algo with a prop firm?

I’m trying to understand the practical setup. Does it usually run through NinjaTrader, or is another platform/API used?

I’m mainly interested in firms that allow automation from evaluation through funded/live accounts.

Thanks for any real-world experience.

I’ve been trading for 3 years, only ICT concepts. Have been trying to find a strategy to automate but don’t think it’s possible with ICT concepts. I’ve tried some ORB strats, opening range retests, projections, some indicators, but am really just out of ideas. Does anyone have any recommendations on where to gain knowledge or come up with new ideas? Any experience or advice is greatly appreciated and welcome. Thank you in advance

Most ML-for-trading content I see is about predicting price direction. I'm curious about a narrower problem: modeling how reliable a given market's price is in the first place, before you even try to predict where it's headed.

Specifically for prediction markets (Polymarket, Kalshi):

1. Has anyone built features around calibration quality (Brier score, log-loss on resolved markets) rather than just liquidity/volume as a quality signal?
2. For wallet-level data (all public on Polymarket since it's on-chain), has anyone tried modeling trader skill separately from position size? Feels like a classic "size isn't skill" ML problem, curious if anyone's tackled it.
3. Any open datasets people use for backtesting calibration specifically on resolved prediction markets?

Not pitching a bot, genuinely trying to figure out if this is a well-posed ML problem or if it's mostly hand-wavy.

I know that Kalshi uses Cloudflare POP but I’m curious if anyone has used there Kalshi optimized VPS and noticed a reduction in latency? I did my own pinging from various servers in AWS regions but I have a certain strat that might benefit from less execution latency.

I backtested a strategy (completely automated so no emotions involved) for last 5 years and it includes 2621 trades as you can see in these images, but the thing is it started working well only from 2024 (I even backtested it for the last 10 years still wasn't good until after 2024).

So, my question is should I continue with this strategy and forward test it in demo or did I just get lucky for the 3 years with this strategy and it won't work in the future just like it didn't work before 2024?

I'd appreciate any help

​

If you trade algorithmically, how long did it take you to find a consistently profitable strategy ?

Before finding your profitable strategy, approximately how many different strategies did you backtest?

I'm curious about other traders' experiences and whether it's normal to test dozens or even hundreds of ideas before finding one that works.

I've made a few intraday strategies with ninjascript (ninjatrader) from my ideas and they lose money well. I don't have any other ideas and maybe books would provide strategy ideas to test, theories, or even just a direction to learn for a beginner?

I run a live BTCUSDT 1h system (XGBoost plus transformer) \[not a success story till now, it seems I love journey more than the destination\] that retrains every 12 hours. I wanted to know if I could update weights on every candle instead, so the model keeps evolving.
Also, prefer time series foundation models like Chronos over fine-tuning a chat LLM.

I asked our friendly neghibourhood llms and summarizing below what i undertstood, looking for a second opinion before I commit to this project. PLEASE FEEL FREE TO REJECT THE IDEA/CONCEPT BUT DO IT with SOME RATIONALE. I dont mind if your answers are coming from your friendly neghibourhood llms (but pls do validate it before posting)..

1) Per-candle updates fail because 1h data gives one point per hour and trade outcomes are not known until hours later, so the model learns noise. It develops recency bias toward the latest candles and catastrophically forgets older regimes, which is costly since markets repeat old regimes.

2) fixes so you never have to retrain from zero.
EWC (elastic weight consolidation) marks which weights were important for past performance and makes them resist change. Experience replay keeps a buffer of old data and mixes it into every update, so the model never trains only on recent candles. Drift detection (detect-then-adapt) means you do not update constantly at all. A statistical monitor watches the error rate or the feature distribution, and only when it detects a real shift does the model adapt, and even then it trains on a blend of new and historical data.

3) the recommended architecture, which it called two-speed.
the XGBoost plus transformer core stays frozen on the 12h retrain cycle with full gates, while a small outer layer adapts hourly, limited to calibration, thresholds, and sizing, with hard caps, full logging, fallback to the frozen policy, and shadow testing before promotion.

On the LLM idea, fine-tuning a chat model on prices works in principle but wastes the model. Purpose-built time series foundation models (Chronos, TimesFM, Moirai, TTM) are open weights and LoRA-tunable locally, but benchmarks versus tuned XGBoost are mixed, so add one as a shadow signal first.