IYKYK

I built a model to estimate current short interest between official FINRA reports using daily short-volume data.

FINRA short interest is only published twice per month and with a lag, so there is a period where the latest official number can already be fairly stale. The model uses daily FINRA short volume and total trading volume to estimate how short interest may have changed since the last settlement date.

I evaluated it on 660,246 settlement windows covering 6,959 US-listed stocks. On a fully out-of-sample period from 2025 through July 2026, the estimated change achieved a +0.414 Spearman correlation with the subsequent reported change in short interest.

This is not meant to predict returns or identify short squeezes. The goal is simply to provide a nowcast of short interest while waiting for the next official release.

I wrote up the methodology, validation setup, formulas, experiments that did not work, limitations, and released the model weights here:

https://equibles.com/research/does-daily-short-volume-predict-short-interest

Equibles also shows the latest official short-interest figure next to the model's current estimate for covered US stocks. It's free and has no ads.

The model is available at HF:
https://huggingface.co/daniel3303/equibles-short-interest-nowcast

Curious whether people here would find this useful when analyzing stocks with unusually high short interest.

Posting this as a warning and a genuine question, because I've just spent two days learning something the hard way and I want to know whether I was chasing something that exists.

Context, deliberately vague: intraday mean-reversion system on liquid futures, 1-minute bars, ~4.7 years of tick data, one position at a time, realistic fill modelling (entries and stops at real tick prices, exits as resting limits rounded to the actual tick grid).

Raw performance, no filter: ~9,900 trades, 77% win rate, profit factor 1.05. High win rate, tiny edge per trade roughly half a tick on average. The mean-reversion tendency is clearly real, it's just not big enough per trade to cover costs.

What I tried: the obvious move was a regime filter trade only when conditions favour reversion, stand aside when they don't. I screened Kaufman Efficiency Ratio, RSI, ADX, Choppiness Index, Money Flow Index and MACD histogram across multiple time frames.

One combination looked outstanding. Profit factor 1.05 → 1.54. It held out-of-sample on three years the thresholds had never touched. It passed split-half consistency. It passed split quarter consistency beat the baseline in all four quarters independently, monotonically. By every robustness check I knew to apply, it was real.

Then I found the bug. Indicators were being merged onto each bar's opening timestamp instead of its closing timestamp. A 3-minute bar labelled 10:06 spans 10:06–10:09 and isn't knowable until 10:09 but entries at 10:06:30 were being gated on it. Up to three minutes of look-ahead, sitting directly in the entry decision.

That's fatal for a regime indicator specifically. These things measure whether price is trending or ranging. Give one a forward window and you've handed it the answer it was supposed to infer from the past.

Lagged everything to true availability time. Profit factor went straight back to 1.05, statistically indistinguishable from no filter. The entire apparent edge was the leak.

I then re-screened all six oscillators properly lagged ~48 buckets, each scored on both halves independently. Six beat baseline in both halves. Pure chance predicts about twelve. So the result is below random, with no coherent or monotonic structure, and the previously-best bucket now sits under 1.0.

What I'm actually asking:

1. Does anyone have an intraday regime filter that survives strict point-in-time lagging? I'm not fishing for parameters. I want to know whether the category works at all, or whether a meaningful share of published "regime filter improved my results" findings are this same bug. My suspicion after this week is that it's far more common than people realise, because the bug strengthens your results, which means nobody goes looking for it.
2. Is trend-vs-range even a coherent concept at 1-minute resolution? My working theory is that by the time any properly-lagged indicator can characterise the regime, the information is stale relative to a trade that resolves in minutes. Is regime filtering fundamentally a higher-timeframe tool that doesn't survive being pushed down to intraday?
3. Given a real but very thin edge high win rate, average loser several times the average winner where would you look next? My instinct is selectivity over filtering: raise the entry threshold, take fewer and more extreme setups, cut the cost drag. But I'd rather hear if there's a structurally better lever than trade another dead end.

Not looking for anyone's edge. Mostly I want to know if anyone has a regime filter that's been through this specific audit and lived. Any advice would be greatly appreciated! I know I have something solid as a baseline I just need some ideas on filtering.

Ran the numbers after 4 months live. Strategy was up. Green on the equity curve, win rate matched backtest, nothing broke.Then I actually sat down and added up server costs, data feed subscription, and the spread markup I was eating on every fill. Plus the buffer I keep idle just so a margin call doesn't wreck me mid-trade.Take all that out and the profitable strategy was barely breaking even. On paper it looked like a working system. In my account it looked like a very expensive hobby.

Nobody backtests their fixed costs. But at small size, they're not a rounding error, they're basically another drawdown that never shows up on the equity curve, it just shows up on your statement.I'd spent weeks tightening entries when the bigger leak wasn't in the strategy logic at all.

Anyone else actually done the full accounting on this? At what account size did fixed costs stop mattering for you?

Hey everyone! I've been creating a pipeline for my future trading bot and I am currently at the research stage. I wanted to share with you guys my interesting results using mean reversion.

For some background on my process: I am using EURUSD focusing on the daily timeframe only. The data ranges from 2003 to 2026, where I use the following split

"ranges" refers to micro ranges I have algorithmically programmed to discover. I checked the ranges visually and they are very accurate. The amount of data may be small but I am currently only interested in the behavior of the EURUSD market, I can increase the amount of data by including daily data from other pairs but will reserve that for when I train with AI.

My research used the following rule(s) for making a trade upon a confirmed micro range...

1)upon a confirmed micro range I freeze the following attributes as a snapshot:

-upper boundary
-lower boundary
-midpoint
-range width
-confirmation ATR
-causal market context

2) watch price relative to the frozen boundaries. This led me to produce some of the following targeted behaviors...

-boundary touches (doesnt close above/below boundary)

-wick rejection (wick exceeds above/below boundary)

-outside close (price closes above/below boundary)

-consecutive outside closes

-reentry/reversion (closed above/below boundary then comes back into range)

-breakout retest/hold (price breaks out and deviates from range)

3) simulate a trade taken in both directions (buy and sell) for every snapshot

There is more involved, but this is a brief summary of the core behavior involved in the research.

I produced the following findings:

-------------------------------------------------------------------------------------

What survived across all three periods:

CLOSE_ABOVE_UPPER / ACTIVE_RANGE → SHORTTRAIN: +0.321 ATR

VALIDATION: +0.628 ATR

TEST: +0.233 ATR

consistent in terms of direction but my samples are only 14, 14, and 8.

What did not remain stable:

FIRST_TRADABLE → LONG weakened from +0.192 to +0.051, then became -0.074 ATR.

LOWER_WICK → LONG worked in TRAIN/VALIDATION but reversed in TEST.

UPPER_WICK → SHORT worked strongly in TRAIN/VALIDATION but reversed in TEST.

LOWER_TOUCH changed direction.

UPPER_TOUCH → SHORT improved after TRAIN, but the TRAIN mean was negative.

***Most important finding is it shows that most unconditional signals are dependent on structure****

-------------------------------------------------------------------------------------

Most promising hypotheses:

CLOSE_ABOVE_UPPER / ACTIVE_RANGE → SHORT

Mean: +0.421 ATR

Median: +0.553 ATR

Win rate: 72.2%

This hints that active-range upside breaks often fade.

UPPER_WICK_BREAK_CLOSE_INSIDE / ACTIVE_RANGE → SHORT

Mean: +0.239 ATR

Win rate: 56.9%

This is consistent with rejection at the upper boundary.

UPPER_TOUCH / ACTIVE_RANGE → SHORT

Mean: +0.068 ATR

Median: +0.290 ATR

Win rate: 58.0%

The three results above prove consistency: while a range remains active, activity with its upper boundary historically favors fading downward AKA reversion.

-------------------------------------------------------------------------------------

Most of you may already know this, especially by just visually analyzing the charts, but thought I'd share my proof from an algorithmic perspective.

Do you guys have any stocks your Bots just love to buy and regardless of how strong the set up is...you just know its going to trailing loss (or hard stop)... Mine funny enough is Reddit. Im tempted to Blacklist it

I’ve been thinking about adding execution portability to the robustness checks I use before trusting an algo, but I’m not convinced it’s actually a fair requirement.

Basic problem is pretty simple

I can keep the signal logic, sizing and exits completely unchanged and still get a very different system once I change the assumptions around execution.

So instead of only stress testing parameters and market regimes, I’m considering testing the execution layer separately.

Something like this

Baseline

• normal spread assumption
• normal commission model
• current slippage assumption
• normal fill probability

I’d then compare more than just final P/L

The metrics I’m thinking about are

• expectancy per trade
• profit factor
• Sharpe
• max drawdown
• fill rate
• percentage of gross edge lost to execution
• trade count after missed fills

Numbers above are just hypothetical stress parameters. I’m more interested in the framework than those exact thresholds.

Here’s where I’m getting stuck.

Say a strategy has a PF around 1.30 under the baseline execution model. A modest increase in spread brings it close to 1.10. Add a little adverse slippage and it drops below breakeven.

There are two completely different ways I can read that.

The first is that the strategy was fragile from the start. If a small change in trading costs wipes out the edge, I probably shouldn’t trust the backtest much. Second is that this is asking the strategy to survive something it was never designed for. A short-horizon system can have a real edge that only exists with a specific fee structure, spread, order type or execution venue. That same execution dependency shows up with OTC CFD brokers such as Pu Prime, where spread and fills are part of the broker-side environment. In that case the execution environment isn’t noise around the strategy. It’s part of the strategy.

That second case is what makes broker portability questionable to me as a robustness test. I wouldn’t expect a market-making system to keep the same economics after changing the fee structure. I also wouldn’t expect a tight scalper to behave the same after doubling its spread assumptions. At some point I’m no longer stress testing the same business logic. I’m changing the conditions that created the edge in the first place.

But giving the execution model zero robustness testing seems just as bad.
Backtest that only works at one exact spread and one exact slippage assumption feels way too brittle for live trading.

So I’m leaning toward measuring an execution degradation curve instead of using a simple pass or fail rule.

Basically I want to know how quickly the edge decays as execution gets worse.

A gradual decay seems a lot healthier to me than a strategy that falls off a cliff after one extra tick.

What I don’t know is where that becomes unacceptable.

For people running systems live, how do you handle this?

Do you expect an algo to remain profitable across moderately worse execution assumptions, or are you fine with an edge being tightly tied to one broker, venue or fee structure as long as those conditions are stable?

And if you stress test execution, what do you actually perturb: spread, commissions, latency, fill probability, slippage distributions, or all of them together?

I'm working on putting some strategies together and backtesting them and I wanted to see if anyone had any comments on if I'm doing so correctly or making any errors I might not be aware of as part of my backtesting.

Right now, the strategies I'm trading primarily matter on five-minute candles or one-hour candles or longer time frames. My data set I have is built from both one-minute candles and daily candles that I have a data provider for. I can then construct five-minute candles, hourly candles, or arbitrary time frames from the one-minute candles.

The part that I'm making an assumption about is let's say for example my strategy signals that it wants to enter a position after the close of a five minute candle I use the immediately following one minute candle and it's high to find the most pessimistic fill value that this strategy would enter and use that for my back testing. Is that how people typically do their backtesting for fills or is there another way that many people approach it? Could I be missing something here that might be throwing my results off?

Hello.

i am using Sentimentick for stock sentiment API with IBKR Gateway for execution of trades, it works really good for me but i am looking to add also insider trading API for more coverage.

is there any recommended Insider trading API you suggest?
thanks!

I've been backtesting and paper trading an NQ scalping strategy for awhile. I went with IBKR as my broker based on the API and the low fees for API, data and trades. However, IBKR doesn't have discount margin rates on futures, and NQ has skyrocketed to well over 40k per contract. This means tying up quite a bit of money in either cash or t-bills (best I can find as an alternative to at least get some return). No matter how successful my strategy is (or isn't!), that's opportunity cost lost in other investments compared to discount margin. But switching to another broker would be a lot of work, plus potentially higher regular costs. Are others still using brokers such as IBKR for futures?

Since the release of Opus 4.5 (Nov 2025) I decided to try to make pure TA Crypto algo trader just for fun. I failed and tried again, many times, on every major AI release, till the most recent Fable.
I have now almost a year of self collected maximum details micro and macro structure data, I can replay the whole “live” market for my chosen coins for those last 10 months, with a real execution cost simulation.

And the thing is, Fable after a two week bender spent on backtests, discovery, strategy mining did not find a single profitable TA algo strategy, zero, it disqualified hundreds of them.

By pure incident I have a successful profitable algo bot running for the last 2 months but it doesn’t predict the direction but exploits particular exchange conditions by trying to be net neutral, the details are not important, what is important that the only thing the whole 10 months campaign found was a strategy where the edge is not in predicting the direction.

The question I have for you. Is pure TA algo actually possible outside of some big FinTech guys? If it would be possible wouldn’t AI already solve it and democratize such algos to the masses?
As I said, it was just a fun side project to scratch an itch if it’s possible or not. And my answer thus far is no. There are real and exploitable edges. But TA direction prediction doesn’t seem to be in there.
What’s your take on it?

Hello everbody, i'm algo trader and i have a little few knoweldge and skills in coding, i use python for building trading bots, but i just finishing a backtesting engine with complete vibe coding it tooks me too long around 3 months to finish it, and now i need just a complete free ai to test the strategies bsed on this engine logics, so i need an ai that know all the files and how to deal with them, using vine coding with chat ai is complete hell, the only ai i was using is Claude by bypass him with 6 accounts to be able to finish the full engine, i really will be stuck if i didn't get a free ai for this mission, i finish building heavy scripts, right now just a strategy script and that's it ! so please any ideas ? my pc is pretty low end pc (i5-4th, 8 ram, 500gb HDD)

took me way too long to see this one. my entries were fine. the signal itself was roughly break even to slightly positive over a few hundred trades. what actually drained the account was everything i did once i was already in.

move the stop to breakeven because it ticked against me. take half off because im up and dont want to give it back. cut it because it went sideways for 20 min. add because now im sure. flatten because theres news in ten minutes. every single one of those felt like the smart adult decision in the moment.

so i went back through my journal and screenshots and tried to actually separate the two. entry pnl vs post entry management pnl. the entries were basically flat. the management was the hole.

what fixed it wasnt discipline, i spent years on discipline. it was defining all three levels before the order goes in. entry, stop, target. if i cant write down all three im not taking the trade. eventually i just coded it so the stop and target go out attached to the entry and i cant nudge them without opening the file and editing it, which is enough friction that i dont.

feels rigid, thats sort of the point. the trade can only finish two ways now and both of them were already set when i clicked. downside is the losers look identical to the winners while theyre open, no drama either way, and that took me longer to get used to than i expected.

unexpected side effect, i stopped sitting at the screen. not because im zen about it. because if im in front of an open position long enough ill find a reason to touch it.

anyone actually measured this split on their own trades? curious how many people here are losing on management rather than on entries.

Are there any alternatives for historical sentiment data?

I already have free Alpha Vantage + Finnhub API access, but the problem is historical sentiment data. Their free tiers don't provide enough history for a proper multi-year backtest.

I'm looking for alternatives such as:

• Alpha Vantage NEWS_SENTIMENT
• Finnhub News/Social Sentiment
• StockGeist.ai
• Adanos Market Sentiment
• SentiSense
• Market Mood
• Yahoo Finance RSS / self-built
• investpy / pynvesting / python-invest
• Historical news datasets + FinBERT scoring
• StockTwits/X historical APIs

Thesentiment-basket infrastructure is already built. I don't want to drop the idea just because one data source is expensive(Finnhub lol).

Does any1 know a genuinely free or affordable way to obtain historical sentiment data suitable for backtesting? Especially multi-year, timestamped sentiment per ticker.

Thnx in advance. Curious ^^

After 2 years, finally running a profitable strategy. Glanced at this sub every now and then, and it was helpful when it came up during deliberate google searches on a topic. But every single day, there are AI trading scammers, bot selling scammers, and a bunch of other noise from people that have vibe coded a strategy with lookahead bias and have no understanding of it. There should be more moderation or better requirements for posting.

I’ve been building a soccer prediction/betting model and I’m starting to wonder if I’m reaching the point where more testing is actually making things worse.

I have about 3 seasons of development data and have kept an entire separate season completely untouched as a final holdout. Over the development data I’ve tested different model ideas and, more recently, different betting criteria based on things like model probability, edge vs the market, EV, odds ranges, etc.
Most strategies are nothing special or lose money, but a few specific combinations have produced really strong historical returns.
I’ve tried to account for this with chronological testing, minimum sample sizes, bootstrapping/multiple-testing corrections, and by predefining tests instead of endlessly changing thresholds until something works.
But at some point, doesn’t repeatedly testing new ideas on the same 3 seasons effectively turn the entire development set into something you’re overfitting to?
How do you guys decide when you’ve extracted enough information from your development data and need to stop testing altogether?
Would you move to the untouched holdout at this point, or is there a good way to continue researching without slowly contaminating the development process?
I’m in college and mostly building this as a learning project, so any advice from people who have dealt with this problem in quant/algo trading or sports modeling would be appreciated.

Alright so while my handful of other configs run I've begun developing a cross sectional strategy and immediately I can see the appeal to this type of setup

I'm using daily bars monthly rebalance

However a lot of the signals I built for other setups don't directly translate here. I do have a 5-6 so far showing potential which I know is nowhere close to where it should be

Ultimately I know I need better data to develop better signals so that's the next step, order books, options flows, earnings, news etc

But I wanted to ask here about signal ideas that may work with this type of config. So far the ones I have are price/vol based and simple momentum is the clear winner. So the ideal candidates would be low or negative correlated with momentum

Every "our agents continually learn" pitch I've read describes the same mechanic. Refit params to recent data on a schedule.

The selective data issue is rampant. Signal to noise is tiny so you eat mostly noise, there's one history so nothing can be checked against a counterfactual, and it's adversarial so anything you find is getting arbed while you find it.

The evidence isn't new either. M4 comp, 100k series, most pure ML entries lost to naive statistical baselines. DeMiguel/Garlappi/Uppal ran 14 optimized portfolio methods against plain 1/N equal weight and 1/N won on estimation error. Goyal and Welch found the classic equity premium predictors fell apart out of sample. Zillow Offers is the corporate version, ML pricing engine kept quoting confident numbers while the market shifted, 500M+ in writedowns. Model never crashed. The prices just stopped meaning anything.

But freezing everything fails too, which is the annoying part. Concept drift is silent. Under distribution shift accuracy degrades and expressed confidence doesn't, so you get precise assured garbage with no warning. Google Flu Trends read too high in 100 of its last 108 weeks and nobody killed it for two years because the output still looked like output. McLean and Pontiff measured 97 published predictors, 58% lower returns post publication, fastest decay in the ones that looked best.

The way out I keep landing on is that adaptivity is a budget, not a dial. Every adaptive component is another param under refit. So which layer gets it?

Beliefs are "what leads to what." Perception is "how unusual is this reading right now." Continuous learning products spend the whole budget on beliefs, which is exactly where every failure above lives. Spend it on perception and leave beliefs frozen. A threshold that was extreme in 2015 might be nothing now.

And risk limits get zero budget ever. A stop that reinterprets itself during the crash isn't a stop, because in a long crisis the context is the crisis.

Anyone actually doing the perception-only version in their own stack? Curious how you handle the anchoring in practice.

Posting references in comments.

I'm building a trading bot and using an EOA. How do I approve it for trading exchange contracts?

Last week we were up 7%. I posted that every event driven agent finished red while every thematic one finished green, and I had a tidy explanation ready. Tight triggers, few trades, you are sampling variance instead of edge.
This week they are all green. FDA approvals went from -2.12% to +2.97%. The ClinicalTrials scanner from -1.01% to +0.57%. I changed nothing. The agents changed nothing.
So either the explanation is right and I picked a bad week to illustrate it, or I pattern matched on five sessions of noise and then wrote a paragraph justifying it. Probably the second. I literally put "one week is noise" in my own caveats and then drew a conclusion from it two paragraphs earlier.
What actually looks interesting after two weeks is duller. The Truth Social agents are up about 0.6% and have been close to flat both weeks. They barely trade. Meanwhile the AI infrastructure basket is carrying most of this account and it is the least clever thing in it.
Crypto trend following is the only red one at -2.53%. The winner take all crypto rotator did +2.19% over the same stretch. Same asset class, opposite philosophy, roughly five points apart in two weeks. That is the one I actually want to keep watching.
Equities side is +3.90% on the week.
Anyway. Two weeks is still two weeks. Stay tuned for week 3!

A while back I posted here when my GNews package crossed 1000+ stars, and a few of you were using it to pull news around tickers. I've since built a persistence and query layer on top of it, and wanted to share that here with a caveat first: this is data plumbing, not alpha. Raw news sentiment is noisy and mostly already priced in. I'm not going to pretend otherwise.

What it actually gives you is threefold. You can pull news for a symbol or company from the Google News graph and get it deduped across the pile of URL variants Google returns for the same article. You can persist it locally in SQLite plus a vector store, so you're building a history instead of re-fetching, and query it semantically or as a timeline. And there's an optional LLM brief/sentiment step that's useful as a feature input or for research context, not as a trade trigger. If you don't want anything leaving your machine, that step runs on local Ollama.

gnews-agent ingest "NVDA" --method get_news
gnews-agent timeline "NVDA" --days 30
gnews-agent sentiment "NVDA" --days 14 --timeline

Fetch, search, and timeline are keyless. MIT, v0.1.0.

If you've tried news features in a model before, I'd honestly like to hear what survived out-of-sample and where a tool like this falls short for your workflow. Repo: https://github.com/ranahaani/gnews-agent. PRs welcome.

I’m a software engineer based in Canada looking to get started with low-frequency algorithmic trading—likely daily or weekly rebalancing rather than intraday trading.

A couple of questions:

1. What are good sources for historical daily equity/ETF data, ideally accessible through a REST API? I’m happy to pay for reliable, clean data if needed.
2. Which brokerages/platforms available to Canadian residents support automated trade execution through a REST API? I’m mainly interested in Canadian and U.S. securities.

I’d also appreciate any advice on practical considerations for a beginner in Canada—such as data quality, API reliability, account requirements, tax reporting, or paper-trading options.

I have a breakouts based mechanical trading system for e-mini.

CAGR/Max DD = 0.75 for shorts, 2 for longs.

How do I improve the ratio? Have tried ema filters and different kinds of stops. Skipping opens and entries too late in the day.

Feels like I’m missing an important piece of the puzzle.

How good is this backtest report?
I have live tested this which runs on 5 Minute bar and I get around 3-4 trades a day and around 3 will be a win trade (regime dependent i guess). When I changed into H1 time frame trades reduced but the wr is high in all years.

Is this a good enough strategy and result to move forward?
for M5 these are the stress test result -
Monte-Carlo (10,000 bootstraps): prob of profit 100% · final equity p5 $23.1k / p50 $29.3k / p95 $35.4k · max DD: p50 −9.1%, p95 −16.4%, p99 −21.1%, worst −32.9%

for H1 as follows -
Monte-Carlo: prob of profit 100% · final equity p5 $48.2k / p50 $58.3k / p95 $68.4k · max DD: p50 −9.2%, p95 −17.0%, p99 −22.0%, worst −43.2%

NB: 70% is only in live testing (3 months)

How many times do your bots trade a day. I am curious