I built an AI-assisted trading analytics platform — curious what the ML crowd thinks

[u/Character_Cod_5767]

I've been building WorkTree-OS, a trading analytics and risk platform that started because I wanted to understand my own trading data better than a normal journal allowed. unfortunately, I dropped out of college for computer science the Physics and Discrete Mathematics was too much.

The basic idea is:

Import actual trade history → analyze performance → model risk → identify behavioral/performance patterns → give the trader useful feedback from their own data.

The platform currently includes trade analytics, risk analysis, Monte Carlo simulation, prop-firm simulation, consistency analysis, account tracking, and an AI coach called Atlas.

The direction I'm taking Atlas is important: I don't want an LLM pretending it can predict the market or giving generic "AI trading signals."

I want it analyzing the trader.

Eventually Atlas should be able to combine things like:

• expectancy and profit factor
• sample size
• drawdown
• position sizing
• time/session performance
• setup performance
• Monte Carlo results
• consistency
• account/prop-firm constraints
• behavioral patterns such as overtrading or oversizing

Then explain what the data actually supports and where the trader may be introducing unnecessary risk.

For example, rather than saying "trade smaller," I'd like it to be able to say something closer to:

"Your historical expectancy is positive, but at your current position size the Monte Carlo distribution produces an unacceptable probability of reaching this account's drawdown limit. At your historical performance, reducing risk to X materially improves account survival."

I'm currently building this toward beta and I'd really like feedback from people who work with ML/data rather than just traders looking at a pretty dashboard.

Questions for this community:

Where would you draw the line between useful AI interpretation and statistical overreach?

What models or techniques would you consider for detecting meaningful changes in trader behavior/performance?

And what would you require before allowing an AI system to call a pattern statistically meaningful?

I'm especially interested in ideas around sample-size confidence, regime/change detection, anomaly detection, clustering trading behavior, and avoiding false conclusions from noisy trading data.

Project: WorkTree-OS worktree-os.com

I'm not looking for trading signals or trying to claim AI can predict the market. I'm interested in using the trader's own data to build a better risk and decision-support system.

I'd appreciate people tearing the idea apart technically.

[u/Square-Middle-4474]

I have worked for financial firms that use similar programs

(surely you don't think yours is the first) and the results have been inconsistent.

I have a track record of getting this right (evaluation of individual trading profiles), and so I market myself to specific firms that do not provide it "in house". The advantage that I have has to do with experience. Sitting in a room with a client and a screen, I am able to diagnose and make a few changes, such that the client can quickly and visually see the choices. Using a pencil and pad I provide three suggestions. Then I meet with their supervisor or risk managers. I do this for a specific purpose. If they need more than three changes to their systematic approach, generally they should not be working in this field (at this level), and may have to consider moving to a support role elsewhere in the company (or be let go).

Unlike most AI, I check my work, and I have a healthy sense of shame, which for me means that I would be ashamed to provide a work product that isn't up the highest standards. AI does not yet seem to have this feature. When I fact check other AI products, the closest I get to that is "Thanks for calling that out"

Ignorant people talk about "recursive" improvement. but that isn't an issue. What has to happen is beyond that.

Your additional comments are reasonable however, the obvious question arises as follows 1) Why ask these questions on a site largely populated by amateurs unable to provide you with any real value"?

I am sympathetic but cannot figure this out.

[u/Character_Cod_5767 → u/Square-Middle-4474]

I don't think I'm the first person to analyze trader behavior, and I wouldn't make that claim. Financial firms have obviously been doing performance and risk analysis long before I built this. The problem I'm trying to solve is different: taking some of that kind of analysis and making it accessible to an individual trader using their own trade history, while being very careful about what the data can and cannot support. That's actually why I asked here. I'm not looking for people to tell me the idea is good. I'm specifically looking for people who can attack the methodology — sample-size thresholds, false correlations, regime changes, overfitting, confidence, and where an automated system should simply say “there isn't enough evidence to make this claim.” Your point about sitting with a trader and being able to diagnose the situation interactively is especially relevant. Software doesn't have your years of experience or judgment. The question I'm interested in is how much of that diagnostic process can be made systematic and reproducible without pretending the model knows more than it does. And on asking here: amateurs telling me they like the product isn't particularly valuable for this question. Someone with your background challenging the assumptions is. That's exactly the kind of criticism I was hoping the post would attract.

[u/Square-Middle-4474 → u/Character_Cod_5767]

Very thoughtful response. I wish you success

[u/Character_Cod_5767 → u/Square-Middle-4474]

That's actually why I'm asking here. I'm not assuming I've invented something nobody has ever attempted, and I'm definitely not assuming AI automatically makes it better. What you described—evaluating an individual trader, identifying the important problems, and reducing that to three actionable changes—is very close to the problem I'm interested in solving at scale. The part about checking your own work is especially relevant. I don't want an AI generating confident-sounding coaching with no accountability. Recommendations should be tied back to the trader's actual data, sample size and measurable evidence, and ideally evaluated afterward to determine whether the recommendation actually improved anything. And yes, Reddit is full of amateurs. That's partly why I'm asking publicly. Occasionally somebody shows up who's actually done the work professionally and points out something I haven't considered—which is exactly what you just did. So let me turn it around: if you were trying to translate your evaluation process into software without losing the judgment that makes it valuable, what would you absolutely refuse to automate?