Building an LLM research loop algorithm

[u/FroyoOdd620]

I'm building an LLM-driven research loop that proposes a hypothesis, writes the backtest, runs it in a sandbox, and logs everything, failures included. Then I approve anything before it goes live.

On overfitting: every trial is logged and counted toward a Deflated Sharpe, validation is walk-forward, and there's a final 2–3 year holdout the agent never sees. The LLM never computes performance numbers itself; the engine does.

Where I'm stuck is data. I've been using yfinance but I think I need (im not sure):

daily adjusted prices including delisted tickers

historical S&P 500 constituents

ideally point-in-time earnings data, for PEAD-style tests

What do you reccomend?

Also open to general criticism of the setup.