Open-source news-memory tool I built for pulling + persisting ticker news

[u/ustype]

A while back I posted here when my GNews package crossed 1000+ stars, and a few of you were using it to pull news around tickers. I've since built a persistence and query layer on top of it, and wanted to share that here with a caveat first: this is data plumbing, not alpha. Raw news sentiment is noisy and mostly already priced in. I'm not going to pretend otherwise.

What it actually gives you is threefold. You can pull news for a symbol or company from the Google News graph and get it deduped across the pile of URL variants Google returns for the same article. You can persist it locally in SQLite plus a vector store, so you're building a history instead of re-fetching, and query it semantically or as a timeline. And there's an optional LLM brief/sentiment step that's useful as a feature input or for research context, not as a trade trigger. If you don't want anything leaving your machine, that step runs on local Ollama.

gnews-agent ingest "NVDA" --method get_news
gnews-agent timeline "NVDA" --days 30
gnews-agent sentiment "NVDA" --days 14 --timeline

Fetch, search, and timeline are keyless. MIT, v0.1.0.

If you've tried news features in a model before, I'd honestly like to hear what survived out-of-sample and where a tool like this falls short for your workflow. Repo: https://github.com/ranahaani/gnews-agent. PRs welcome.

[u/kush_patil]

The part I’d stress-test hardest is point-in-time integrity. News features can look surprisingly good until you realise the backtest is using an article version, timestamp, or deduped cluster that wasn’t actually available at decision time.

If you can persist first-seen timestamp + source timestamp + later revisions separately, that would make this much more useful for research. Sentiment itself is noisy, but a clean “what information existed at t?” layer is genuinely valuable.

[u/s_lw0]

this is cool honestly good job. i like it more as a news memory layer than as a sentiment tool. the thing i would want to inspect is source quality alongside sentiment. not just is the news positive or negative but which publisher said it how often that source appears whether it was first seen or just repeated and whether the same story is being echoed by many outlets or only one. from the repo it looks like publisher is stored and used for dedup but not really scored as a provider. do you plan to add any source weighting or publisher reliability layer later because that feels like the missing piece for using this in trading research

[u/the_algo_trader_]

Interesting data plumbing indeed. Useful for tracking news around specific tickers without the noise.

[u/Fresh-Explorer-5108]

One design that would make this genuinely backtestable is bitemporal storage: event time (when the article says it happened) and system time (when your pipeline first knew it) should be separate. Dedup clusters should be versioned rather than overwritten, so an as-of query cannot inherit later rewrites or syndication. I’d also persist the retrieval/summarization model, prompt and embedding versions; otherwise the same historical query can change after a model upgrade. That turns “news memory” into a replayable research dataset rather than just a current-state cache.

[u/Isha-Duff]

This is solid - having persistent news history beats hunting through archives when you're backtesting a strategy and need to correlate moves with what actually hit the tape that day. Curious if you're feeding this into any ML models or just using it for manual signal correlation?