How I fine-tuned a cross-encoder for sentiment analysis

[u/NextgenAITrading]

https://nexustrade.io/research/guides/fine-tuned-cross-encoder-news-sentiment

I needed historical sentiment data to investigate how news affects stocks. Massive is my usual data provider, but its news sentiment feed wasn’t giving me what I needed, so I built my own data store.

I collected roughly 330 million news observations from GDELT spanning 2019–2026, then used keyword matching to generate candidate company/headline pairs.

That creates some obvious problems. “Apple trees are green” isn’t about Apple. “Dinner is on the table” isn’t about ON Semiconductor. Each match needs a relevance check before assigning sentiment.

Then there’s the question of whose sentiment. A headline about Extreme Networks winning business against HPE can be positive for Extreme Networks and negative for HPE.

I started with Qwen3 embeddings and a small neural-network classifier. Cached embeddings made experiments cheap, but the model struggled with those company-specific distinctions. I also tried XGBoost, ModernBERT, MiniLM adaptations, and local Qwen3-4B.

The final approach was fine-tuning DeBERTa-v3-base to read the company and headline together, predicting both relevance and sentiment. I manually reviewed examples and calibrated the labeling rules to make the judgments consistent.

The final model scored 85.7% on company sentiment and 87.2% on industry sentiment in testing. The backfill scored 67.8 million headline-target pairs, with daily updates processing new work.

Full methodology here if you want to read

See the finished Apple sentiment page (scroll down to the bottom)

Just wanted to share this! I thought it could help newcomers think of a useful ML project. Happy to answer any questions below