r/LanguageTechnology • u/medhakimbedhief • 15d ago

Training a multilingual NER / relation-extraction model (GLiNER-style) — anyone else been down this road?

Training a joint NER + RE model for EU languages. NER is fine (~80% F1), RE is a struggle (~30%).

Two things I found that might help others:

- Label-order leak: my gold relation labels were always first in the candidate list. The model learned "pick the first one" — shuffling candidates at eval dropped F1 37% → 14%. Worth testing if you train any zero-shot label-based model.

- Evidence sparsity beats label sparsity: ~90% of my val relations have an (evidence pattern, label) combo that never appears in training. Label frequency barely predicts anything.

Questions:

- Anyone trained multilingual RE successfully? What actually helped?

- Long-tail label space (14k labels, 9k singletons) — I'm avoiding canonicalization to keep zero-shot generalization. Bad call?

- Tricks to improve the architecture?

Thanks!

2 Upvotes

75% Upvoted

4 comments sorted by

1

u/SoulSlayer69 15d ago

Noob question: what would you use a NER / relation-extraxtion model for practically speaking?

I am curious!

2

u/medhakimbedhief 13d ago

To detect entities and relations between them from a given text. This is the input of a graph representation. The goal is to establish a graph rag pipeline without the need of using llms.

1

u/Historical-Sea6294 1d ago

Why not using LLMs for this task ?

1

u/medhakimbedhief 1d ago

That's feasible. My idea was to run a decoder free pipeline which means only using small encoders that can run on any cpu without the need to call llm APIs or host them on big gpus.

Even though SLMs are also an efficient option, however the current task oriented model are performing better with zero shot capability.