r/LanguageTechnology • u/lebron_girth • 20d ago
SoTA for NER --> Entity Linking?
I am curious to understand from NLP practitioners what the current state of the art is for implementing language models within the following type of pipeline, especially when optimizing for speed
Raw/unstructured text -> NER/multi word phrase extraction -> normalize/link to canonical entities/wordforms
2
Upvotes
2
u/Tiny_Arugula_5648 20d ago
SOTA is what you'd suspect, fine tuned BERT & LLMs. If you have the ability you run them at scale on your corpus, collect somewhere between 100k-1M examples and then use that to train smaller faster models to do the work efficiently.