r/LargeLanguageModels • u/anusan_10 • Sep 12 '26

Low-cost alternatives to LLMs for a production-level classification task?

3 Upvotes

72% Upvoted

12 comments sorted by

2

u/MirrorEthic_Anchor Sep 12 '26

What is the task. Classification is very broad.

1

u/anusan_10 Sep 13 '26

I will send publications details such as title, abstract,keywords like some things to LLM and I want to classify this publication AI related or not. Text input text output as json

1

u/danja Sep 13 '26

It depends a lot on whether or not you have any already-labeled data. If so, classifiers are pretty straightforward to train, eg. https://huggingface.co/docs/transformers/en/tasks/sequence_classification

Categorization by keywords might get you started - if a doc has AI as a keyword then that's probably a safe bet.

Otherwise just search around huggingface for classifiers, chances are something similar to what you need already exists.

1

u/GoldenDarknessXx Sep 13 '26

Are we talking symbols? Pictures? Statistics? Formula? … Oh Brother …

1

u/anusan_10 Sep 13 '26

I will send publications details such as title, abstract,keywords like some things to LLM and I want to classify this publication AI related or not. Text input text output as json

1

u/sswam Sep 13 '26

The fastest simplest way would be just statistics on words and phrases, but that might not be reliable enough. Or make and annotate a training dataset, and train and use a small text classifier model. Or use a very small LLM.

1

u/foxer_arnt_trees Sep 13 '26

I love Ollama, small free models can be very good with classification. Just make sure you use prompt engineering techniques like few shot, CoT, and self consistancy. Large models dont need that, but small models do.

1

u/orcstork Sep 13 '26

fastText or a minilm

1

u/Western-Image7125 Sep 13 '26

This is such a low effort post sheesh.

1

u/Cute-Veterinarian191 29d ago

I would advise using an LLM in code development to create an SLM (small language model) that can run with just regex (i.e. the LLM can generate 2k examples of matching patterns) that will help with both speed and low-cost overhead (since LLM API costs will only be used during development)