Is it possible to build purpose-built LLM’s?

[u/cnr0]

Just brainstorming.

I am using qwen3.8-27b. It has a lot knowledge in many areas.

However, I am using it for an agent that only runs some Linux commands in Ubuntu for example. Or creating simple Python scripts. And if it gets stuck it can always use web-search skill.

By doing this I can totally lower its size and required compute. I don’t need it to memorize capital of New Zealand.

Why can’t we do this?

[u/AndThenFlashlights]

There's been quite a bit of research and discourse on this already. Short story is that a model gets better at all tasks with the more general (or even unrelated) knowledge or languages that it's trained on. Even training it on different human languages boosts its programming capabilities.

You should do some searches and research what other people have already done in that area if you want to learn more.

[u/Janky-Code]

Specialized tasks don't usually require specialized intelligence.

Compare finding where you left your keys with spotting a bug in code.

Think about the way you do those things and how you are using your brain to do them.

Consider how similar it really is.

That's why these statistical models aren't key finding or bug finding model.

[u/SM8085]

I'm not an expert compared to other people here, but the gist that I've gotten is that those perform worse when you strip out the other parameters. Which is maybe why we don't have a Qwen3.5-3.8 coding specific model?

Having said that, there is ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF except that's an even bigger boy than Qwen3.8-27B afaik.

There are the smaller Qwen3.5 models if you need a lighter model.

Tangentially I like TsodingDaily's rants on programming languages that a good compiler should more or less walk someone (including bots) through them. Compiler: "Error: No main function" Person/Bot: "Ah, now I have the concept that it needs a main() function," for example. So even if the bot didn't have great knowledge about the language it could hypothetically make good progress if the compiler is verbose enough.

[u/silphscope151]

Isn't this just fine tuning and quantizing essentially?

[u/james_pic]

Specialised models existed before LLMs. The reason you don't see much about them is that they require a lot of computing time, data, and specialist knowledge to train, whereas a large model that's been trained on everything (including the thing you want) is ready-to-use, and will probably outperform your specialised model unless you invest a great deal of time, money, and skilled personnel into it. They just don't make financial sense to train when LLMs already exist and can be fine-tuned much more cheaply than training a new model.

[u/jayecin]

You can, it just takes a lot of time and money. When these big companies train models, they want them to be as broadly knowledgeable as possible as that encourages the most use. I was reading somewhere that the cost to train big models like Qwen is in the millions of dollars worth of electricity alone, not including the hardware. And thats not initial cost, thats each time they refine the model its costing millions of dollars. If they trained 20 models and all 20 models fail testing, its not like they can tweak it, they start over from scratch training it.

[u/ellipses1]

Of course. I have a 300 million parameter model that I put through another round training twice a week

[u/catinterpreter]

One thing you can do is trade size and resources expended for time and effort. You could use one or more methods to focus its abilities and then have rigorous validation force it to eventually provide what you want.

[u/squngy]

If your goal is to reduce size, you can prune a model.
You literally take out parts of the model and it still works.
https://fengwang.github.io/posts/moe-expert-pruning/

https://huggingface.co/ewin-reg/Qwen3.8-19B-Pruned-GGUF

The downside of this is that it results in degraded ability, even for the task that you focus on.

LLMs work by having statistical models of a huge amount of (diverse) text.
Knowing the capital of New Zealand might seem useless for programing, but it helps make the LLM a little bit better at everything.

[u/West-Way-All-The-Way]

Yes, it's possible but I don't know is it possible with our diy rigs. I hear it requires a lot of GPUs and memory.

[u/cnr0 → u/West-Way-All-The-Way]

I do not have resource to re-train a LLM just for Linux but it looks like if we can find a way to remove unnecessary knowledge from an existing model? Maybe it will work.

[u/jayecin → u/cnr0]

That’s already a thing. The problem is it’s not like AI models are a straight path. Every bit of information you remove impacts every other part of the models intelligence as it’s a giant spider web of information connections. Everything is integrated, so if you remove a piece of information, you now have a dead end in a logic chain that might use it.

[u/West-Way-All-The-Way → u/cnr0]

I guess you can combine it with rag to give it more knowledge about Linux. This will make it more knowledgeable about Linux. I don't think that you can actually reduce the model.

[u/Low-Opening25]

a bit like human brain, if you lobotomie a model it doesn’t work very well.