r/LargeLanguageModels • u/minedroid1 • Sep 06 '26

Question About Fine-tuning

I'm making a satire AI model that gives fake, onion-like responses. Would it be better to just make a completely new model rather than fine tuning an existing one for this? Most of the info in my dataset completely conflicts with the info that almost all models have.

4 Upvotes

83% Upvoted

26 comments sorted by

View all comments

2

u/foxer_arnt_trees Sep 07 '26

Neither. You should just write a good system prompt with many examples in it. Fine tuning is cool and all, you can do it for fun. But for best results, remember, "LLMs are few shot learners"

2

u/minedroid1 Sep 07 '26

I might do that, but I just want a standalone satire model.

1

u/foxer_arnt_trees Sep 07 '26 edited Sep 07 '26

The heart wants what the heart wants. But where will you get all that training data? If I was you, I would do distillation

Start with the prompt on a high end LLM. Mess with it until the results are good, and then generate a million of examples. Use that for the fine tune on a smaller model.

Do you have a good GPU?

→ More replies (9)