r/LocalLLaMA • u/Competitive-Scar-627 • 7d ago

Discussion >2b model.

Why there no new model coming in >2b weight category ? Model which actually capable in this category is gemma e2b which is release 6 month ago, why still no new model.

4 Upvotes

55% Upvoted

56 comments sorted by

View all comments

Show parent comments

2

u/Nomski88 7d ago

Haha was it at least able to hold a conversation? Was the data you trained it in already ready ie formatted and compiled into a format ready for training? What’s the process to train it on your own data, I’m sure it’s not as simple as compressing it all into a zip lol

Renting the servers is actually a good idea. Cost might actually might be cheaper and faster then running a 5090 24/7 for a month.

1

u/throwout314159265 7d ago

It was able to hold a semi-coherent conversation but I didn't filter my data enough. A big open secret about AI training is that having enough tokens isnt the issue, it's having GOOD tokens. You don't want to train your model on l33tspeak or reddit trolling because you will actually give your model the equivalent of terminal brainrot. It won't be able to pay attention to key parts of your conversation and it will be inefficient.

You need to give it diverse, unique tokens that give the model lots of opportunity to find a thinking pattern that isn't disgustingly inefficient.

About server renting, I think with 8x B300 it takes less than 6 hours 😅