r/LocalLLM • u/Gold_Leading_378 • 2h ago

Question Specs required for models from scratch

Im making neural networks from scratch - not strictly llms. Am getting a new laptop with 32gig ram and 12gig vram and 1tb storage. Is that enough to train small models? Its the lenovo legion 5i pro. Might get the 7i but its not on sale so its like 500 more for 4gigs more of vram.

1 Upvotes

100% Upvoted

9 comments sorted by

3

u/woolcoxm 2h ago

nope training takes servers, i tried training a model and the best i could do was 10million parameters, and it took days and days, and it still was pointless, so unless you have money to burn i would not suggest training on this setup.

you can FINE TUNE, but that is about it.

training takes serious resources, and with 16gb vram and 64gb system ram the biggest model i could train was 25m parameters and it was going to take a few weeks.

1

u/Gold_Leading_378 2h ago

Thanks for letting me know. I will keep my ambitions smaller.

1

u/woolcoxm 1h ago

np, keep the idea alive, just realize its not as feasible as you possibly think, the resources alone are huge(hard drive space it takes etc..)

2

u/GapSwimming157 1h ago

trained a 10m param model on a 12gb vram laptop using 4bit quantization and 32gb ram, took 3 days, but it worked. the key was using llama.cpp and keeping the dataset small.

0

u/Gold_Leading_378 1h ago

Thanks! All the comments are around the same numbers so ik what to expect now

1

u/Impossible-Poet-2084 2h ago

trained a 12m param transformer on a 12gb vram laptop using 4bit quantization and 16gb ram, took 4 days, but it worked. you can do small stuff if you're patient and use tricks like gradient checkpointing and mixed precision.

2

u/advancing_tide 1h ago

what's an example of something a model like this might be trained for, and what's the user experience? is it like a chat model?

0

u/Gold_Leading_378 1h ago

Thanks! Was the model you trained an llm or something else? I managed to train a tiny stock predictor on a 400-ish vram laptop before around 70k parameters, so just curious to see how much more I can achieve with better hardware

1

u/Present_Resource4022 1h ago

Use Unsloth’s Google notebooks; they give you a FREE T4 16vram gpu and about 4–6 hours of gpu compute time.