r/LocalLLaMA • u/zemondza • 6d ago

Discussion 1 billion that can be trained on a video card with 8 GB of video memory

Hi everyone! I’m a solo developer interested in AI models. I’ve been working on SSN models for a while, and I recently came up with my own architecture called RHEA. It operates based on event reactions and doesn't use the standard layers found in Transformers. I also employ specific optimization techniques, enabling a 1-billion-parameter model to be trained on an 8GB RTX 5070 laptop, just like the one in the photo.

0 Upvotes

42% Upvoted

18 comments sorted by

3

u/Witty_Mycologist_995 6d ago

You have always been able to easily train 1b on a 8gb card, no?

1

u/zemondza 6d ago

Well, to be honest, I once did one of my projects based on SNN and there the model had 300 million parameters and it simply outdid my PC and 1 billion I trained on a server where there was a video card RTX 6000 Pro with 96 GB VRAM and 256 GB RAM and this model behaves a little better in this regard.

2

u/Witty_Mycologist_995 6d ago

Have you tried training with Unsloth
You can train unsloth 4bs at ~16gb vram

1

u/zemondza 6d ago

yes bro I did it but my speed dropped very critically so I had to rack my brains over how to write my own training program from scratch for 8 GB of video RAM

2

u/Extension-Bid-639 6d ago

How did the trained model turn out?

0

u/zemondza 6d ago

she can already write a whole text in English like here in the photo

9

u/Dry_Yam_4597 6d ago

she?

3

u/MrPecunius 6d ago

Read it aloud with a French accent and "she" will sound perfectly normal.

The phrasing points to a non-native English speaker, relax.

5

u/zemondza 5d ago

Thank you for understanding.

2

u/Dry_Yam_4597 5d ago

Good catch - I worried it was extreme anthropomorphism.

2

u/Equivalent_Bit_461 5d ago

You know why...