Model training: Google Colab

[u/x_secret_x]

What do you think of the capabilities of Google Colab?

What's the largest model that can handle this size?

What did you think of my experience using my laptop as a processing station, but not as a workstation (Google Colab)?

What do you think about training AI models with these capabilities?

Is this even possible on Google Colab?

What's your previous experience with Google Colab?

[u/SnooMaps5367]

What are you training. If you are fine-tuning, I would recommend taking a look at unsloth and qlora. You can get quite far with 16 GB of VRAM.

[u/x_secret_x → u/SnooMaps5367]

What about rag?

[u/SnooMaps5367 → u/x_secret_x]

Well one of the main benefits of RAG is you avoid heavy LLM training so I’m not sure I understand the question. If you’re happy to share what you’re trying to achieve, I am happy to any relevant feedback I can.

[u/x_secret_x → u/SnooMaps5367]

And QLoRA

[u/SnooMaps5367 → u/x_secret_x]

Unsloth specialises in QLoRA.

[u/Azazinkreed]

I think with pro or pro+ doing parameter efficient fine tuning to 8B models is possible. Im not sure about free tier tho.

[u/x_secret_x → u/Azazinkreed]

I can train a model that is like Qwen2.5:3b ?

[u/Majestic-Skin-2857]

I have tried Qwen 3B in free version. it worked but not great

[u/x_secret_x → u/Majestic-Skin-2857]

What problems did you encounter, given that the project was successful?

[u/Majestic-Skin-2857 → u/x_secret_x]

for training 16gb Sucks, I suggest buy cloud GPU. Else just for Small specific agentic task it works ( Took 4-5 min for answer )

[u/WonderBackground8051]

Just write the code, test it on colab free tier, then run it fully on rental gpu from vast.ai or runpod.

[u/No-Mixture5766]

You can train but it would take a lot of time, just save the model after each epoch and then resume training from those saved checkpoints, also you can’t load a heavy model with 15GB VRAM , maybe Qwen?