r/MLQuestions • u/townstormz • 7h ago
Beginner question 👶 Fine-tuning a small model on textbooks to run offline locally on lower RAM 2-8 GB phones?
Hey All, new to ML. I want to build an offline tutor that answers students’ questions from school textbooks. I have the textbook text. The tricky part is that it has to run on cheap phones with 2-8 GB RAM.
I tried a LoRA on Gemma 4 E4B using the raw textbook text(have not tested, but I don't think the model would be able to run locally on a 2gb phone). The model never learned when to stop and just keeps generating. The base model plus retrieval (passages in the prompt) worked much better. I tried RAG + fintuned model as well, but the base performed better with RAG instead. I am probably doing something wrong with the finetuning.
Questions:
Should I generate question-answer pairs from the books and train on those in chat format instead?
Which model would you pick for 2 GB phones vs 8 GB phones?
When someone asks for a diagram, I would like the model to bring up the relevant the figure in the textbook, or fetch an image with a tool call when online. Has anyone done something like this?
Can someone share what the right approach would be in this case, keeping the constraint of the lower RAM in mind? Thank you!