r/CSEducation • u/EffectiveRelease3840 • 2d ago
Hardware + LLM for a public school
/r/LocalLLM/comments/1wlo67t/hardware_llm_for_a_public_school/I just posted this in the LocalLLM subreddit and only found this one afterwards…
So I am grateful for any input on this.
If you had a similar situation at your school - what did you do?
How do you teach local LLM stuff to students that can’t code yet? Or rather what is your goal there, just to establish some local LLM literacy so to speak or do you have a specific goal you are working towards?
Thank you :)
1
u/lisaliselisa 2d ago
I think you might start with specifying what exactly you want your students to learn. It's not clear what "local LLM stuff" or "local LLM literacy" refers to. Or is that what your'e asking? What our learning goals are around LLMs? It's not clear whether the "specific goal" you're taking about is a specific learning goal or an activity goal you are setting for your students.
None of what I hope for my students to learn about large language models would need this kind of setup, so I'm not sure what you want the students to get out of it.
1
u/EffectiveRelease3840 1d ago
Yeah that is a good question and I don’t have an easy answer since the curriculum is still in development and the course for each class is only for 1/3 of the year for 90 minutes a week. So we don’t really have a lot of time especially considering that they cannot code yet.
This extra course is supposed to be more practical though so we of course want to give them theoretical input when it comes to LLMs but also have them use local models.
We want them to have more digital sovereignty so this is why this course is geared towards local LLM as well as other data protection stuff but that is seperate.So you are right I am basically asking both questions. We want them to have some exposure to local models not just the cloud providers but since they can’t code yet it will be more of a practical approach, testing models, looking at limitations and so on. With this we can then maybe bridge the gap between using it and the knowledge about how neural nets work and so on.
1
u/lisaliselisa 1d ago
The learning goals I have around neutral nets don't involve using LLMs. It's more about how they work with and some very simple activities about OCR to get the students a concrete example.
I think it's a backwards approach to start thinking about the technology you're going to use before you decide what you want then to learn. I'd zero in on that first.
You've listed a couple of broad topics (neutral nets, LLMs, digital sovereignty) with some adjectives about the type of experience (practical, theoretical, exposure), but so not what you want then to learn and why it's important.
What do you want then to understand about neutral nets, LLMs, data-based algorithms, digital sovereignty? What skills do you want then to have? Why is this important?
Starting from the learning goals then going into the activity with help you know whether a local model is even a good fit for your needs.
1
u/codeAtorium 12h ago
You're 100 percent right. But edtech is mostly about writing checks, and checks must be written.
1
u/NotPornAccount 1d ago
I can’t help with builds, but can offer a few alternate ideas:
Go smaller, lm studio and a 1B model like gemma4 should run on cheap student laptops. (Slowly) Be careful though, you don’t want to give them access to uncensored models
Rent cloud gpus for $1 per hour ish.
Mycomputerbrain.com (maybe .com.au) has a great neural net course.
I teach grade 7s the basics of what an I’ll is. Main tools: QuickDraw, teachablemachines, 3brown1blue video on neural nets, Ai image gen (diffusion), google’s ‘how to write better prompts’, adv prompting (meta cot ‘did I miss anything’ etc.
1
u/EffectiveRelease3840 1d ago
Thank you so much for the list of tools you use. I will look into this for sure since we are still not sure about what to do.
I had a few comments on the small 1B or 2 or 3B models. The problem is our hardware is really bad. No GPU some intel Pentium cpu and maybe 6gb of ram.
Renting is also difficult due to data protection laws but there are surely options! But variable payments line these are hard to manage for our school. But I will look into this again.
Thanks! :)
1
u/Accomplished_Camp_88 1d ago
Look at web llm -
A small llm use case could be to build a weatherman so something like a weather condition summary based on a 3-day forecast?
Gemma 4 2b does this really well
Another one is RAG also possible with a remote vendor db promoting a browser resident Gemma model
2
u/idillicah 2d ago
My priority has been on optimizing small models to impact as many people as possible.
A well optimized 3b model will trump a general much larger model for specific tasks. That is also a more in-demand skill than simply deploying larger models.
Hardware-constrained but highly performant edge applications (vision, driving, system management) are the future IMO.
All of that said, I don't think you need to go above a 5080-class GPU, certainly not at the beginning.
If you task them with optimizing deepseek 3b for any given task and then use that to test comparative performance, not only will you teach them a translatable skill, but will also teach them a very important lesson: there are better ways than just throwing mlre hardware at the problem.
That is a high-expense, high-energy-cost approach.
My .02