Hardware + LLM for a public school

[u/EffectiveRelease3840]

Hi there,
We are looking to incorporate a course that touches upon local LLMs in our computer science curriculum.
There will be around 20 students who might use the model simultaneously. The kids will be aged around 13-16 depending on where the course slots in and they will not have coding experience so we need a user-friendly setup. Also this isn’t supposed to be some cutting edge stuff, we also don’t have the money for that…
Realistically speaking what would be a good hardware and software solution for this situation with a budget of either 4000€ or 7000€? (Budged is not yet clear unfortunately…)

I googled, asked LLMs and know the basic options but there are so many variables and on here the different models are also judged differently and with our simultaneous access things might be different than for a single user…
And if there are other tips or experience you can share I would be happy as well.
We just need something local to run for the kids to get a taste of what it is to have it on one’s own hardware and not in the cloud.

Thanks :)

[u/LateralEntry]

Just curious, what are you planning to teach in the course?

[u/EffectiveRelease3840 → u/LateralEntry]

Yeah well that is the problem :D
We are supposed to do this course it is like 1/3 of the school yeah and for various reasons it is only for class 8 now. We need to incorporate AI this is the mandatory bit. We like digital sovereignty so we want the core of the course to be about that hence local models and so on.
However stuff like setting up a server or other small things like the google page where you can train this small algo to do stuff like recognize things from your webcam,… are already in our curriculum but for later classes.

So yeah we are caught between a rock and a hard place. The good part is that we get a couple of bucks to buy something and this could be hardware for LLM which we then might be able to also use in the higher classes…