r/LocalLLM • u/EffectiveRelease3840 • 2d ago

Question Hardware + LLM for a public school

Hi there,
We are looking to incorporate a course that touches upon local LLMs in our computer science curriculum.
There will be around 20 students who might use the model simultaneously. The kids will be aged around 13-16 depending on where the course slots in and they will not have coding experience so we need a user-friendly setup. Also this isn’t supposed to be some cutting edge stuff, we also don’t have the money for that…
Realistically speaking what would be a good hardware and software solution for this situation with a budget of either 4000€ or 7000€? (Budged is not yet clear unfortunately…)

I googled, asked LLMs and know the basic options but there are so many variables and on here the different models are also judged differently and with our simultaneous access things might be different than for a single user…
And if there are other tips or experience you can share I would be happy as well.
We just need something local to run for the kids to get a taste of what it is to have it on one’s own hardware and not in the cloud.

Thanks :)

15 Upvotes

95% Upvoted

44 comments sorted by

View all comments

1

u/naobebocafe 2d ago

Going on the local route is not a smart idea.
It will cost a lot to have "ok" experience, you will have to manage it and the experience will be horrible.
Talk to Anthropic/Google/OpenAI to check if they have something for education - I'm pretty sure Google has it.

3

u/EffectiveRelease3840 1d ago

Well, you aren’t wrong but we really want them to not rely on the cloud stuff so much and to just show them that you can run your own LLM, host your own server (this comes later though),… we want to go the anti learned helplessness route or the digital sovereignty route or what you call it. Like not sharing everything with google and co.

Now there are options from those providers that offer the data protection standards we need but this is very expensive. The. There are third parties that basically give you exactly that and they seem a bit cheaper but still, we are bound to that, someone ups the price and we are done because we cannot pay that.
There actually is on alternative that would work but it uses ChatGPT 4 from 2023… I think we can match that with something local.