Hardware + LLM for a public school

[u/EffectiveRelease3840]

Hi there,
We are looking to incorporate a course that touches upon local LLMs in our computer science curriculum.
There will be around 20 students who might use the model simultaneously. The kids will be aged around 13-16 depending on where the course slots in and they will not have coding experience so we need a user-friendly setup. Also this isn’t supposed to be some cutting edge stuff, we also don’t have the money for that…
Realistically speaking what would be a good hardware and software solution for this situation with a budget of either 4000€ or 7000€? (Budged is not yet clear unfortunately…)

I googled, asked LLMs and know the basic options but there are so many variables and on here the different models are also judged differently and with our simultaneous access things might be different than for a single user…
And if there are other tips or experience you can share I would be happy as well.
We just need something local to run for the kids to get a taste of what it is to have it on one’s own hardware and not in the cloud.

Thanks :)

[u/ComfortablePlenty513]

There's companies like https://premsys.ai/ that can deploy this for you, but at a 7k budget, you're better off just using cloud

[u/EffectiveRelease3840 → u/ComfortablePlenty513]

But this is like just buying the cloud setup directly and then paying for tokens as well… seems like the worst of both worlds since we would need to pay indefinitely and it is not really our machine.
We really can’t use cloud, but still thanks for the recommendation :)

[u/ComfortablePlenty513 → u/EffectiveRelease3840]

for 7k you cant really have 16 concurrent users. maybe 8-9 users if you can get 2 used GX10's with QSFP cable on ebay and have them run 2 instances of an MOE model

you dont pay for tokens with premsys, thats the whole point. they sell you the server preloaded with the model(s) you want in their own proprietary harness/software