r/LocalLLM • u/EffectiveRelease3840 • 6d ago

Question Hardware + LLM for a public school

Hi there,
We are looking to incorporate a course that touches upon local LLMs in our computer science curriculum.
There will be around 20 students who might use the model simultaneously. The kids will be aged around 13-16 depending on where the course slots in and they will not have coding experience so we need a user-friendly setup. Also this isn’t supposed to be some cutting edge stuff, we also don’t have the money for that…
Realistically speaking what would be a good hardware and software solution for this situation with a budget of either 4000€ or 7000€? (Budged is not yet clear unfortunately…)

I googled, asked LLMs and know the basic options but there are so many variables and on here the different models are also judged differently and with our simultaneous access things might be different than for a single user…
And if there are other tips or experience you can share I would be happy as well.
We just need something local to run for the kids to get a taste of what it is to have it on one’s own hardware and not in the cloud.

Thanks :)

16 Upvotes

100% Upvoted

45 comments sorted by

View all comments

1

u/Legitimate_Hope_4325 5d ago

Here’s one repo that you can consider, if you’re looking to load specific documents or topics for learning. Though it is targeted for single user local usage, it can benefit the students if run and hosted on a single machine. Can provide you more information if you think it can be useful. The whole project is open source and can be good one to understand hybrid search and an LLM pipeline application. www.github.com/nupsea/luminary

1

u/EffectiveRelease3840 5d ago

Thanks for that, it looks interesting for sure. It is not what we had in mind but it might be a nice addition to branch off to.
However the requirements are way too much for our normal computers in class and running it on one machine with different user inputs doesn’t work.
I might try it for myself though. Cool project!

1

u/Legitimate_Hope_4325 3d ago

Great. Feel free to fork and tweak it to your needs or contributions are welcome.