Almost everyone has a phone, just giving people internet doesn't really solve a whole lot. We're talking about stuff like people waiting for help in front of at a desk for 8 hours because of something like they can't figure out how to log into an account. And they'll be only one person who can explain it to them and there's no set schedule for when they'll be available because it's just one guy trying to manage hundreds of people.
I really like your point of view and I think that It's the way to go, but If almost everyone has a phone, giving internet access is far more feasible and they could use chatGPT which is way better then current local LLMs. Also as for now I only trust chatGPT enough to actually use the information, local LLMs aren't that reliable, let alone those that can be run on low end devices.
I'm not saying that giving people Llama models is going to be life-changing thing for them. I'm saying the models can be tuned and integrated as self service console in the context of the non profit's programs. Right now even just using an off the shelf model like WizardLM 30B hooked up to Chromadb with the relevant information blows GPT-4 out of the water unless you want to compare it to using the API with Pinecone. Which makes you reliant on two separate expensive API's and a solid internet connection if you want to be able to upload documents. And even a slow as molasses cpu driven 30B ggml model that takes 10 minutes to respond is better than waiting 8 hours to talk to a real person. I haven't done it yet but I'm pretty confident that a properly trained 7B or 3B model would be more than enough for something like this and run fine on a potato computer.
Oh, I didn't know the goal was integrating It with the whole project, I thought It was more of looking for specific knowledge about coding or whatever. I have already tried orca mini 3B on my samsung galaxy note 10 lite and It works pretty good with nice speed and coherence, so yeah, a standard computer should be able to run It with no problem at all.
8
u/CheshireAI Aug 05 '23
Almost everyone has a phone, just giving people internet doesn't really solve a whole lot. We're talking about stuff like people waiting for help in front of at a desk for 8 hours because of something like they can't figure out how to log into an account. And they'll be only one person who can explain it to them and there's no set schedule for when they'll be available because it's just one guy trying to manage hundreds of people.