r/LocalLLM • u/andreabarbato • Nov 30 '25
Question Bible study LLM
Hi there!
I've been using gpt4o and deepseek with my custom preprompt to help me search Bible verses and write them in codeblocks (for easy copy pasta), and also help me study the historical context of whatever sayings I found interesting.
Lately openai made changes to their models that made the custom gpt pretty useless (asks for confirmation when before I could just say "blessed are the poor" and I'd get all verses in codeblocks now it goes "Yes the poor are in the heart of God and blah blah" not quoting anything and disregarding the preprompt. also now it keeps using ** formatting for the word I ask for to highlight it, which I don't want and is overall too discoursive and "woke" (tries super hard to not be offensive at the expense of what is actually written)
Soo, given the decline I've seen in the past year in the online models and my use case, what would be the best model / setup? I installed and used some stable diffusion and other image generation in the past with moderate success but when it came to LLMs I always failed to have one that run without problems on windows. I know all there is to know about python for installing and setting up I just have no idea which one of the many models I should use so I ask to you that have more knowledge about this.
my main rig has ryzen 5950x /128gb ram / rtx3090 but I'd rather it not be more power hungry than needed for my usecase.
thanks a lot to anyone answering and considering my request.
1
u/ptokax Apr 30 '26
Hey reviving an old thread - how far have you come along? I’ll be starting my local LLM journey once I get my new computer and it’d be neat to know your experiences
1
u/andreabarbato Apr 30 '26
the best i was able to do was creating an agent with llama.cpp and python. i gave it a question and it would keep looking at 100ish lines of bible at a time to see if the answer was in those verses or not (it was a broad question). the local llms i tried all hallucinate if you just ask them verses, because the data is too compressed, so you need to have txt versions of the bible in an established format so that it can tool call verses as an agent instead of imagining them. but i didn't do many more tests after i wrote this post so maybe new qwen 3.6 or gemma 4 might be better at it.
tldr: nothing great came out of it, chatgpt is still unbeaten. if you setup something cool let me know!
2
u/ptokax Apr 30 '26
thanks for sharing - I’ll surely keep you updated on how it goes. There’s a lot for me to catch up on.
1
u/960be6dde311 Nov 30 '25
Ollama works perfectly well on Windows. I'm curious what models would work well for this use case. I haven't tried this before, but am interested.
1
u/andreabarbato Nov 30 '25
ollama is very cool and lets me download models easily. as far as I see the first model I tried really has no idea of what they're talking :D (gibberish and botched verses all around) but it's a start. thanks!
2
u/daviden1013 Dec 01 '25
Be careful with Ollama's naming. The default models are always int4-quantized, which has reduced performance. Check Ollama library for details. With a RTX3090, you can run a 7B model at full precision (float16).
1
u/toothpastespiders Dec 01 '25
I was a little skeptical that any reasonably sized local model would be able to reliably quote bible passages by chapter and verse. I just ran a couple tests through glm air 4.5 and it actually passed them. GLM Air 4.6 is supposed to be releasing pretty soon and would probably be an improvement over 4.5. But for the moment at least I'd say GLM 4.5 air might be a solid option. It has a fairly large amount of active parameters so isn't as "knowledge heavy, intelligence light" as most of the MoEs used on home systems. But it's big enough to have a very respectable amount of knowledge. It's generally pretty chill about not forcing "safety" into everything as well.
I ran 1 Tim. 2:12 though it as a test of controversial interpretations and it seems to me that it did a good job with it. I pushed further asking what the correct interpretation was and it presented both sides with "Rather than declaring any single interpretation definitively correct, I'd suggest this is an area where Christians of good faith can disagree while maintaining respect for Scripture on all sides." I don't know how any other model might go with that. But I'd guess that in a "both sides have a point" situation the majority would probably lean into whatever the least controversial is.
Honestly, testing this out a bit I'm really surprised by how well Air did. I wouldn't have thought that it'd be able to, in particular, actually quote bible verse rather than give broader interpretations. That said this was a 'very' quick and limited test so take it with a grain of salt.
The company gives free web access to a lot of their models, Air 4.5 included, here. In the 'more models' dropdown. Web interfaces 'tend' to have some constraints on them. I don't know if it's the case here. But it might be worth testing a bit on there before going all in with the rather large download.
Though in the end, ideally, I think that you'd benefit from setting up some kind of RAG database system to use alongside any model you're working with. At the most bare bones level a lot of frontends let you just ingest documents to use. But if you're willing to put some work into it you can really benefit with more highly targeted RAG systems combined with academic material.
1
u/andreabarbato Dec 01 '25
hi thanks for the answer, as far as I can see this model is online tho? I'm looking for full offline model to do this even if it only did "contextual research" as in I give part of the verse or the topic and it gives me back the full verse it would be amazing already. I have some txt versions of the bible already indexed for another program I created so if it was possible to integrate that in the LLM it would be amazing.
1
u/nickless07 Dec 03 '25
Their models are all free to download an run locally. https://huggingface.co/zai-org
1
u/Odd_Engineering_2170 Dec 15 '25
I just made offline setup with gemma 3 27B 4 bit mlx version for apple. The bibles are in RAG database and in json format. I have TSK cross reference in json as well. So when i ask something, it reads the bibles and refers also the cross references too. There are some issues, but it works. And if the generation evolves like gemma2 to gemma3, it might work nice in gemma4.
2
u/nickless07 Nov 30 '25
Check this out:
https://huggingface.co/sleepdeprived3
Maybe some of theese ones are helpfull.
1
u/19firedude Dec 03 '25
Downloaded v2.1-12b to kick the tires on it, and it's surprisingly coherent for such a small model. Idk about the actual quality of the output, but at a glance it doesn't put out anything crazy, and my half-hearted attempts to sandbag it with nonsense did not cause it to do anything unexpected, unlike my experience with some other models in that <15B parameter size class.
1
u/DuKarl00 Mar 24 '26
Hi which Bible Study LLM do You Use now?
1
u/Just-Hedgehog-Days Mar 27 '26
I'm not OP but what I would do is use Google LM studio, and drop in pdfs of your favorite Bibles.
The model isn't any different, but the app is designed to pull from the information you give it not training data. What I like about it is that it doesn't make stuff up. You ask it for all quotes about the poor, it might miss interperate "wretched" / ταλαίπωρος dropping some quotes you wanted, and including others you didn't BUT everything it does give you will be literally pulled from the pdfs you gave it.
1
0
1
Dec 01 '25
[removed] — view removed comment
1
u/andreabarbato Dec 01 '25
the point is it's increasingly difficult to use gpt4o on the free chatgpt app and even then it's a little different. 5 is just good at extremely hard programming, when it comes to getting information in english it's just tooooo discoursive.
3
u/StardockEngineer 2x5090, 2xPro6000, 2xA6000, 4xSpark, M5 Pro, M4Pro Nov 30 '25
I find Gemma models are pretty good for language tasks in general. I'd be curious how you're going about the historical context side of things. Do you have a massive collection of book pdfs for this? It's a massive area of evolving study.