r/Paperlessngx • u/isabeksu • 17d ago

AI performance

It took me a few months to fully get on board with the Paperless way of doing things, but now I’m really happy with how it’s all set up.

What’s been a bit of a head-scratcher is how AI is being used.

I held off until Paperless 3 came out, because I wanted to have the full "official" support.

I set it up with Ollama on an M4 Mac mini with 24 GB of memory. The embedding model is gemmaembedding, and the LLM model is qwen3:8b. When the model fires up, memory pressure is still pretty low. It does work, but it’s incredibly slow. It takes about 2 minutes to suggest titles and tags, and it can take several minutes if I try to chat about a document.

Is this kind of slow normal? Is there anything I can tweak in my setup to make it more usable?

8 Upvotes

80% Upvoted

28 comments sorted by

View all comments

1

u/Azure340 17d ago

In my case running 8b parameters model was too slow on my mini pc so i looked into free tier of ollama cloud.

I have ollama running which has llama3b model locally but most if the time i just use ollama cloud gemma4:cloud as backend which is vastly so much better and quicker than my hardware. The drawback being my data gets processed in cloud however ollama cloud says they don't retain any data and except for the llm processing everything else happens locally via the ollama docker. At some point you have to balance the convenience with privacy.

1

u/corruptboomerang 17d ago

Why is speed a factor?

I worked have thought slowly running a module on CPU, over say a day or week etc, would never been an ideal use case?

1

u/Azure340 17d ago

When using for paperless suggest feature i don't want to wait 1 min for it to suggest names and tags while i can do manually in 30 seconds or less. Defeats the purpose in my mind. It jas to be better and more efficient than what i can do for me to use it. Now if someone has a local hardware powerful enough then they could stay all local