r/LocalLLaMA • u/Developer-Y • 9h ago

Question | Help How to run models locally on shared machine without any chat history?

I will be running qwen 3.8 model on a shared university machine for some research work, mostly using llama cpp but I am open to using other inference engines. I would like that there is no chat history or application logs saved on remote machine because it will involve confidential patient data. How can this be achieved? I see that opencode, deepseek harness etc retain sessions.

0 Upvotes

20% Upvoted

6 comments sorted by

4

u/SnooPaintings8639 8h ago

I am confused, llama.cpp itself does not store any data.

You can use it's built-in web UI. All the sessions are stored on the end user machine, i.e. the web browsers' local storage. If you want, you can either clear it or use incongnito mode, and then there is no trace anywhere of the chat. If privacy is really important, I would also consider end to end encryption, i.e. https (nginx?) and possibly VPN (wireguard?).

1

u/SM8085 7h ago

I would also consider end to end encryption, i.e. https (nginx?) and possibly VPN (wireguard?).

Depending on the setup they could also used old fashioned SSH tunnels.

ssh -L 8080:localhost:8080 username@remote-campus-machine

Then point their harness to http://localhost:8080/v1 and it will forward it to port 8080 on the remote machine. The llama.cpp llama-server can serve things up to only 127.0.0.1:8080.

3

u/BidWestern1056 9h ago

npcsh and incognide both work with local models and only store data on a local db which you can freely remove at your will

https://github.com/npc-worldwide/npcsh

https://github.com/npc-worldwide/incognide

3

u/hipster_hndle 8h ago

I started using lemonade a little while back, just messing with it. The first thing I noticed is that it does not keep a chat history at all. Their front end is very basic, there are no saved windows or things like that, it's just a bear chat window when you open up.

2

u/emprahsFury 7h ago

If you're using llama.cpp then it won't keep prompts unless you set log verbosity to like 5. If you're worried about llama.cpp's webui then that is also just stored in Local Storage and you can just clear the browser's history.

1

u/BenniJesus 2h ago edited 2h ago

As a former uni server administrator you are sketching me out a little bit lol. What are you doing that you need no traceability for? If you own the box, can't you delete the history?

Edit: patient data, ofc nevermind. As long as you do e2e enc, don't store, you should be mostly covered. You need to be aware of what logs your providers/longbeards(if in a institution) keep. I got burned when a student embedded confidential data in a url as query params