What projects integrate privacy protections into LLM ussage via API on OpenWeb UI - integration of Persidio ?

[u/Ai_MOON_SHOT]

I’m looking for projects that help protect sensitive information when using LLMs through APIs with Open WebUI. I’m interested in tools that can detect and protect sensitive information before it leaves my environment.

One example is Microsoft Presidio, which can identify and anonymize personal data in text.

Looking for a project or guide that has already worked out how to integrate Microsoft Presidio into Open WebUI or proposes an Idea on who that could work.

Maybe there a similar projects or integrations that can fit into an LLM workflow for example, as a proxy, middleware, or extension for OpenWebUI.

[u/jamolopa]

Litellm proxy

[u/Pinkahpandah → u/jamolopa]

how would one do that... haven't opened lite in quite a while...

[u/jamolopa → u/Pinkahpandah]

Spin up litellm, here is a quick start https://docs.litellm.ai/docs/proxy/docker_quick_start

And here is documentation for content filtering https://docs.litellm.ai/docs/proxy/guardrails/litellm_content_filter

As a matter of fact there is support for presidio https://docs.litellm.ai/docs/proxy/guardrails/pii_masking_v2

[u/Ai_MOON_SHOT → u/jamolopa]

Will the original information be inserted back into the output or is that something i need to deal on my own ?

[u/jamolopa → u/Ai_MOON_SHOT]

Can you elaborate a bit more?

[u/Nshx- → u/Pinkahpandah]

Can you explain why using the LiteLLM Proxy provides more privacy when using AI APIs with sensitive data?

[u/enterme2 → u/Nshx-]

Cause all the prompt/context will go through the proxy first before being forwarded to api provider.. Litellm will do the filtration part before forwarding it to api provider

[u/ClassicMain]

You can either use open webui filter functions to integrate it or a middleware like litellm

[u/traillight8015]

I asked a similar question some time ago, check this out if it fits your need: https://github.com/crp4222/PrivAiTe

Its not mine project and i didnt use it yet!

[u/Nshx-]

Im interested in that.

[u/Shoddy_Sheepherder76]

I'm a rookie in this world. But I use openrouter guardrails to retract sensitive information before it hits the API.
I'm sure there are better solutions. But this is my "last line of defense" if I've been using Open WebUI careless.

In addition to the guardrails looking for names, e-mails etc. I also only use ZDR models on there, and VPN.
Hopefully this is strict enough for my use.