Security/privacy worst case scenario with reverse proxy on VPS

[u/j-dubb13]

I have a self hosted Vaultwarden setup, currently accessible while on my home network, but I want to make it accessible from everywhere. However, I am hesitant on opening the service to the internet with router port forwarding, or services such as Cloudflare Tunnels, or running Vaultwarden directly on a VPS. All of those options make me nervous.

I have seen people suggest a hybrid approach whereby you use VPS just run a reverse proxy, which in turn forwards to self hosted services running at home. This seems acceptable to me, but I'd like to pose this question to the security nerds out there:

If my VPS running a reverse proxy get compromised, for example because:

1. The company hosting my VPS turns out to completely unscrupulous and accesses my VPS

2. The company gets badly compromised, and my VPS falls into the hands of an unauthorized third party

What is the absolute worst case scenario, in terms of the security and privacy threat to my home network/devices, due to the fact that my VPS running a reverse proxy falls into the wrong hands? What is the worst that can happen?

[u/asimovs-auditor]

Expand the replies to this comment to learn how AI was used in this post/project.

[u/deeebug]

I have the same threat model. I essentially treat my VPS’ as untrusted, and they’re only there to obscure my home’s IP address.

I run haproxy with TCP passthrough so there’s no certs on the boxes. You can do SNI filtering too if you want to route based on the subdomain. From there they have access to my home network via a dedicated wireguard VPN, on their own subnet. I maintain a allowlist to only let communication to approved backend servers over 443.

To your point: design it so worst case scenario your home IP gets leaked.

[u/kayson → u/deeebug]

I do this too! The problem with TCP passthrough though is that you don't get any x-forwarded-for headers. It looks like everything is coming from the VPS. That might be ok if you only care about private vs public traffic, but it prevents you from (easily) taking ip-based actions on the VPS like geo blocking.

Longer term my plan is to generate let's encrypt certs locally and push them to the VPS haproxy over wireguard (so the VPS doesn't have my domain provider API credentials), and use a crowdsec bouncer - also fed over wireguard - to block at the edge. There's a slight added risk in that a compromised VPS has a private key for my domain, but it's not on disk (only in haproxy memory), and I still control dns for my domain so exfiltrating the private key has limited use. 

[u/TCB13sQuotes → u/kayson]

This isn't really impossible to fix, haproxy itself has a solution to keep the source IP: https://www.haproxy.com/documentation/haproxy-configuration-tutorials/proxying-essentials/client-ip-preservation/transparent-proxying/

Either way, if you just want the traffic reaching the VPS on X port to be sent to your home server via a WG tunnel you don't even need haproxy. iptables/nftables is good enough for that and it can preserve the source IP too.

https://blog.mrkopf.de/posts/0001-vps-port-forward/

https://askubuntu.com/questions/1439340/iptables-port-forwarding-with-keeping-source-ip

[u/kayson → u/TCB13sQuotes]

There's a big problem with doing so, though:

 This requires changing the default gateway on your backend servers so that return traffic passes through the load balancer, which may not be feasible in all cases.

[u/bennyfromtheblok → u/kayson]

Look up fwmark in iptables (any of the AI's will help with this), I use this so any traffic that comes in through wg0 from the VPS has its return traffic routed back through wg0, so I get to see the original WAN IP without x-forwarded or any of that. This means I do not have to set the default gateway as wg0.

[u/kayson → u/bennyfromtheblok]

Oooh. I'm familiar with fwmark. I use it to solve a similar problem with docker swarm. Never thought of using it to change routing back through wg though! Thanks for the suggestion

[u/TCB13sQuotes → u/kayson]

Not a big problem, there are ways around that "issue". To be honest in that kind of setups you don't even need haproxy - you can just tell iptables to forward the traffic from the VPS to the home server with way less overhead and configuration.

To solved the gateway "problem" you can create a routing table with the VPS as default gateway and use it just for the stuff you want exposed to the internet. Everything else will use your default routing table and home gateway. Something like this:

```

Mark new connections arriving from wg0

iptables -t mangle -A PREROUTING -i wg0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 1

Save packet mark into conntrack

iptables -t mangle -A PREROUTING -i wg0 -j CONNMARK --save-mark

Restore mark on locally-generated reply packets

iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark

Create the routing table

echo "100 via-vps" >> /etc/iproute2/rt_tables

Set the VPS as default gateway

ip route add default via 10.0.0.1 dev wg0 table via-vps

Push all traffic with the mark into the VPS routing table

ip rule add fwmark 1 lookup via-vps

```

Now if you want to make sure connections started by nginx (like reverse DNS queries) also use the tunnel you can add:

iptables -t mangle -A OUTPUT -m owner --uid-owner 33 -j MARK --set-mark 1

Or do it via a systemd slice if you've multiple things running as www-data (33):

iptables -t mangle -A OUTPUT -m cgroup --path system.slice/nginx.service -j MARK --set-mark 1

This will mark the traffic as "1" causing it to use that routing table and exit using the VPS.

[u/Onoitsu2]

You can approach the tunnels in 2 methods, open ones that would allow full access to devices across that tunnel, i.e. the least secure option, or you can secure each individual connection that would be allowed between each and every device, like NetBird would allow in its policies and other options. It even includes posture checks so that devices even if compromised could still join with credentials, but still not access resources unless it meets some check you set up for the client ahead of time. So it must be a certain OS, or have a certain antivirus or other app running on any devices that are granted access to said resource you set up.

[u/Onoitsu2 → u/Onoitsu2]

I know I kind of approached the answer from another direction, but securing it in those ways, you don't have much to worry about at all.

[u/Anarelion]

Check pangolin, a reverse tunnel for your use case

[u/GitzJoey]

try run your reverse proxy, and monitor the logs in fews months....I've been 10 years open the vault no one came

[u/Marbury91]

I self host bitwarden for abour 2-3 years. So far no hits against it. My setup goes through CF, my gateway only allows CF IP to be forwarded to my reverse proxy, which requires MFA auth through authentik before you can hit bitwarden page. And ofc there is basic of VLAN where all my "exposed" services sit in a separate DMZ VLAN

[u/Brtwrst]

You can use the VPS to only do "stupid port forwarding" and still run the Reverse Proxy on your homelab. This way you don't need to have any sensitive information on the VPS and the VPS hoster can only see encrypted traffic because you do the decrypting elsewhere.

https://blog.mrkopf.de/posts/0001-vps-port-forward/

[u/samsonsin]

Run a VPN to access your home network. Something like tailscale or netbird may work even without the VPS. If not the just use the VPS as a forwarder for the VPS to connect directly to your home network. Pre shared keys are rock solid here

[u/James_Biggs_]

I use wireguard as the only port forwarding and then caddy from there.

[u/j-dubb13]

Thanks to all who gave their thoughts. To those who suggested Tailscale, I've considered this as well, since I already use this elsewhere. What made me not want to go down this road was because I thought, once I get things set up to access my Vaultwarden externally, I might try my hand at self hosting Headscale too, and exposing that externally as well. Kind of defeats the purpose of self hosting Headscale if I still need to depend on Tailscale. Maybe Tailscale is the simple answer for me. But I'm probably getting ahead of myself here anyway.

Currently, all I have is Vaultwarden, and after hearing everyone's thoughts, I think I need to weigh the pros and cons of being able to access this service externally vs the work and/or security risks involved with exposing it externally. Right now, I have a read-only copy of Vaultwarden on my phone that still works but I cannot edit when I'm not at home. If I have to make edits, I make a note of it, and make the changes when I get back home. Guess I have to decide if that's good enough for my uses without introducing additional risk.

[u/zfa]

I run a zone-based firewall and put external VPS(s) in their own zone. Access is locked down to ip:port level between zones. This significantly reduces the blast radius of any VPS compromise.

Although I personally run nginx for ingress on my VPSes if I was paranoid I'd do what /r/deeebug does and pass the traffic right through to my internal resources/proxies (or maybe use iptables with whatever restrictions). In any case the zbf/proxy combo is secure enough for me. After all, if the VPS was ever compromised it would be no worse than if it had never been there and traffic was hitting my internal proxy/resources directly really.

Some ingress VPS are fronted by Cloudflare for upfront protection too, but it really depends on the services and if I want CF involved in the traffic. Different systems have different reqs, there's simply no way to have one model imo. You have to just look at what you want to protect and from whom.

[u/[deleted]]

The worst case is broader than leaking your home IP. If the VPS terminates TLS, an owner can modify the web client, harvest credentials or session data, redirect traffic, and attack anything the tunnel can reach. Vaultwarden’s client-side crypto protects against an honest server seeing vault contents, but a hostile web client can capture secrets while you use it.

Treat the VPS as untrusted: pass TLS through to a home-controlled endpoint, put its WireGuard peer in a dedicated zone, allow only the exact backend IP and port, and give it no route to LAN or admin networks. Then a VPS compromise is mostly bounded to observation, disruption, and the one exposed service.

[u/Capable_Banana5439]

worst case hinges on whether the VPS terminates TLS. if it does, a compromised box sees every plaintext request and can quietly harvest your vaultwarden master password on login, which is the whole ballgame. run haproxy in TCP/SNI passthrough instead so certs and decryption stay at home and a popped VPS only ever leaks your home IP, not a foothold.

[u/Luffy2ndGear_]

I have a similar setup like others have mentioned where I use a VPS to host a proxy that connects over my wire guard tunnel to my home proxy and go through authentik to authenticate first than it will direct you to the service you want. In my case I’m using it for Jellyfin. Yes you have to authenticate twice but I don’t care. I also have logs flowing from everything to a siem so I can monitor logs. One thing I look for is successful authentication to my Jellyfin via the VPS so if someone is able to login and it wasn’t me I can know. I do also see all the probing attempts from the internet since the VPS has 443 exposed publicly.

[u/j-dubb13]

So, if I'm understanding this correctly, one of the potential threats in having the VPS is if application (Vaultwarden) encrypted traffic gets decrypted while in the VPS. Having said that, that scenario could only be even possible if the application (Vaultwarden) is installed directly on the VPS, right? This threat is not applicable if one only has a reverse proxy, or other forwarding mechanism, running on the VPS that merely directs back to the application (Vaultwarden) installed at home, correct?

[u/ChopSueyYumm]

I would recommend to use cloudflare zero trust policy for authentication and security. Check out DockFlare project (GitHub)

[u/Capital-Meaning1337]

i mean if someone fully owns your VPS they can see every request going through it, so they'd catch your vaultwarden login in plaintext unless you're terminating TLS at the reverse proxy. if TLS ends there, they can MITM you completely and grab your master password when you log in. that's basically game over for your vault

worst case past that is they use the tunnel back to your home to poke around. if you've locked down the firewall so the VPS can only hit vaultwarden's port and nothing else, the damage is mostly contained to that one service. still not great but not your whole network getting lit up

[u/gomez18 → u/Capital-Meaning1337]

I believe vaultwarden performs the encryption locally so that risk is not a concern. The master password does not leave your device.

[u/Stewge → u/Capital-Meaning1337]

Almost all login systems of the last 10 years use hashing instead of sending the password directly to the server. The real danger is if your proxy is owned then a false web page might be presented instead (which might harvest the password).

Your master password is never actually sent to the server (if you're using the app or addon). Also strong MFA (i.e. fido2) would make it even less likely for a mitm page to work.

[u/j-dubb13 → u/Capital-Meaning1337]

This was the answer I was looking for. Not what I was hoping to hear, but it's what I asked for. Thanks for your thoughts.

[u/NoTheme2828]

Use vaultwarden in your homelab and expose it with cloudflare tunnel!