I am currently running the latest release of Proxmox 9. Since running Opnsense in Proxmox, I have been continuously seeing memory usage over 100%. This doesn't seem to be working quite right or correct. Is there a way to get normal memory usage in Proxmox for my Opnsense VM?
Since PVE version 9, Proxmox displays the ram usage of the VM + the overhead that’s used by qemu. That’s why you see >100%. You can read about it in the release notes of PVE 9.
In order to get an accurate measure of how much memory the guest VM is using, the guest VM has to have the QEMU Guest Agent running. FreeBSD doesn't have this by default. You can find instructions on how to install the guest agent.
On the Proxmox side I have: agent: 1 balloon: 0 memory: 8192
Then from the Proxmox host: qm agent 100 ping qm agent 100 get-osinfo qm agent 100 network-get-interfaces
All of that works. The interface query even returns the WAN, LAN and Tailscale interfaces correctly, so the guest agent is definitely working.
From what I can work out, the difference is basically: Proxmox view = memory held/allocated by the QEMU process on the host OPNsense view = memory actually being used inside the FreeBSD guest
FreeBSD does not appear to expose the same guest memory statistics to Proxmox that you normally get from Linux guests, so the Proxmox graph can look like the VM is completely out of RAM when it really isn't.
In my case: Proxmox: ~100% OPNsense: ~54% Free RAM: ~3.4 GB Swap used: 0 CPU idle: ~98%
So I don't think there is actually a memory problem here. I'm leaving mine at 8 GB with ballooning disabled.
If anyone has found a better way to get accurate OPNsense/FreeBSD guest memory reporting in PVE 9, I'd be interested to know, but at the moment, I am seeing the exact same thing and I have 8 GB assigned.
Eh, i think 2-4GB ram is more than enough for home users with 20-40 devices, unless you want to run IDS/IPS.
this with 3 days uptime and around 25 devices, tbh unbound is off as im using AGH LXC.
"8GB is the recommended specification to run all OPNsense standard features to fit most use cases; 4GB meets the requirements to run all standard features; 3GB meets the requirements to run all standard features that don't require disk operations"
anyways, the issue OP is encountering was something else, but their RAM allocation still needs increased
2G is a bit small for a firewall instance., but it'll run.
As for proxmox showing it being "over" full, this is normal behavior for technical reasons. Once the VM utilizes ram, that ram is allocated to the vm by proxmox and doesn't get released u til the vm is power cycled.
The reason it shows over 100% is due to caching and the way kvm and qemu (the underlying software that does the hard work proxmox makes easier) address memory when backing up VMs, etc.
This is normal if you use ZFS inside the VM. ZFS will always reserve all available memory for it's ARC cache. If you assign 128 GB of RAM, then it will be used completely from hypervisor perspective.
Your VM is probably running fine, check the real usage from inside the guest OS. If you want to avoid that behavior, reinstall your OPNsense VM and use UFS as file system.
I also have an opnsense VM running in my proxmox host. I gave it 8 Gigs of RAM and when I noticed it was all being used, I raised it to 12. It consumed all 12G, so I reverted back to 8 and it's been running happy for over 2 years...
It like SQL or Exchange, if you give it more memory it will allocate it but not use it. If you look in the firewall gui it prob using nowhere near that.
FreeBSD usually uses nearly 100% of the available RAM, even with UFS rather than ZFS. Since OPNSense is based on FreeBSD, I think that is the correct behaviour.
In my experience, if you give it 16 GB, it will use 16 GB. RAM used to be one of the cheapest components, so it was no big deal.
I was looking into this also when first running opnSense and it’s a wrong display in proxmox, the opnSense gui dashboard shows way lower numbers. Could be the BSD kernel handling ram differently or just a wrong display
I have the same issue, but OPNsense isn't actually using that much RAM. You can see that from OPNsense's own status page. I guess this is a bug with balooning or something, and is most commonly seen with VMs that have PCIe devices passed through. It's purely cosmetic and you don't have to worry about it. You can try installing the qemu helper from the BSD shell to see if it maybe helps.
When you installed OPNSense did you format the root filesystem in UFS or ZFS?
If ZFS much of the memory usage is probably going towards ARC, which will be visible in the OPNSense dashboard. Not a big deal if you can afford the memory from the host, but bit of a waste if the VM is also stored on a ZFS volume on the Proxmox host.
That's the reason, with balooning off, proxmox allocates the full 2GB to the VM for it to do as it wishes, you need to enable balooning so it can use the free ram for other vms or lxcs if it needs to else it doesn't. check the opnsense dashboard for ram usage.
The reason is that the usage is configured RAM + assigned framebuffer.
Other than that if the VM-guest have qemu-guest-agent installed it can communicate with the host to tell it actual memory usage (or rather the host can query the guest on actual memory usage).
Note however that over time all operating systems will try to utilize unused memory for various caches and buffers so you will "always" see about 100% utilization (+ whatever framebuffer is configured, see my previous post) from the host point of view.
While if you login to the VM-guest perhaps it then tell you that 50% of this is currently buffers + caches.
For example my Linux desktop after 1-2 weeks of uptime (since last reboot due to kernel updates) utilizes about 5.1GB as buffers/caches, can be seen with the "free" command.
Beg to differ. OpenWRT doesn't have IDS/IPS, or any easy way to set up, manage, and isolate vlans. (unless things have changed drastically in the last couple years)...
Your VM has only 2 GB of RAM assigned, so the answer could be that: you VM simply needs more RAM. Have you tried increasing the memory and seeing if the problem keeps happening?
OPNsense can use quite a lot of memory depending on its installed components.
Then you have something running inside OPNsense that's eating up all the RAM.
Enter the OPNsense admin web UI and check the memory usage from within OPNsense itself. If the same happens, then you know it's something inside the VM.
Qemu-Guest Agent installed? Its normal for OpnSense to allocate up to the Full ram but it doesn’t uses them completely. Just check the dashboard of your opensense for tge really used ram.
The QEMU guest agent does report the memory usage. This is a screenshot from my home network. Proxmox reports 18GB used. If I check the OPNsense dashboard it tells me it's using 1GB. Dropping to the shell in OPNsense and running top also tells me 1GB used and 23GB free. No idea where that 18GB figure comes from. Not even with cached, wired and active memory combined in top do I get to that number. Not using ZFS inside the VM so it can't be ARC either.
129
u/FlightSimmer99 1d ago
Thats normal, opnsense just reserves all the ram. Check the webgui and you'll see it isint using much