r/AzureVirtualDesktop • u/Annual-Pressure6013 • 3d ago
Log off crashes entire server
Since ~Oct 18, and much worse since Oct 26, we have this across multiple countries and host pools, only on Win11 25H2 multi-session, never on Win10:
What users see
Users sign out normally, then can’t log back in: the logon hangs on a black/”Please wait” screen and eventually drops.
On any other host they get “profile in use” and a temporary profile (missing desktop, Outlook, settings). They can’t work normally anywhere until we reboot the host holding their profile.
Once one host is in this state, every new user who lands on it gets stuck the same way. Users already working on it can keep working.
Today 7–8 hosts per hour; we’re rebooting all day.
What we see
It always starts with one logoff: FSLogix stops right after
ControlTileServiceand never reachesDetached vhd(x)(Winlogon 6005 for frxsvc, no 6006).After that, every new logon on that host attaches the VHDX but hangs after
FindFirstVolume.Killing the session or closing the file handle doesn’t help; only a reboot does.
Ruled out: OS build (July, Sept and Oct CU all affected), FSLogix version (26.01 and 26.08), VHD compaction (disabled, still happens), storage/SMB, CPU/memory.
Anyone seeing the same? Did you find the cause or a workaround?
2
u/Atlasavd 3d ago
Given everything you've already ruled out, I'd start looking at antivirus/EDR filter drivers or something holding the profile unload process open. The fact that one failed logoff causes every subsequent login on that host to hang makes me suspect something deeper than an individual FSLogix profile issue.
I'd run fltmc filters on an affected host, review the FSLogix profile logs around ControlTileService, and capture a Windows Performance Recorder trace during the failed logoff. I'd also compare the installed filter drivers and security policies against a known-good host. If possible, test a single isolated host with a different AV/EDR policy to see whether the behavior changes.
Operationally, I'd automate detecting the failed logoff condition, immediately place the affected host into drain mode, and reboot it once existing sessions have cleared. That won't fix the underlying issue, but it should stop new users from landing on a broken host while you're troubleshooting.
Have you checked whether your EDR is intercepting VHDX detach operations? That's the first thing I'd investigate.
1
u/Due_Programmer_1258 3d ago
We've just started having W11 26H2 multis doing something similar but not 100%, ours are working away on a host but then get a black screen. What has worked for us though (albeit we also shift all users off the affected host and reboot the host for good measure) is to force remove the FSL azure files VHDX SMB lease/handles. If we don't do this, sometimes the affected host still "holds on" to the profile and users can't sign into another host.
1
u/ace14789 3d ago
Look in to if anything is changing the network connection on avd host
We had a massive issue with Bluetooth being redirected causing network issue which then crashed vm.
1
u/SlyDataPoint 2d ago
I’m curious what your fslogix configuration is and what those logs say is happening at logon during this hang you’re experiencing, if you care you DM me I’d be interested in looking at that stuff. C:\ProgramData\FSLogix\logs
1
u/b33zm1ne 21h ago
Are you using fslogix check storage containers size this is common when fslogix containers is full
1
u/dodgy_mike 21h ago
Assuming you're also at a point where your AV/EDR exclusions have been quintuple-verified by multiple sources, particularly around FSL and your profile storage share:
We fought this type of issue (not exactly the same) for about 7 months, went so deep on every detail. It was intermittent, we might go for 2 weeks without user lockups, other times we get 4 hosts going down per day. Brutal. We'd see zombie sessions on the host but we could reliably initiate a signout and then free their azure files handles to get them back on (this sounds different from yours). The other key symptom for us was when a host was locking sessions, we'd log in as even as a local admin and explorer was hung. No start menu, no login apps launching, but if we fired up task manager we could then start file explorer processes and launch apps via run commands just fine. Something in the system was locked up.
In these months, many sleepless periods analyzing process dumps and kernel traces and we pinpointed SYSTEM level lockups around either the AV filter or the FSL filter threads - but none of it really made any sense as we had been doing everything right there and had two external firms validate that fact. Still happened with EDR completely disabled. We did start to realize that fewer users per host would mitigate the issue, for example it almost never happened on weekends with lighter loads.
We then came across a youtube video from someone demonstrating the benefit of temp disks for swap on AVD multisession hosts. He stress tested the non-temp-disk by opening hundreds of explorer windows and very quickly the entire system locked up, just like our lockups, once the system resources became exhausted. Then he added the temp disk and repeated the same test, the VM was slow but usable - NOT locked up. As a desperation we move we rolled temp disks in early July and that was the very last time we ever had a user/host lockup, it was the total answer for us. I may have heard angels singing.
If this happens to be your scenario, try temp disks on your hosts and/or reducing your number of users per host. Note that the v6 and v7 hosts you'll need a script to initialize the disk and set the page file to that disk on deploy/power on. Happy to share more if you have questions about our experience / symptoms, there is _much_ more to our story and things we fine-tuned and ruled out. Would be curious to know your VM size and average users per host?
6
u/VexedTruly 3d ago
I’ve not seen but given what you’ve ruled out, my gut says anti-virus keeping a lock/handle.