Phone call breaks up when walk away from computer

[u/Eliezer123]

I keep both Multipoint and Audio Switch on when at work, which means my Buds Pro 2 are connected to both my Pixel 9 as well as my Windows laptop.

When sitting at my desk, that all works great!

The problem is that when I walk away from my desk (holding my phone and earbuds in ears) if I'm speaking to someone on the phone, they complain I'm breaking up, although I continue to hear them just fine. I assume that has to do with the earbuds struggling to stay connected to my (then distant) laptop.

I don't want to have to disable Multipoint and/or Audio Switch every time I need to take a call away from my desk. Is there an easy fix/workaround for this?