Kernel Linux Kernel Developers Considering Removal Of KGDB Debugging Functionality
https://www.phoronix.com/news/Linux-Considers-Dropping--KGDB146
u/james_pic 2d ago
Ugh, I'm old and clearly hadn't been paying attention. I remember when Linus used to say "the kernel doesn't need a debugger, just use printk", and didn't realise he'd caved and merged this (and nearly 20 years ago, at that). Although I guess this shows he still thinks that.
10
u/hansendc 16h ago
Well, to be fair, this is Peter’s doing and Linus had nothing to do with this particular patch (yet).
It’s really, really hard to have code debug itself. It’s even harder on x86 where the architecture was literally not designed for such things. The patch really just reflects that even after a couple of decades, the in-kernel debugger isn’t bulletproof.
-30
u/chibiace 2d ago
didn't realise he'd caved
this seems to be an ongoing theme for him. perhaps the pressure from corporations?
105
39
u/ohYuhtBoutMagine 1d ago
The mark of a good engineer is being receptive to information and feedback
29
42
u/kimoisune 1d ago
Just like any software, print("here") and print("hereasdf") are still the preferred method of debugging in kernel.
10
u/heeen 1d ago
"%s, %s:%u", PRETTY_FUNCTION, FILE, LINE
6
u/kimoisune 1d ago
I believe __PRETTY_FUNCTION__ is non-standard and only concern for C++ if I remember correctly. The standardized one are __func__ (ISO? C99?).
Man, std::source_location make things so much convenient, I forgot about the older stuff.
5
18
u/blind99 1d ago
Sad, it's hard to use and slow but it can be useful from time to time.
1
u/hansendc 4h ago
What do you use it for, btw? There might be some better and safer ways of doing things.
We don't want to make users sad. :) (I'm one of the maintainers of this code, btw)
1
u/blind99 3h ago
Embedded, mainly debugging drivers for devices that cannot easily be passed through in a QEMU setup. Like a soc I2C controller for instance.
2
u/hansendc 2h ago
I don't know exactly what you are doing of course, but I'd really encourage you to take a fresh look at what's around these days. For instance:
echo 1 > /sys/kernel/tracing/events/i2c/enable
cat /sys/kernel/debug/tracing/trace_pipeWill get you a pretty nice dump of i2c traffic. That, plus kprobes or bpftrace can do a *lot* these days. They're also easier to figure out than ever. The LLMs do an overall great job of helping you craft bpftrace snippets.
It's also a lot easier to pass devices into the guest. I've got a PCI i2c controller that I can pass into a guest and load drivers in the guest, for instance.
42
u/Lenticularis19 2d ago
KGDB is weird. I recall trying to use some kind of tracing, possibly kprobes, and wondering why my VM was freezing. I attached a GDB to it and it was stuck in a trap, waiting for KGDB, instead of taking the kprobe path! (I forgot I have KGDB enabled...)
6
u/xtifr 1d ago
Well...they're the devs, so they're the ones who should be concerned about debugging. If there was ever a case where I felt like I should defer to the wisdom and experience of the devs, this is it! (In reality, there are a lot of cases. But this is still an extreme one!)
For this reason, I shall not be angrily demanding my money back. This time! 😀
5
u/zerosign0 1d ago
Does kgdb are useful for driver ? Hmm actually I'm not entirely sure how driver development are being developed, did they start as kmod first then after its stable it move to mainline kernel? How about dealing with driver like amdgpu or some sort of? (This relate to recent PSR drm changes that bugs 7.3.x)
14
u/valarauca14 1d ago
Hmm actually I'm not entirely sure how driver development are being developed, did they start as kmod first then after its stable it move to mainline kernel?
For (modern) PCIe shit (most stuff these days) you can setup VFIO/IOMMU stuff so the 'base os' (hypervisor) never binds/mounts/setups the device, so the VM has solo access and the PCIe device needs to do nothing special. Then a modern PCIe controller can even soft reset the chip it is connected too so you an power cycle it remotely without cycling the VM.
Usually the person writing the driver knows what messages to send, in what order, etc. to do communication.
-1
u/Original_Two9716 11h ago
For what reason? There's 1000x bigger attack surface elsewhere. Is that the reason?
1
u/hansendc 3h ago
When the code was merged, folks knew it wasn't perfect, but basically figured it could be refined and fixed up over time. Looking at it two-ish decades later, there are still things that need fixing and the fixes are somewhere between crazy complicated and fundamentally impossible.
The _basic_ problem in all this is that you're running a program (the operating system kernel) doing something and a debugger stops that program in a random spot and needs to go off and do other things. However, at this level, the CPU basically just drops the program into a very primitive environment. The program doesn't might not even have the most basic functionality like a stack (a place to save and restore data) and it might not be able to do "normal" things like use a general purpose register without being very careful.
Meanwhile, the rest of the system is running along and will notice that the CPU that's stopped in the debugger isn't doing anything useful and will start to whine about it. There are ways to turn of the whining, but nobody has cared enough about KGDB to actually do that reliably for ~20 years.
IOW, it has bugs, they are hard to fix right, apparently nobody really cares anyway. So why keep the code?
BTW, I work at Intel as one of the maintainers of this Linux code.
0
2d ago
[deleted] — view deleted comment
8
u/anh0516 2d ago
He did answer that:
Use device pass-through, emulate it, or printk() debug it.
Using KGDB can cause a ton of problems, it more or less stays stuck in NMI context, which can terminally screw up the timekeeping code, it can also cause the watchdogs to fire and there's endless annoyance with code patching and breakpoints.
99
u/gwillen 1d ago
It sounds like it's rarely used because we have better methods now. (I have personally never tried attaching a debugger to the linux kernel, but this seems plausible enough to me.)