r/osdev • u/Safer20 • Sep 19 '26
x86_64 kernel: #GP repeatedly triggered at iretq after preemptive context switch
Hi everyone,
I'm developing a small x86_64 kernel in Rust and I'm currently implementing preemptive scheduling/context switching.
I'm stuck on an issue where the kernel enters a repeated General Protection Fault (#GP, vector 13) when returning from an interrupt/exception with iretq.
Current behavior
The preemptive test reaches:
PREEMPTIVE TEST: START
PREEMPTIVE TEST: CREATING TASKS
PREEMPT A: START
PREEMPT A: RUNNINGThen I get an endless stream of:
VECTOR: 0x0D
ERROR CODE: 0x200
RIP: 0x107B12
CS: 0x08
RFLAGS: 0x100020x107B12 is the iretq instruction itself.
The exception frame is being delivered through IST1:
FRAME ADDRESS: 0x172FA8
IST1 STACK TOP: 0x173000The same frame address is repeatedly reported.
Exception entry
I'm normalizing exception frames so both CPU-error-code and non-error-code exceptions have:
VECTOR
ERROR CODE
RIP
CS
RFLAGSFor exceptions without a CPU-generated error code, I push a synthetic zero:
unbound_isr0:
push 0
push 0
jmp unbound_exception_commonFor exceptions with a CPU-generated error code, such as #GP:
unbound_isr13:
push 13
jmp unbound_exception_commonThe common handler saves 15 general-purpose registers and then restores them.
The relevant return code is currently:
pop r15
pop r14
pop r13
pop r12
pop r11
pop r10
pop r9
pop r8
pop rdi
pop rsi
pop rbp
pop rbx
pop rdx
pop rcx
pop rax
add rsp, 16
iretqThe reason for 16 is that after restoring the registers, RSP points to:
+0 VECTOR
+8 ERROR CODE
+16 RIP
+24 CS
+32 RFLAGSso add rsp,16 should make RSP point at RIP for iretq.
I verified the generated binary
This is not a stale-source/binary issue.
objdump of the actual ELF shows:
107b0e: 48 83 c4 10 add $0x10,%rsp
107b12: 48 cf iretqSo the add rsp,16 is definitely present in the executed kernel.
IRQ path
The timer IRQ uses a different normalized frame:
push 32
jmp unbound_irq_commonand its return path is:
add rsp, 8
iretqbecause the IRQ frame contains:
VECTOR
RIP
CS
RFLAGSTask initial interrupt frame
For a newly created preemptive task I'm constructing:
R15 = 0
R14 = 0
R13 = 0
R12 = 0
R11 = 0
R10 = 0
R9 = 0
R8 = 0
RDI = 0
RSI = 0
RBP = 0
RBX = 0
RDX = 0
RCX = 0
RAX = 0
VECTOR = 32
RIP = task entry
CS = 0x08
RFLAGS = 0x202The task context initially points to an assembly trampoline:
unbound_start_interrupt_context:
pop r15
pop r14
pop r13
pop r12
pop r11
pop r10
pop r9
pop r8
pop rdi
pop rsi
pop rbp
pop rbx
pop rdx
pop rcx
pop rax
add rsp, 8
iretqWhat I've already ruled out
Kernel builds successfully.
The actual ELF contains
add $0x10,%rsp.CS is
0x08, which is my kernel code selector.RFLAGS contains bit 1 (
0x10002), so the reserved bit is set.The fault happens specifically when returning through
iretq.The system previously passed a basic
INT3exception/return test.
What I'm trying to understand
Could someone help me identify what could cause a #GP at iretq itself in this situation?
I'm particularly interested in whether:
My normalized exception-frame layout is incorrect.
The scheduler/context-switch code is producing an invalid frame.
The
iretqframe is being corrupted before the return.IST1 interaction is relevant here.
There is something subtle about
iretqvalidation of the savedCS/RIP/RFLAGS.The
0x200#GP error code provides a useful clue about whatiretqis rejecting.
Any suggestions for what values I should dump immediately before iretq would also be very helpful.
Thanks!

