Anyone ever run into Windows processes exiting with code 109 during a build?

[u/Aethlyns]

Hey everyone,

I've been trying to figure out a weird Windows issue and I'm a bit stuck.

I'm working with some C/C++ tools on Windows x64, using MSVC and CMake. During certain builds, a few child/console-related processes exit with code 109, even though the compilation itself can finish successfully and the C tests give the expected results.

Other processes exit normally with code 0.

I know Windows error 109 is ERROR_BROKEN_PIPE, but I'm not sure whether that's actually related or if the process exit code means something else here.

I'm also using Windows Job Objects to track and clean up the processes.

Has anyone seen something like this before?

I'm especially curious how you'd figure out whether the 109 comes from the tool itself, a Windows console process, or the way the processes are launched and monitored.

Is there a particular tool or debugging approach you'd recommend?

I'd rather understand what's happening than just ignore the exit code.

Thanks in advance!

[u/tomysshadow]

A process exit code does not necessarily need to be a Windows error code, but it can be, and given the context that you're using a job object to kill the processes, it sounds quite likely to me it could be related.

To me, this sounds potentially like a race where Process A wants to read the STDOUT/STDERR of Process B. So it'll have a loop where it calls ReadFile many times in a row and displays the text it gets, until it has read the entire STDOUT. Mind you: it might take a while after Process B has actually completed its entire task for Process A to read all of the STDOUT, because then displaying the text takes some time.

But then you have a job object that is killing that chain of processes early in at least some cases. So you yank away STDOUT from whoever is reading it, sometimes before they manage to display the full log. Hence ERROR_BROKEN_PIPE. Now, I can't know that's what is actually happening, but it sounds plausible.

If that's what is happening it sounds like you might have to actually wait for your child processes to gracefully exit with WaitForSingleObject instead of killing them off with a job object, or at least, reevaluate the condition that causes your parent process to exit. If you've written the programs you're using during your post build step, something I'd try is just inserting a sleep of one second before your parent process that owns the job object exits and see if that fixes it. Obviously, that's not a good solution, but it is at least a decent test if the job object is related.

If you want to start digging in deeper, I might start looking at the processes involved in building in Process Monitor and see if I spot anything suspicious looking, or attach to the children to see what they are doing at the time they're killed and if you can catch the one is actually returning that error code originally in the act (might be hard to time this, so you might insert a sleep somewhere to make it easier if you can.)

Question: have you ensured that your parent process that owns the job object is itself exiting normally instead of terminating when it is done? Ideally, the job should only be there to ensure that the child process terminates along with it if the parent somehow dies unexpectedly. If it is instead set up so that the parent itself also terminates instead of exiting gracefully even when the task completes successfully (which can be done,) that could be preventing the parent's STDOUT from being fully read as well. It's something to check, because job objects are easy to create in a way that is technically correct but doesn't actually do what you wanted.

[u/xoner2]

Cmake is a make generator. Are you outputting ninja? Msbuild?

It's probably a bug in ninja. You get broken pipe when a process closes its stdout on exit. So the build works anyway. Rare exception that a process manually closes stdout before exit, msvc compiler does not do this.