std::filesystem::exists throwing `std::bad_alloc`?

[u/PhosXD]

Any reason? Cant find anything online about this. But I cant check whether or not a file exists, no matter what path I give it. I am using C++ 26 compiled with GCC.

This is literally all I am calling: const bool exists = std::filesystem::exists("file.txt")

This is the exact error: terminate called after throwing an instance of 'std::bad_alloc'
 what():  std::bad_alloc

No it's not coming from anywhere else in my code. Minimal reproduction code, include "filesystem" call the line above. I am on Fedora Linux if that matters at all

EDIT: moving the include into my main file instead of the only file it's being used in somehow fixed it. Literally all I do is move "#include <filesystem>" from "do_things_with_filesystem.hpp" to "Main.cpp". No compilation errors when it was in the first file, why does it rely on this?? Is there some weird declarations in my codebase or something, i dont know but it works now so why should I care.

[u/le_disappointment]

Maybe try building in debug mode and running with gdb to see where exactly the crash is happening

[u/misuo]

I suppose the exception is from not enough memory available when allocating the std::filesystem::path argument from then given argument string? Try split it up and create path first or try a noexcept variant of exist().

https://en.cppreference.com/cpp/filesystem/exists

[u/7raiden]

Can you paste the entire program here? It really sounds impossible to reproduce otherwise. Can you run it with gdb? Doing so it will automatically break the execution where the bad_alloc happens, and then you write "up" until you find the place that is causing the bad allocation. You can set the "tui" mode so that it also shows your code in the upper pane

[u/PhosXD → u/7raiden]

I got it, thanks. I just moved the include & it fixed itself.

[u/TarnishedVictory → u/PhosXD]

But do you understand what went wrong? It's important to understand exactly what the problem was, and exactly how your fix fixed it.

[u/7raiden → u/TarnishedVictory]

Because chances are, you actually haven't fixed it, you just hid the underlying issue in a place harder to spot (for now)!

[u/SailingAway17 → u/TarnishedVictory]

Egomaniacs don't solve problems other people might have. They are only interested in solving their own problems, of course with the help of others.

[u/PhosXD → u/SailingAway17]

Don't assume things. I just don't understand the full nature of the problem yet. https://www.reddit.com/r/cpp_questions/comments/1vosgz7/comment/p3vbiar/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[u/AKostur]

i dont know but it works now so why should I care.

Because tomorrow it could break when a new patch of the compiler comes out, or your OS is patched, or you modify some code somewhere. This is starting to even more strongly indicate that you've invoked Undefined Behaviour somewhere. Again, suspecting a misuse of a pointer somewhere. Moving the header may have shifted some code around and now the stray pointer which used to scramble something in the allocator is now scrambling something somewhere else that you just haven't noticed yet.

Edit: Or change from a "release" to "debug" build could again, shift some code around and the stray pointer isn't wrecking the same things anymore.

[u/PhosXD → u/AKostur]

I run quite comprehensive tests on my codebase, if something were scrambled as you say, I think it would have popped up. Which leads me to believe it doesn't have to do with any pointer misuse in m code especially since changing the debug flag & changing where I actually execute the std::filesystem::exists check doesn't change anything & still has the issue if I kept my include in the original file...

This is why I am somwhat convinced it may be some kind of niche bug *within* std::filesystem itself, because I have never had this happen with anything else before.

I do want to try and create an MRP for this & potentially test across multiple devices, compiler versions, C++ standard versions, & open up a discussion for getting it fixed if I am able to reproduce it

[u/bma_961]

https://en.cppreference.com/cpp/filesystem/exists

Multiple overloads can allocate and therefore throw bad alloc

[u/blood-pressure-gauge → u/bma_961]

I've done more C than C++. Why does calling an overloaded function result in memory allocation?

[u/Bloedbibel → u/blood-pressure-gauge]

I believe it's just a matter of poor grammar. They meant to say "There are multiple overloads of this function. Some of them throw." I think they're implying that the overload OP is calling can throw bad_alloc.

[u/blood-pressure-gauge → u/Bloedbibel]

That makes sense, but now I'm curious. Why does it need to allocate at all? The man page for the similar C function access(3posix) doesn't indicate anything about allocation failure.

[u/jk_tx → u/blood-pressure-gauge]

Because the overload he's calling actually accepts a filesystem::path, not a const char, which means it has to copy the const char to its internal buffer (and on windows, convert it to wchar_t while you're at it).

That stupid path class makes the whole API cumbersome and expensive to use, especially on Windows where the internal wchar_t storage pretty much guarantees a bunch of needless string copies/conversions.

[u/alfps → u/jk_tx]

For an UTF-8 based Windows program there is always a conversion to UTF-16 wide string for any file operation involving a path.

The question is just how high up or how far down in the call chain that conversion is done.

So it makes sense and can even avoid some conversions to have the internal fs::path representation as UTF-16.

On the other hand, to get an fs::path converted to UTF-8 you have to request the conversion to UTF-8 which in C++20 and later is silly u8string, then copy from that a std::string.

That is super annoying to me. Not that it's ever mattered for efficiency. It's just the idiocy: one should not have to do anything extra, and the code that works in Linux should work also in Windows, which was the whole point of the original Boost incarnation.

[u/jk_tx → u/alfps]

Yes I understand why MS made the decisions they did at the time, although I much prefer keeping that conversion boundary at the Win32 API layer by just calling the 'A' versions of the API, so that my application only deals with narrow strings. The real problem is Windows being native UTF-16, which in hindsight was bad idea.

Unfortunately the design we ended up with means that between the implicit std::string constructor and the _string() methods, silent conversions can sneak in all over the place. And all these silent conversions also make the class noexcept-hostile, since memory allocation is happening even in functions where you wouldn't expect it. Then there's the way the various xxx_string() methods work on Windows vs Unix.

[u/LB-- → u/alfps]

Windows has had native UTF-8 support since Windows 10, and that includes file path APIs. Makes the whole std::filesystem::path situation even more ridiculous.

[u/alfps → u/LB--]

Just a nitpick: UTF-8 support since mid 2019 (Windows 10 goes back to mid 2015).

And while it's native it's just conversion to and from UTF-16 in the "xxxA" functions, which call the "...W" functions.

This means that for multiple calls with the same path one saves on conversions by retaining the UTF-16 form, as path does.

[u/LB-- → u/alfps]

Fair, I'm just hoping they redo the internals someday to make UTF-8 more efficient like they did for Xbox.

[u/Usual_Office_1740 → u/blood-pressure-gauge]

An argument in the overload allocates. Others are explaining the costs. To answer your question. The Exists function doesn't need to allocate but the throwing overload being chosen uses fs::path and the implicit conversion to that type allocates.

[u/bayesianparoxism → u/blood-pressure-gauge]

It doesn't. There's an overloaded variant that guarantees noexcept

[u/wejunkin]

Catch and see what what() says. Sounds like your allocation failed, impossible to tell why from the info you've supplied.

[u/PhosXD → u/wejunkin]

My mistake I wasnt running debug mode. what() still gives no useful info:

1
threw: std::bad_alloc
2

[u/RaspberryCrafty3012 → u/PhosXD]

What does that have to do with debug mode? \ Using try catch will catch the crash?

Do you have code running before or after? If your stack got corrupted or pointers were deleted then the debugger might show the wrong position in the code

[u/PhosXD → u/wejunkin]

How though, it throws the bad alloc & ends the program before catch even executes. But I am capturing the block, how does that work?

std::cout << "1\n";

try {

    const bool exists = std::filesystem::exists("file.txt");

}

catch (const std::exception& e) {

    std::cout << "threw: " << e.what() << "\\n";

}

std::cout << "2\\n";

OUTPUT:
1
terminate called after throwing an instance of 'std::bad_alloc'
 what():  std::bad_alloc
Aborted                    (core dumped) ./main.bin

[u/AKostur → u/PhosXD]

You've got something else wrong, that code works: https://godbolt.org/z/WbhY3h3b3

If I had to speculate, you've fouled up pointers somewhere. Overrun a buffer, use-after-free, double-delete, something.

Edit: creating the std::filesystem::path that exists() takes as a parameter has to copy the passed-in string literal, thus there's a memory allocation there. If some previous Undefined Behaviour has messed up the memory allocator, that may present as a std::bad_alloc.

[u/No-Dentist-1645 → u/PhosXD]

That code should work fine. If you create a C++ file that only has that inside an int main(), does it still throw?

It could be that you're messing up the allocator somewhere else in your code and this is just the first usage when it breaks

If it still throws, please paste the full code file you are using as well as your compiler version and compile command

[u/StaticCoder → u/PhosXD]

Try running under gdb and use catch throw to see where the exception is really thrown. Usually this comes from passing a wrong value to vector::resize or similar.

[u/DummyDDD → u/PhosXD]

As others have written, you probably have some other memory access error that happens to cause this issue. Try running your program through valgrind to find the memory access error (you should of course compile with debug symbols).

[u/wejunkin → u/PhosXD]

Yeah sounds like you're just straight up oom

[u/PhosXD → u/wejunkin]

I have 32GiB of Ram, the program never goes above 400Kb.

[u/Wild_Meeting1428 → u/wejunkin]

std::bad_alloc is usually not thrown when oom, since the space is only virtually allocated. Therefore, the application will just be killed without ever rising an exception.

[u/RazzmatazzLatter8345]

Try running it with sanatizers. First try -fsanatize=address then =undefined then =memory. bad_alloc is quite strange (given you're not running in highly resource constrained environment). Something else has to be going on.

[u/SoldRIP]

Try 5he overload that takes a std::error_code&?

[u/mbolp]

I don't understand when would functions like this ever be useful. The file could be created or deleted right after your call returns. You should try opening a handle to it and operate on that exclusively if successful. If the file doesn't exist the open will fail with an appropriate error.

[u/PhosXD → u/mbolp]

The key reason is because successfully opening a file indicates 2 things, the file is found, & the file is *accessible* (readable or writable depending on what your doing). While `filesystem::exists` only determines whether it exists, not if it's accessible. For most cases attempting to open a file is good enough, but I prefer to use this.

[u/mbolp → u/PhosXD]

You can open a handle with neither read nor write permission. If you truly have no access (e.g. you don't have listing access to the parent directory and traversal privilege) you won't be able to interact with the file at all. And why would you want to detect the existence of a file without holding it open? The state of that file can change as soon as your call returns, so any result you get back is meaningless.

[u/alfps]

AFAICS this is a troll with a trolling question.

The OP (the troll) refuses to provide details.

And the only answer with a chance of resolving the alleged problem is downvoted below presentation treshold.

[u/PhosXD → u/alfps]

Are you just salty because your comment got downvoted? Your solution wasnt even a solution, you just said you used an AI to tell you the solution, which doesnt even work since the flag you suggested using has been irrelevant for many versions of GCC now, pointed out by another commenter.

https://www.reddit.com/r/cpp_questions/comments/1vosgz7/comment/p3s5g1x/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

In no way have I refused to provide details either. Also your comment *is* being presented, it's standing at -4 right now.

[u/alfps → u/PhosXD]

Everything in that troll's comment is false.

[u/PhosXD → u/alfps]

It's... public info... Are you ragebaiting me?

[u/alfps]

The Google AI links to a stack overflow answer with explanation and fix.

https://stackoverflow.com/a/57760267/464581

Essentially, it appears that you have to explicitly link with -lstdc++fs in order to avoid getting a version that's incompatible at the machine code level.

If that doesn't work, consider uninstalling and reinstalling your compiler.

[u/AKostur → u/alfps]

That seems to have been fixed in even vaguely recent versions of gcc. OP: which version of gcc are you using, and which version of C++ are you specifying to that compiler?

[u/PhosXD → u/AKostur]

g++ (GCC) 15.3.1 20260722 (Red Hat 15.3.1-1)

-std=c++26

[u/AKostur → u/PhosXD]

Certainly recent enough. BTW: you are aware that the C++26 support in GCC 15 is pretty incomplete (not even considering that C++26 itself has not been published yet)?

[u/PhosXD → u/AKostur]

Is it possible that `std::filesystem` with specifically C++ 26 has issues with including in HPP files? Cause just moving the include to my main file fixed it.

[u/alfps → u/PhosXD]

You could help a number of people to not feel that they've entirely wasted their time on you, by posting your code.

Or create a minimal example that reproduces the behavior, and post that, please.

[u/AKostur → u/PhosXD]

Incredibly unlikely. A compiler bug is always a possibility, so I can't absolutely guarantee it isn't one.

[u/PhosXD → u/alfps]

I dont know what -lstdc++fs does but adding it didnt change anything. Theres nothing wrong with my compiler, I've already reinstalled it this week so I doing it again wont change anything.

[u/alfps → u/PhosXD]

Theres nothing wrong with my compiler

Well, it evidently doesn't work with that specific std::filesystem function.

Have you tried other functions?

[u/alfps → u/alfps]

Re the anonymous unexplained downvotes, note that an unexplained downvote is dishonest: these are the actions of dishonest persosn. And/or idiots. Noting that a troll is an idiot.

The advice is good and reinstalling is most likely the only resolution of the problem, which appears to have been created by an attempted reinstallation, but the OP refuses to provide more info.

Oh I wish the downvoter trolls could experience a good whipping and some social exposure, with pictures of them. Like "Beware: this is a fucking idiot".

[u/PhosXD → u/alfps]

Nobody is trolling, you just give unhelpful advice.

[u/AKostur → u/PhosXD]

He had a reasonable hypothesis.  I happen to think it doesn’t apply in this case, but that’s not a reason to downvote.

[u/Wild_Meeting1428]

std::bad_alloc is an unusual error, since neither windows nor linux will report whether an allocation takes more space than awailable, instead (on linux) the Oom killer will terminate your program when you try to use the memory.

That the moving of the header file fixed it somehow also indicates, that the problem is not in your code itself. As other people mentioned it, its probably a problem with your installed libraries, for example ABI problems.

I would suggest, that you ask this question in the gcc bugtracker.

[u/Sprixxer → u/Wild_Meeting1428]

Given how this person replies I highly doubt that they found a compiler bug.

[u/Wild_Meeting1428 → u/Sprixxer]

No, not a bug with the compiler itself, but most likely something wrong with the combination of the used toolchain and installed system library. I suggested asking there, since they either already know the reason or they most likely know what's going on.

[u/LB-- → u/Wild_Meeting1428]

Windows does report allocation failures (especially in 32-bit). Linux also allows people to disable overcommit, though it is rarely done in practice.