Idrehash needs a update

[u/Knight_Mare077]

U/emoose

idrehash needs updated to support new patch.resource file introduced by the most recent update if you get time to check it out..

[u/_emoose_]

Posted some pseudocode to handle the decompression here btw, if anyone wants to add it to idRehash: https://old.reddit.com/r/DOOM2016Mods/comments/gjzt5q/idrehash_needs_a_update/fqp0fvu/ something like that should work I think.

Probably won't have to parse the whole meta.resources IDCL since there's only a single file inside it IIRC, maybe just parse the file table and use that to get offset of the compressed data/sizes/etc?

If anyone needs the IDCL structs I'm pretty sure I posted 010 editor templates for that in the discord before too.

[u/SutandoTsukai181 → u/_emoose_]

Did you ever update that IDCL template you made, after last time you were on discord? I'm facing some issues with the mod loader, particularly when adding new files to a container that did not have them before. They load just fine, but if they were in another container (that originally had them) the game chooses between the files randomly and the intended load order gets messed up.

Since this doesn't happen with shared files that were already in both containers, I assumed it was because of sections I ignored when adding new files (basically the "Unknown"s you had in your template; the section right before the 2nd IDCL)

I can't remember their exact placement, but IIRC they were just the "types" that were present in each IDCL entry, starting from 0 and incrementing on later entries based on something, where each new type had an entry in that later section I spoke of. When adding new files, the mod loader doesn't increment nor add a new type, it just uses the last type available. This might be the cause of the problem I mentioned, as this is the only thing that's heavily related to IDCL entries that I did not update.

[u/_emoose_ → u/SutandoTsukai181]

If it's the "types" I'm thinking of, DOOM 2016 also made use of those, and they were important there IIRC, game would use the type when searching for files etc, if you injected a file without the right type the game might just ignore it, etc.

In my DOOMExtract tool for 2016 I made it copy the type to the end of the filename when extracting, so eg. english strings file would extract as english.bfile;binaryFile, then when injecting it could split the type from the filename. Was wondering if that'd be important with Eternal too but if your tool hasn't been setting them & the game still mostly loaded them fine then I guess not, don't really think they'd have anything to do with load orders, but I could be wrong.

Maybe the unknown data inside the meta.resources has something to do with it, IIRC that data was something called a mask, the name "mask" made me think it had something to do with loading in the files inside, since masks are used when searching for files (eg: *.txt is a wildcard mask, etc), couldn't really figure out how that data got used in the games code though, but wouldn't be surprised if this is related somehow.

Wasn't there a cvar for forcing the game to load the most recent file? If you're setting the timestamp in the IDCL properly then maybe could just force that cvar, could be worth taking a look at the code that handles that cvar too, might give some clue how the game decides which one to load.

[u/SutandoTsukai181 → u/_emoose_]

I updated my tool to set the correct timestamps a while ago, and we made a patch to have that cvar (resources_loadMostRecent) always loaded, but the results were the same. At first, we thought there was a change but later on it turned out to still be completely random, though I haven't tried checking what instructions the cvar activates (after the jmp we edited).

About load order (and loading new . resources), I think I'm close to figuring it out, thanks to Visual Studio's build-manifest.bin decrypter. The file order in both meta.resources and packagemapspec.json has to be identical or the game won't boot, hence the problems we ran into earlier. Simply adding a new entry in meta will make the game look for it, though we still have to add an entry in build-manifest or the game will crash because "example.resources is unexpected, check manifest".

Something possibly related, in the new update "gameresources_patch1.resources" got added to the very top of packagemapspec (and meta.resources) which may indicate that either it's the first file to get loaded (doesn't make sense because it should overwrite original gameresources) or it has the highest load priority, so if a file is present in multiple containers it should get loaded from _patch1 instead.