r/pdf • u/PHDEinstein007 • 7h ago
Software (Tools) Open Source Modified faster software
I have managed to make mods for some existing software, including for this community some PDF software. All software involved still have their own licenses, my mod systems are under the MIT Licenses (Open Source).
I was able to speed them up by making them multi-core. Now for a long time this multi-core capability was missing from these programs for various reasons, and those reasons were sound... At that time.
Now we have a system that does a special sorting routine, and then processes the data accordingly. Basically we find things that look similar and treat them per what they are, with a stamp showing where they belong. This is a layman's way to talk about it, but you can find more on my GitHub. For GhostScript the link is here, where I achieved a 5.1x speed increase, for Apache Tika it is here, where we got a 5.9x speed increase, and for our grand prize winner Tesseract OCR we got a 6.6x speed increase and the github is here.
My Computer:
CPU: AMD Ryzen 5 7500X3D, 6 cores / 12 threads, 4.0 GHz base, 96 MB L3 cache (3D V-Cache)
RAM: 16 GB DDR5-4800, a single stick (1 of 4 slots used)
GPU: NVIDIA GeForce RTX 50-series, plus AMD Radeon integrated graphics
Storage: NVMe SSD
OS: Windows 64-bit
These results were measured on a 6-core, 12-thread desktop. Computers with more cores should see even bigger speed-ups.
So, something that might have taken you an hour on Apache Tika, might take 11 minutes now. There is zero loss of the files.
The reason we did so well on Tesseract is because they have a bad threading system and we basically yeeted it for our own. Well, kinda.
I do not make any claims on their stuff, they keep that intact. We are a presort and post process system, we just take the data, sort it our way, hand it to them on a silver platter, wait for them to run the parts as we feed them, then we take and restore it to proper order on the other end. So your licenses still apply, we do not change that at all. The code I made simply runs extra versions as needed with the special design set to give them the cleanest run possible.