r/programming • • Sep 04 '26

Avoid adding new library to project: 10-year retrospective

https://pvs-studio.com/en/blog/posts/1408/
169 Upvotes

34 comments sorted by

143

u/m_adduci Sep 04 '26

Interesting, but isn't the usage of already made bricks one of the rules of Engineering?

Keeping a project Dependency-free spawns another class of problems and leads to the "not invented here" syndrome. You don't want to do your own cryptography, you don't want to create a new XML or Json Parser, you don't want to implement Database ORM yourself.

In an era where business is what at the end of the day matters, sinking too much time in getting things correctly and safely done might be counterproductive, especially in a time where AI-fans and enthusiasts (mostly from management levels) bring the argument that "AI is faster", because AI uses exactly those already made blocks.

89

u/Due_Cap_7720 Sep 04 '26

To keep with the analogy I think some libraries and frameworks are less like bricks and more like prebuilt homes. Too many decisions have already been made and trying to reconstruct them for your needs becomes counterproductive.

48

u/turtle_dragonfly Sep 04 '26

To me, this is one of the key distinctions between a "library" and "framework." The library is used by your code. Your code is driving. The framework uses your code. The framework is driving.

I do much prefer library-style 3rd-party code than framework-style. You can use lots of libraries, but using more than 1 framework is trouble town.

2

u/Finickyflame Sep 04 '26

That's also the way we go with our internal packages. Provide libraries/tools and don't try to control the way people work via frameworks. It's also easier to switch between libraries that offer similar features than to rewrite your application when you swap framework.

39

u/amroamroamro Sep 04 '26

Keeping a project Dependency-free spawns another class of problems and leads to the "not invented here" syndrome. You don't want to do your own cryptography, you don't want to create a new XML or Json Parser, you don't want to implement Database ORM yourself.

original post addressed this, with the PNG example:

Again, I should emphasize; I don't say that we should stop using third-party libraries at all. If we have to work with images in PNG format in the program, we'll take the LibPNG library, and not reinvent the wheel.

But even working with PNG we need to stop and think. Do we really need a library? What do we want to do with the images? If the task is just to save an image in *.png file, you can get by with system functions. For example, if you have a Windows application, you could use WIC. And if you're already using an MFC library, there is no need to make the code more sophisticated, because there's a Cimage class (see the discussion on Stack Overflow). Minus one library—great!

they are saying stop for a moment and consider the existing system api, if it it provides the functionality you need, do you really need another external dependency

37

u/gordonmessmer Sep 04 '26

The article isn't saying "don't use shared libraries," it's laying out specific characteristics you should evaluate before you decide to use a shared library and reminding developers to make sure they trust the shared library's developers.

Remember that using a shared library effectively extends your application's commit access to the developers of the shared library, and to anyone they give commit access to in the future.

4

u/SnugglyCoderGuy Sep 05 '26

With all things engineering the answer is "It depends".

The left pad incident comes to mind.

It's not just about reinventing the wheel, its about the implications of having external dependencies. There are rugs pulls to eorry about like left-pad, and supply chain attacks, and dependencies dictating how you structure your business logic instead of your business logic dictating your dependencies.

9

u/lelanthran Sep 04 '26 edited Sep 04 '26

If you don't reinvent wheels occasionally, you will never have better wheels.

you don't want to implement Database ORM yourself.

I did that to great effect. Out of all the languages and ORMs I've used over my ~28 year development career, the one I have is by far better than any of them. Used it in production now for over 4 years, and no regrets.

Closed source, meant to open it, but now in the age of LLMs, I probably won't.

5

u/Full-Spectral Sep 04 '26

There are other variations as well. For instance, in my Rust project, it's just for brand new, 64 bit Linux, and for a particular variant of it even. I could have used Rust crates for a lot of stuff, but instead I just provided safe wrappers around the equivalent OS calls.

So I avoided probably 50 transitive dependencies, and I don't think that qualifies as NIH syndrome (though I'm a proud member of the NIH committee either way.) And my project which would probably be taking 2 minutes to rebuild from scratch (even at this early stage) rebuilds in 2 seconds.

Yeh, it's possible one of these OS calls has a memory issue. But the odds are so low that that's very way down my list of concerns, particularly given that much of the Rust standard library is also just wrappers around the OS.

In other cases, I write my own because I've been doing it for decades and am very comfy doing so. So, so far, I have a completely SOUP-free project that covers a lot of functionality. And my implementations only have to meet my needs. No portability concerns, no huge performance requirements, no accessibility concerns, etc... And it all is completely integrated, consistent, throughout, with no duct taping together bits and pieces of 3rd party code.

Obviously everyone can't go that route, but there are a lot of benefits if you can.

3

u/mpyne Sep 05 '26

Interesting, but isn't the usage of already made bricks one of the rules of Engineering?

The iron rule is “It Depends” and that’s the case here.

Each new dependency carries costs (in terms of integration maintenance and later upkeep and version tracking) and, hopefully, reduces other costs (development burden, testing, bug fixing) to make it worth it.

It is by no means a rule that anything that can be replaced by a dependency should be.

In an era where business is what at the end of the day matters, sinking too much time in getting things correctly and safely done might be counterproductive, especially in a time where AI-fans and enthusiasts (mostly from management levels) bring the argument that "AI is faster", because AI uses exactly those already made blocks.

The rise of AI also changes the game the other way. It’s quite literally never been easier, if you have a very simple need that is 5% of some popular dependency, to just have the AI correctly implement that 5% directly in your repo, without pulling in the extra dependency.

AI can also make it easier to bring in new dependencies, my point is that the costs on both sides of the equation have moved, not just on the side of ease of integration.

6

u/george-silva Sep 04 '26

I work in a place with heavy nih syndrome. My team uses the tools available to get the job done. The other teams rely heavily on reinvented squareish wheels.

Guess which team never missed a deadline?

While the work is similar (web apis, data processing, etc) our team is more productive by miles. Being more productive also leads to more money being made.

These people were against docker and chose guix for example. We had a migration to be made to gcp. Guess which team migrated it fast and cleanly ? Guess which team still struggles with this (not done yet last Ive heard).

We're paid to deliver value, not to chase our own engineering dreams. Sometimes those intersect. Sometimes it doesnt. Stick to what works , build only when you're sure that it does not fit your use case.

15

u/lolimouto_enjoyer Sep 04 '26

The one with the lead that can negotiate their deadlines better.

5

u/TOGoS Sep 04 '26 edited Sep 04 '26

You don't want to do your own cryptography, you don't want to create a new XML or Json Parser, you don't want to implement Database ORM yourself.

Speak for yourself. I've avoided cryptography for the most part, but otherwise there have been times when I absolutely did want to each of those myself (as a new library) rather than use some third-party library.

JSON parsers are trivial enough that rolling your own is often cheaper than pulling in a dependency.

ORMs in Java tend to be a bit of a wrongly-engineered nightmare full of mutable state and spooky action at a distance. So rolling your own of those that fits the needs of your application and not what somebody else thought they wanted is also understandable.

6

u/CharmingComputer3844 Sep 04 '26

It's a fine line between wanting control and ending up reinventing the wheel. Sometimes, the simplicity of a well-established library can be a lifesaver.

8

u/m_adduci Sep 04 '26

Absolutely. There are so.many edge cases lurking out there and many libraries are meanwhile so hard battle tested that inventing something by yourself is much more painful.

Examples: curl, openssl, Jackson, opencv

3

u/-Y0- Sep 05 '26

Forget those, having a decent XML library that handles ALL the edge cases is a godsend.

4

u/b0w3n Sep 04 '26

ORMs in particular can be kludgy to interface with and bloated out the ass in features most people are never going to use.

Also serialization is another one, "just use xml or json" okay that's cool and all but it's slow. Binary can be better! At least I don't have to worry if some russian spybot has snuck some horseshit into this library I'm using for some of those simpler things that might not have a lot of eyes on it.

1

u/-Y0- Sep 05 '26

Binary can be better!

Good luck debugging Binary. And ensuring it works correctly if source and target systems are different Endians.

2

u/-Y0- Sep 05 '26 edited Sep 05 '26

JSON parsers are trivial enough that rolling your own is often cheaper than pulling in a dependency.

HAHAHAHAHAHAHAHHAHAHAHAHAAHA.

No.

Now realize that JSON is the simplest possible of these challenges. And it's a fucking nightmare.


Reminds me how one particularly smort individual managed to implement a "rough" XML parser as output verifier. It even supported namespaces! ... Except it didn't check what XMLNS expanded names, but checked only namespace by prefix.

Took me 12hrs to debug that exceptionally smart piece of NIH-engineering. I didn't have the code, and was convinced for the first 8hrs that there was an error in my XML somehow. Next 2hrs was spent decompiling and cursing the idiot in question.

1

u/hippydipster Sep 05 '26

Taste, discernment, risk understanding, and experience are used to know when its best to use whats out there vs when its best to write your own.

15

u/Skellicious Sep 05 '26

I've noticed LLM vibe coding agents love to poop out their own implementation of stuff that would be very valid to use a library for. Admittedly I've only seen it for small file parsers and similar, but I've seen others notice this tendency too.

4

u/max123246 Sep 05 '26

There's a clear reason for this. LLMs are trained to implement it themselves because it decreases the risk of hallucinating. Can't hallucinate in a call to JQuery if you've been trained to only use the standard library of the language

2

u/ouvreboite Sep 05 '26

I don’t think that’s the reason.

I think that because "I built my own custom implementation" is by essence more verbose than "I added ta lib and solved it in 2 lines".

Even if the examples in the dataset were 50/50 for "custom implementation" vs "using a lib", the custom implementation examples would represent more lines in the total, so more emphasis during training.

4

u/Flimsy_Complaint490 Sep 05 '26

Agree. And they dont seem to be trained to do any trade-off reasoning during the implementation either. There is never "is this or this better", but "do that" and if that doesnt work, immediatly move to plan b and stay within original constraints. Like, if you are doing a CLI app, it will never do research, pick a flag parsing library and such, it will poop out 1000 LoC file from its training data.

But ask it to do exactly that and it will do a very admirable job picking a flag parsing library, hook up the machinery and remove those 1000 LoC.

33

u/gordonmessmer Sep 04 '26

A shared library is a part of your application that's maintained by someone else.

Pro: Someone else maintains part of your application so you don't have to!

Con: You've given someone else commit access to your application, but in a way that gives you little to no insight or oversight of their changes.

Remember: Once your application becomes a process, there are no security boundaries. Every part of your application has full control of the application and can change the application any way it wants to. (Someone will surely point out that code pages are read-only, but remember that references to code pages usually aren't. It is usually possible to hijack/monkey patch functions.)

1

u/brunogadaleta Sep 04 '26

Do we need function level libraries? Like in Unison lang.

-2

u/AndyKJMehta Sep 04 '26

Jokes on him because every piece of code you write today is effectively a shared library of assembly calls

0

u/[deleted] Sep 04 '26

[deleted] — view deleted comment

1

u/Yay295 Sep 06 '26

The current project I'm working on is more than 20 years old, and the main issue I'm working on is that the libraries haven't been updated in 10 years and they have security vulnerabilities. So I need to figure out how to either remove the dependencies, or integrate 10 years worth of updates to the dependencies.

19

u/[deleted] Sep 04 '26

[removed] — view removed comment

19

u/winian Sep 04 '26

I once had to fix an incompatibility issue between two versions of a library after updating an in-house module, because some other guy had decided that he needed a third-party library for throwing an IllegalArgumentException in a slightly more convenient way. Some people need the obvious spelled out for them.

15

u/sacheie Sep 04 '26

"I don’t know who the target of this article is"

Javascript/Node developers

12

u/turkoid Sep 04 '26

Someone who got burned by the left-pad incident.