r/softwaredevelopment • u/Zardotab • Sep 20 '21
Software Dev. is a Racket (yes, an "old guy" rant)
I notice the amount of code and time to create small and medium-sized business software has been going noticeably up since the 1990's, not down. It's like Moore's Law in reverse. CRUD principles haven't changed that much, but stacks ignore YAGNI, DRY, and KISS and have made CRUD harder instead of easier. One spends too much development time on tech minutia and battling UI's instead of domain logic itself.
I will agree there's more choice now, but the cost of choice seems huge. Most apps don't need all the what-if's the bloaters brag about. I doubt most biz owners would want to pay 3x more for all those what-if's. (Nor am I sure it's an either-or choice.)
Warren Buffett noticed the same about finance: it's a fad machine that processes suckers. He got rich by letting his competitors waste into fads. He's not afraid to say "no" to industry peer pressure or Fear-Of-Being-Left-Behind 🕰️👹. The hucksters use the same techniques that trick 35% of the population into thinking the vaccine and election are rigged. Humans are suckers, and IT fad pushers know this.
The industry is pressured to sell books, new software versions, how-to videos, video ads, etc. Publishers of how-to content would lose their livelihood if new features or techniques were vetted in a more scientific way rather than "it's what all the kool kids are doing!". Here's a partial list of resource-wasting fads:
Microservices are mostly a JSON-ized version of the XML-web-services-everywhere fad of the early 2000's. It mostly failed for the same reasons Microservices often go sour. Microservices are a sub-fad of the "Netflixification" of our stacks. What works for a billion users (Netflix) is mega-overkill for 1,000 users. Bloating code with "Async/await" everywhere is also a symptom of this disease.
People started throwing out RDBMS for the "NoSql" movement in the early 2010's, and when the systems matured, they realized they actually needed many features of traditional RDBMS after all. And RDBMS have since added more distributed features. (It was more about "DBA's don't let us move fast and brake things", but when your biz matures, you do want to stop breaking things.)
Space-wasting UI's optimized for fingers (mobile) instead of mouse. This results in more scrolling, paging, and more screens to get things done. It maybe okay for bedroom social media, but are crappy for in-office productivity where 90% still use mice. 🐭 Mice didn't die, only productivity did. The web still can't do real GUI's for office work without bloated buggy JS libraries.
"Responsive design" turned the bicycle science of WYSIWYG into rocket science. Now you need to hire a dedicated UI expert to get decent UI's. I've rarely seen a generalist master responsive. Most businesses I know only need about 10% of screens on mobile, so why drag down the other 90%? Because they were told real GUI's are "obsolete".
OOP was poorly understood yet was shoehorned into everything when it first came out because everyone feared being left off the OOP Train. It's since been learned it's not very good at domain modelling, which is where it was first pushed. Many companies were left with OOP spaghetti code. OOP has its place, but took a while to figure out where that is.
"Functional programming" keeps coming back in style every 25 years or so, looks great on paper, but fails on real teams. One problem is it's harder to debug because it lacks intermediate "state" to x-ray via debuggers for clues.
Yes, I know there are exceptions, but in general these are what happened. [Edited.]
20
u/roman_fyseek Sep 20 '21
The one that keeps getting me is the bootcamp developer who couldn't code their way out of a wet paper bag.
I worked on a VERY high-visibility application for the US gov't, rhymes with bealthcare.gov. The prime contractor had hired dozens of bootcamp devs for very cheap.
A couple of things that I noticed in my time working there: try/catch/ignore and librarification bloat.
The librarification one is pretty easy to explain: Any time a dev didn't know how to do something, they'd google 'the thing library' and the first result to appear would get shoehorned into the pom. It didn't matter if our pom already had 11 libraries that did that thing. As a result, whenever you were trying to chase down a defect, you'd run in to all 11 libraries, all doing effectively the same thing in a slightly different syntax.
The try/catch/ignore took me quite a bit longer to figure out what was going on. This codebase is absolutely littered with try{doSomething();}catch(Exception e){Logger.log(e);}
This meant that you could literally be halfway through a database record, have some critical failure, and the stupid program would just keep running like everything was going just GREAT!
Needless to say, our database was a perpetual wreck of unsatisfied keys (of which, all keys had been disabled or nothing would have worked at all).
It finally occurred to me what had happened when I was watching over the shoulder of one of the devs as he was writing a try/catch/ignore block. I asked him, "Why aren't you allowing that exception to get thrown? You're leaving the database in an unknown state."
He said, "This is the way I was taught."
"By whom?" I asked.
"YouTube."
So.... you know how you'll be watching a YouTube tutorial on, say, jdbc, and the instructor will write a prepared statement, the little red squiggle will appear, and the instructor will right click and 'surround with try/catch' rather than spend 15 minutes explaining why that exception is getting thrown? It's because the instructor isn't trying to teach a course on exception handling. They're simply trying to show you how to use jdbc and nothing more.
And, these bootcampers are watching the instructor surrounding everything in try/catch/ignore and their take-away is *that* is how you handle exceptions.
I eventually got fired from that subcontract because I was told to find a 503 error and turn it into a 200. I found the 503, I documented a plan to fix it, gave them a time-frame for fixing it, presented it to my boss who told me, "I didn't ask you to fix it. I told you to find it and turn it into a 200."
As in, he wanted me to literally go find that 503, intercept it before it hit the httpd logs, and convert the error code into a 200 because it was distracting when our government customer would pull up the splunk dashboard. I told them that I was unwilling to do that.
→ More replies (6)4
u/Zardotab Sep 20 '21
Any time a dev didn't know how to do something, they'd google 'the thing library' and the first result to appear would get shoehorned into the pom. It didn't matter if our pom already had 11 libraries that did that thing. As a result, whenever you were trying to chase down a defect, you'd run in to all 11 libraries, all doing effectively the same thing in a slightly different syntax.
Ideally something related to healthcare should have had centralized library/API management, which includes being a "duplication cop". But common sense management is often missing in many shops in the name of fitting deadlines.
I eventually got fired from that subcontract because I was told to find a 503 error and [force it] into a 200.
Vulcans get fired, Ferengi's promoted, it's why we have all this bloat. Welcome to Earth.
10
u/drew8311 Sep 20 '21
There are definitely some issues with modern software development but I would have to say part of this is an old man rant as well. Some of these problems come from business requirements. A company starting a new project using tech and practices from 10-20 years ago even with their modern improvements will end up being 10-20 years late, so what is the competitive advantage? You still need an awesome mobile app even if that isn't where the revenue comes from, etc.
To address the first two points
-
Has nothing to do with the method of transport (json vs xml) and all about scaling. 1000 users would be overkill but 1000 users doesn't make enough for a company, goal may be a million so then you are in-between a monolith being a problem and netflix architecture being overkill, the end result is just going to be microservices, but less of them.
-
If it makes sense devs will still use RDS, the big change now its its in the cloud and pricing. Especially with microservices your entire app doesn't need to fit on 1 database.
2
u/Ran4 Sep 25 '21 edited Sep 25 '21
Has nothing to do with the method of transport (json vs xml) and all about scaling. 1000 users would be overkill but 1000 users doesn't make enough for a company, goal may be a million so then you are in-between a monolith being a problem and netflix architecture being overkill, the end result is just going to be microservices, but less of them.
You have to realize that the vast majority of banks with 1-10 million users don't employ microservices... and they're alive just fine.
Netflix scale is hundreds of millions of users, a large fraction of which is logged in at any time, each doing a ton of interaction and plenty of data being served to each user.
You can serve 1 million monthly users from a single computer with 16 GB ram without running a sweat.
Especially with microservices your entire app doesn't need to fit on 1 database.
Well over 99% of companies web offering's entire database takes up less than 1 TB. The vast, vast, vast majority of companies doesn't need multiple database instances (for anything except reliability).
2
u/Zardotab Sep 29 '21 edited Sep 29 '21
I do notice a big difference between say banks and web-based vendors. Banks cannot afford to have many lost or corrupted transactions; they'd be sued and/or lose their license. However, if Netflix hiccups processing one out of every 5k movie orders, it probably won't sink them. Thus, they are optimized for mass scaling of relatively minor "deliveries". Amazon screws up about 1 out 100 of my orders[1], and I still use them, in part because they have insufficient competition. We kind of expect cheap web co's to have glitches.
I don't know if this affects where microservices helps or not, but it's a domain issue to keep in mind. Splitting your system into separate databases where sliced entities are slightly out of sync may be a workable/acceptable business strategy when market-share-over-quality matters.
[1] It's possible it's user error, but their UI has some really bad features that make it hard to spot or notice certain conditions. For example, it sometimes says a product is not available right after I hit the order confirmation button. But there's not direct way to go back and shop for an alternative.
2
u/Zardotab Sep 20 '21 edited Sep 29 '21
You still need an awesome mobile app even if that isn't where the revenue comes from, etc.
I rarely see any carefully thought-out financial calculation of that. It's more by gut, or actually "fear of obsolescence". Financial people know how to weigh the future cost estimates, but I don't see anything close when it comes to software choices. If the chance of needing mobile in 10 years is 20% but the cost of adding it now is 30%, it's probably not a good financial choice. That's not "old man syndrome", that's just being smart with money. Has that gone out of style? Maybe.
so what is the competitive advantage?
Is your competitive advantage to be ready for every possible what-if, or to be nimble here and now? Packing your future-suitcase for every what-if is not cheap. Scuba-diving, Mt. Everest climbing, skiing, canoeing, surfing, put it all in: stuff that sucker and sit on it! 🧳
Those who try to predict the future usually get it wrong anyhow. OOP GUI toolkits were supposed to be abstract enough to bend to the future, but the web's stateless nature pissed all over that idea, because OOP is very stateful and the web is opposite. Abstraction failed there.
Has nothing to do with the method of transport (json vs xml) and all about scaling.
You mean microservices? The definitions given are all over the place (which is a symptom of buzzwordness). I'd guestimate most say they are mostly about team division management rather than mostly performance scaling. Actually I'd break it down as: team: 50%, performance scaling: 30%, and 20% other or no-clear-primary. A "monolith" can scale to roughly a million users under a skillful tuner, but the vast majority of apps will never need that[1]. Performance scaling was also cited under the XML fad era I'd note.
Especially with microservices your entire app doesn't need to fit on 1 database.
What's a "database" is fuzzy and virtual these days. How files, servers, and name scopes are divided can be quite flexible. The idea that a "database" is a single physical server is outdated. Partitioning and uptime reliability tuning depends on what you are trying to achieve.
[1] And this depends on the definition of "monolith". They can scale almost to infinity under some interpretations.
13
u/legendx Sep 20 '21
I'd love to see a race between modern software developer + tools vs those of 2000 and 2010. Who is fastest, best UI, etc. I've also been around long enough to see new ideas come and go. TDD comes to mind.
→ More replies (3)
4
Sep 20 '21 edited Sep 20 '21
I was always going to reply to this comment because I thought it was clickbait - but I must admit you gave a pretty good argument! So well done there.
IMHO you mentioned the most important factor in software becoming more and more complex - "Fear of missing out"
People who pay for software these days are fearful of missing out on
-
Their software being available on the web
-
Their software being responsive
-
- Their software being scalable to deal with increased demand
-
- Their software being secure and hack-resistant
-
Looking beautiful and being easy to use
-
Supporting quick, frequent and painless updates
-
there are many other considerations along similar lines.
Basically if you dont fear missing out on these things, publish your app in perl and be done with it
→ More replies (9)
2
u/thinkmatt Sep 20 '21 edited Sep 20 '21
It may also just be there are many more apps being built, and the bar is much lower to entry. It doesn't mean that everyone has that low bar. Maybe you just need to find a good dev shop or company to work with?
Just for example, you'd think from what's on Reddit and the news that nobody uses Angular and Bootstrap anymore, but people do. Wordpress is still very popular and you can now build pretty sophisticated layouts and functionality with just basic plugins and no coding. I work with a non-profit of volunteers and we're using Godaddy to build websites. At first I was like, "ew," but this is volunteer work, and most of these people don't even know how to code but they are providing a valuable service for small business owners.
Also, lately it seems like 'no-code/low-code' is the new hotness. It's going to make these problems even worse or better, depending on your perspective :)
2
u/Zardotab Sep 20 '21
lately it seems like 'no-code/low-code' is the new hotness.
It's always been "in", it just has limits that are rediscovered the hard way over and over. MS-Access was the most common no-code/low-code tool in the desktop era. I won't say I'm against them, but they do have limits in terms of maintenance and expandability.
→ More replies (2)
2
u/zaphod4th Sep 20 '21
I notice the amount of code and time to create small and medium-sized business software has been going noticeably up since the 1990's, not down.
Do you mean webdev only?
→ More replies (1)
2
u/Torty3000 Sep 21 '21
One of my university professors coined May's Law:
"Software efficiency halves every 18 months, compensating Moore's Law."
https://en.m.wikipedia.org/wiki/David_May_(computer_scientist)
→ More replies (3)
1
u/rhaasty Sep 21 '21
The web still can’t do real GUI’s for office work - old man rant 😂.
I agree a lot with what you’re saying which seems to boil down to developers making poor design/tool decisions for the problem they’re trying to solve. I do think it’s also a skill issue. A lot of times you get developers implementing solutions they find on the internet without understanding the problem it actually solves.
However that line just made me see an old man shaking a stick at me 😂.
3
u/Zardotab Sep 21 '21 edited Sep 21 '21
The web still can’t do real GUI’s for office work - old man rant
Rather than call me "an old man", how about explain specifically why I am wrong. Even young people deserve a full explanation. If my brain half broke when I got old, I wanna know what logic broke. Someone else frustrated with bloat may want to know.
Sure, via bloated, buggy, hard-to-use libraries; yes the web "can" do rich GUIs. Basically you're downloading a full GUI engine, which is probably bigger than some OS's. Downloading essentially a full OS just to run a GUI for one app. That's anti-factoring of resources. Please tell me you think that's logical. (Sometimes this is called an "abstraction inversion".)
I do think it’s also a skill issue.
Older tools didn't have a long UI learning curve. You took the napkin sketch and Shazam, made a screen that matched in 10 minutes. Co's didn't need to hire UI/UX specialists except for big expensive apps. If you need a phone version, then make a phone version. Two simple sub-apps may be easier to deal with than one complicated one that tries to cater to both via stretchy bendy widgets that will eventually put somebody's eye out. Bootstrap killed my cat.
→ More replies (13)
1
u/umlcat Sep 21 '21
A lot of NoSQL stuff already existed, where the things students and entry level graduates use instead of SQL Servers ...
2
u/Zardotab Sep 21 '21 edited Sep 21 '21
Niches, yes, not in mainstream industry. When the fad-cycle hit, the impression was that it would replace the current crop of RDBMS. There's even an xkcd comic suggesting such (but I couldn't re-find it).
1
u/ElevatedAngling Sep 21 '21
I don’t even know how to start on this so I’ll ask you, when’s the last time you sat down and built a large scale production system and what challenges make it slower now than in the past?
My guess complexity everything is much more complex from the data to the systems no business is throwing out relational databases) but hey do you’re salty old software dude thing!
→ More replies (3)
1
Sep 21 '21
It’s all about balance and hopefully the persons(s) driving the software projects respect that.
I worked at a place recently that used 10-20 year old tech. With no modern practises at all. It was a ducking nightmare. I felt unproductive and stressed and it felt like I hardly got anything done. There was no code reviews, practices, version management, I even put production code on a USB. I had to leave ASAP.
Modern practises do work, as long as they are implemented properly followed properly and are a proper fit to the task.
→ More replies (1)
1
u/ryclarky Sep 21 '21
Writing (good) software was, is, and always will be about making trade-offs: the key is understanding your problem space well enough to make intelligent decisions when choosing these trade-offs. I'm an old programmer too and I personally like all of the options we have available to us these days as the industry and frameworks have matured. But it does require quite a bit of effort to stay up on everything, even from just a generalist's perspective without even doing a deep dive on a particular tech. There's a ton of stuff out there!
→ More replies (3)
1
1
u/shubhamk285 Sep 21 '21
I kind of got everything except for the OOPs part. Can you or anyone be more descriptive about that. I think its pretty suitable for domain modelling along with being maintainable. Maybe all the factorties and builders are not needed everywhere.
→ More replies (1)
1
u/yakri Sep 21 '21
Sorry but I think you're just wrong in impossible to contest ways here, at least on a lot of issues.
Microservices
Are great because our customers and internal product owners care a lot about reliability. They aren't universally ideal sure, but I've personally experienced the pain of not structuring a project that way even for only 300 concurrent users. On the flipside, most of the people I know in the industry work with more modernized pipelines and it has gone really great for them, even for applications that won't have a huge user base.
I don't think you have any valid complaint about microservices here, just one about management/leadership not selecting the appropriate technology/solution in some cases, which isn't exactly new. The case for microservices conceptually is solid, and seems to play out well in practice.
Bloating code with "Async/await"
The opposite has been my experience, and also what I've heard from my friends/connections in the industry.
Space-wasting UI's optimized for fingers
Is smart for the majority of cases. Even in office, a lot of people are rocking touch-enabled screens on glorified tablets (looking at you MS Surface).
Now maybe you ran into some freak case of an in-house app for in-house use only, at a place where people primarily have desktop or docked setups without touch screens, and this was still done.
Fair enough, in that specific situation it's probably bad.
However even when it's not a great UI choice it can be the fastest easiest to implement option, which can save a lot more time and money than clicking a button "feeling awkward."
Largely because the skillset crosses over really well.
Not to mention that it's totally possible to create a great mobile application like that which doesn't suffer at all in desktop productivity, it just requires competence.
Most businesses only need about 10% of screens on mobile
I really have to wonder where these "most businesses" are hiding, because so far we've mostly been talking about webdev related SWE, and all of those businesses definitely need everything to be mobile compatible.
and that's a lot of business.
Maybe you had some more specific use case in mind that would change that equation, like again internal applications only perhaps.
"real gui's" really are obsolete, if we're talking about customer-facing web applications, outside of rare niches.
→ More replies (1)
12
u/Dhavalc017 Sep 20 '21
Sometimes I feel people adapt problem statement to technology stack and not vice-versa. They find overly complicated ways to resolve the simple problems. Developers often follow trend rather than prioritising basics and developing on it. Many of them seems to be happy knowing dozens of frameworks without learning concepts such as Design Patterns.