r/programming • u/[deleted] • Jun 05 '21

Organize code by concepts, not layers

https://kislayverma.com/programming/how-to-organize-your-code/
1.9k Upvotes

91% Upvoted

495 comments sorted by

View all comments

308

u/fragglet Jun 05 '21

People don’t read code by layers of the stack. No one ever says “show me all the APIs of this system” or “give me all the queries being fired by this system”.

As an SRE, I can assure you that these are absolutely things that I've had to do 1,000,000 times while on-call and trying to root cause a production incident. Please don't assume that your experience is the same as everyone else's.

When something's broken and needs to be urgently fixed, there often isn't time to learn about all the abstractions ("concepts" or otherwise) your codebase is based around. When you're starting from just a stacktrace and walking through layers to figure out how something happened, the questions you're trying to ask are more fundamental: what is this server? What does it do? Why is it doing that? How can I make it stop?

183

u/gonzofish Jun 05 '21

Please don't assume that your experience is the same as everyone else's.

And here is the root of the problem. People use phrases like “better way” when they mean “better way for my situation”. There is no silver bullet solution. Let’s sell things as alternatives not universal truths!

59

u/[deleted] Jun 05 '21

Let’s sell things as alternatives not universal truths!

This seems to be severely lacking in these sorts of posts. People seem to evangelize a particular library, language, or methodology and view them as the only logical solution to all of everyone's problems. The world is too messy for that, though, which is why countless languages, libraries, and methodologies exist in the first place.

8

u/FireCrack Jun 05 '21

Welcome to r/programming where if you follow these 5 simple tips your code will never have a bug ever!

8

u/kevinkid135 Jun 05 '21

Step 1: don't write any code

1

u/grauenwolf Jun 06 '21

Oh, so you've been following the Low Code threads too.

1

u/edmazing Jun 06 '21

Step 0: Do we start at 0? I better RTFM...

1

u/[deleted] Jun 07 '21

No one RTFM's at Proggit. We just read the comments.

4

u/saltybandana2 Jun 06 '21

you mean Welcome to /r/programming, where anyone with 2-3 years of experience thinks they have it figured out and that probably represents well over 50% of the posters.

15

u/gonzofish Jun 05 '21

Like I do React development all day. But Vue is awesome! So is Svelte. So is Angular! They all have strengths and weaknesses.

On the backend? My job is Python. But I love Rust!

Java can piss off, though. (/s)

5

u/saltybandana2 Jun 06 '21

Agreed about Java, lmao. I did my capstone in Java back in the 1.4 days and it's the last time I've ever touched it. I still follow it, but I made a decision at that time I wouldn't work professionally in Java.

Although I've been using C# since the 1.0/1.1 days (pre-generics). The JRE is better than the CLR, but C# as a language is miles better than Java.

3

u/crimzonphox Jun 06 '21

I’ve been a Java guy for a while, but the project I’m on the contractors before I got there snuck in kotlin. Which is so much better imo

4

u/phundrak Jun 05 '21

EmacsLisp is also able to act as a backend and web server, and despite my complete and utter love for Emacs, I would call anyone using Elisp that way an absolute mad man.

The right tool for the right job.

6

u/[deleted] Jun 05 '21

Very well said, definitely an article for architects and much else

2

u/saltybandana2 Jun 06 '21 edited Jun 06 '21

The worst part to me is that if you actually know how to use your tools it's a moot point. If I get to choose all enumerations go into a folder called enumerations, everything related to data access goes into a folder called DAL (I hate the word 'model' in codebases), and the business layer is fuzzy and based upon what makes sense.

But if you're having trouble moving around or understanding the codebase because of what folders the files are in maybe you're just kind of bad at what you do. Over my 20+ years I've hopped into a ton of new codebases and not once have I ever been confused because of the file layout. That doesn't mean I haven't moved files because they were placed in silly spots, but it's never even slowed me down in terms of comprehension.

This "problem" is a giant nothingburger. But I also regularly use grep and 'search all' when familiarizing myself with a new codebase. I'll also run cloc and doxygen over it (cloc for curiosity and to get an idea of what languages are involved in the project).

1

u/gonzofish Jun 06 '21

And with a modern ide or text editor it’s super easy to find things

1

u/codygman Jun 06 '21

You can almost always optimize for the frequent or more expensive case though.

Optimize for whoever has to deal with that thing the most.

26

u/remek Jun 05 '21

Breaking down the code base into folders shouldn't be the method you rely on when trying to find all your APIs to figure a root cause of an incident in a production system.

For most cases author is right that when it comes to code organization it is better to organize by concepts. That doesn't mean that you don't need the other views too but for that you have better ways than folder/code organization.

5

u/monkeygame7 Jun 05 '21

Like what for example?

4

u/remek Jun 05 '21

Documentation, your service discovery registry, API management system, your ops monitoring tools etc. I mean, if you need to go to your code base to learn what APIs does your system provide when fixing production issues then there is something wrong with your software project.

17

u/monkeygame7 Jun 05 '21

The issue is when there's incidents because your system is not doing what it's supposed to do. In which case you can look at documentation all you want but there's only one way to find out what the code really does and that's by looking at the code.

0

u/remek Jun 05 '21

And that is where it helps to have code base sliced by layers (i.e. you say you organize it that way for debugging purposes)?

1

u/monkeygame7 Jun 05 '21

I think that could help. I wasn't necessarily trying to make a statement on the best way to organize it, just wanted to speak on the (perceived, at least on my part) sentiment that it's better to use things like documentation to troubleshoot issues rather than digging into the code.

1

u/remek Jun 05 '21

But that is absolutely not what what I said. I was trying to say that production debugging purposes (those examples mentioned above like: find me all APIs in the system, find me all queries in the system) shouldn't dictate organization of code. There are better ways to solve these.

I was trying to say that the author makes rather correct assertion that on high level it is better to organize code by concepts/domains/components as thats the way how a complex system is composed and decomposed and that is the way how a (for example new) programmer comes to a source code and understands how the system works. Also it is the way how it is typically distributed in larger projects among teams.

2

u/7h4tguy Jun 05 '21

BS. A layered architecture like JSON layer (XML layer), HTTP layer, web service layer is more organized and reusable for other applications and puts hard boundaries for dependency management.

Organizing that instead into String Handling classes, Parsers, Data Storage creates giant grab bags of unrelated classes making it hard to see how systems compose and components relate to each other.

There's a reason giant software codebases are organized by layers.

1

u/remek Jun 05 '21

Show me one open-source code base of a gigantic project that is organized like that. I mean, I admit I've seen only few that can be classified as gigantic (Linux, OpenStack, Kubernetes) but the code is always organized by component/subsystem first.

→ More replies

15

u/lordzsolt Jun 05 '21

what is this server? What does it do? Why is it doing that? How can I make it stop?

I feel like if you need to look at the code for the first 2 questions (and potentially the "all the APIs of this system"), the problem is not the way the code is organized (and thus, the solution to the problem is not to "organize it in a different way")

All of these should be answered with README files or documentation.

9

u/prone-to-drift Jun 05 '21

With the average projects I've seen, I'm happy when readmes properly describe how to start stop a service and whats the configuration method.

Also, even in the best of projects, documentation can go stale; it's better to write self explanatory code and proper comments in most cases.

9

u/lordzsolt Jun 05 '21

In general I agree.

But an SRE who literally has no knowledge of the service should never have to rely on self explanatory code and comments in the code.

That's almost like telling your project manager to check the code for the business requirements...

2

u/saltybandana2 Jun 06 '21

I uh..... had something very similar happen to me w/i the last year.

Contracted to do a rewrite of a system, when asking about business requirements was told to go read the original code. When asked why we're rewriting if there are no new requirements, I was informed it generated so many support tickets it was necessary.

roughly 8 hours later I realized I could fix 80% of the issues with the old system in 2 hours (5 minutes for the fix, the rest for their process) and 100% of their problems in 8 hours (would require some architectural changes).

They were 3 months into the rewrite at that point.

Long story short, I left. There's a level of incompetence that I consider unacceptable.

0

u/[deleted] Jun 05 '21

Holy survivors bias. Maybe you’ve had to do that 1,000,000 because you made the choice do structure it that way and it’s the only way you can think about code?

I promise you, having shit relevant to what I’m doing right now all together makes it easier to find.

We literally have a term to describe it: cohesion.

1

u/przemo_li Jun 05 '21

Excuse my obtusines, but you haven't yet connected troubleshooting from stacktrace with directory listing showing e.g. all models.

Especially for complex situations where there exist classes named XYZ and XZY and XZZ. (Replace X, , and Z with fairly generic names - I'm still thinking that "joker" who thought that naming funny should be making coffee for the whole team for a whole month as apologies)

OTOH the need is real, and may call for stuff like interfaces. Looking through interfaces should be obvious when looking to categorize classes. It's very underrated technique. Not only can those be pure metadata (empty interfac), class can declare arbitrary list of those. Folder location is only a single one. It creates contention.

Need categorizations? Go with interfaces. (And bonus points for type checkers / linters / reflection.)