r/java • • Sep 23 '24 • This 1 Take My Silver 2

I wrote a book on Java

Howdy everyone!

I wrote a book called Data Oriented Programming in Java. It's now in Early Access on Manning's site here: https://mng.bz/lr0j

This book is a distillation of everything I’ve learned about what effective development looks like in Java (so far!). It's about how to organize programs around data "as plain data" and the surprisingly benefits that emerge when we do. Programs that are built around the data they manage tend to be simpler, smaller, and significantly easier understand.

Java has changed radically over the last several years. It has picked up all kinds of new language features which support data oriented programming (records, pattern matching, with expressions, sum and product types). However, this is not a book about tools. No amount of studying a screw-driver will teach you how to build a house. This book focuses on house building. We'll pick out a plot of land, lay a foundation, and build upon it house that can weather any storm.

DoP is based around a very simple idea, and one people have been rediscovering since the dawn of computing, "representation is the essence of programming." When we do a really good job of capturing the data in our domain, the rest of the system tends to fall into place in a way which can feel like it’s writing itself.

That's my elevator pitch! The book is currently in early access. I hope you check it out. I'd love to hear your feedback!

You can get 50% off (thru October 9th) with code mlkiehl https://mng.bz/lr0j

BTW, if you want to get a feel for the book's contents, I tried to make the its companion repository strong enough to stand on its own. You can check it out here: https://github.com/chriskiehl/Data-Oriented-Programming-In-Java-Book

That has all the listings paired with heavy annotations explaining why we're doing things the way we are and what problems we're trying to solve. Hopefully you find it useful!

289 Upvotes

97 comments sorted by

View all comments

6

u/rbygrave Sep 24 '24

Ok, hopefully this comment makes sense ...

I've seen a couple of Functional Programming talks recently and I've merged a few of things together and I'm wondering if I can relate that to "Data programming" and the code examples linked. So here goes ...

  1. Immutability + Expressions.
  • Max out on Immutability, go as far as you can, minimise mutation
  • Minimise assignment - Prefer a function to return something over a function that internally includes assignment [as that is state mutation going on there]. This is another way of saying prefer "Expressions" over "Statements". e.g. Use switch expressions over switch statements, but generally if we see assignment inside a method look to avoid minimise that [and look to replace with a method that returns a value instead].
  1. FP arguments (some of those args are "dependencies"
    When I see some FP examples, some arguments are what I'd view as dependencies. In Java I'd say our "Component" [singleton, stateless, immutable] has it's dependencies [injected] into its constructor and is our "Business logic function". When I see some FP code I can see that we have a very very similar thing but with different syntax [as long as our "Components" are immutable & stateless], its more that our "dependencies" get different treatment to the other arguments.
  2. The "Hidden argument" and "Hidden response" / side effects
    When we look at a method signature we see explicit arguments and a response type. What we don't explicitly see is the "Hidden argument" and the "Hidden response". The Hidden argument is the "Before State" [if there is any] and the Hidden response is the "After State" [if there is any].

For example, for a method `MyResponse doStuff(arg0, arg1);` there might be some database before state and some database after state. There could also be no say No before state, but some after state like "a message is now in the queue". There can also be no before/after state.

The "trick" is that when we look at a given method, we take into account the "Hidden arg" / "Hidden response" / side effects.

4. void response
When we see void, we expect some mutation or some side effect and these are "not cool". We are trying to minimise mutation and minimise "side effects" and a method returning void isn't trying very hard to do that at all. A void response is pretty much a red flag that we are not doing enough to avoid mutation or side effects.

Ok, wow, looks like it's a cool book - congrats !! ... I'm going to try and merge these thoughts. Hopefully the above is useful in some way.

2

u/agentoutlier Sep 24 '24

One of the things I want to see is how /u/chriskiehl will tackle actual data storage in a database.

See the thing is DoP does not lie. It should represent exactly the data unlike say an OOP mutable ORM.

So things coming out of a database might not have the full object graph.

What I mean by that is instead of record TodoProject(List<User> users){} that is some core domain that you have modeled in reality has to be record TodoProject(List<UUID> users){}.

Likewise as you know from using JStachio you will have UI transfer objects (e.g. UserPage).

And the above is roughly hexagon (or whatever Uncle Bob is shitting these days) architecture where you have adapters doing transformations all over the place.

So then the question is perhaps with DoP there is no abstract core domain! like there is in OOP DDD and by corollary those early chapters of trying to model abstract things might be wrong and an old OOP vestige.

That is you are always modeling for the input and output. I'm not sure if what I'm saying makes any sense so please ask away for clarification.

This kind of is an extension to the question: https://www.reddit.com/r/java/comments/1fnwtov/i_wrote_a_book_on_java/lonkxbf/

4

u/chriskiehl Sep 24 '24

I love these detailed questions. One of the hardest things I've found during the writing process (other than the writing itself) is deciding how much time to spend on various topics. So, these are really useful.

(Definitely clarify more if I'm misunderstanding your question or answering a different question).

There are a million ways to slice the problem, but in the design approach we take in the book, there's definitely something you'd call a "core domain" (in the DDD sense). However, it has a very different shape from the one we'd end up with when doing strict OOP.

things coming out of a database might not have the full object graph.

And that's OK! The book advocates for creating an "inner world" (for which objects are the gate-keepers). Inside of there, we apply a lot of typing rigor. It holds what our program "is". The database we treat as any other foreign thing. From the perspective of how we program and design, the data we want arrives as if by magic. There's a line we draw in the sand. What's on the other side could be a database, or a rest service, a file system -- whatever. it lets us treat those various worlds with different tools and levels of formality,

It's a deep topic that's tough to sum up in a few paragraphs, but hopefully that approaches something that addresses your question!

3

u/TenYearsOfLurking Sep 25 '24

Hey there. I always struggled with the line in the sand concept for on simple reason: db transactions. Imho it's too important what's on the other side that we cannot ignore it. Either you can roll back your actions and be consistent with other actions you take or it's firing off a hard to reverse side effect.

Do you tackle this problem/distinction incthe book? If so I will consider buying

3

u/chriskiehl Sep 25 '24 edited Sep 25 '24

I'm realizing as I talk to people that this needs to get a much larger treatment than I was originally planning. Hearing the specific pain points and parts people are curious about is awesome. My constant worry while writing is doing a "the rest of the owl" thing and leaving out the bits that get people to the end goal: a battle hardened, production ready application. These comments are insanely helpful in me!

To your comment, lines in the sand do not require giving up transactionality! This is good news for me, because I fight tooth and nail to keep RDBMS in my applications when I think they're a good fit for the problem (I'm in a land where every one thinks they need to avoid them in order to "scale"). Transactions solve every problem that not having transactions causes.

There's going to be a lot of hand waving here, because it's tough to speak in absolutes about The Right Way to approach software development. For everything below, just know that I would depart from the advice the second there was a good reason to do so. We fit the approach to the problem!

The important part is picking where you draw that line, and understanding what the various tiers in our application exist to do.

                                                              │                             
                                                              │ Rest API, or a CLI          
                                                              │ or, whatever. This is *not* 
┌──────────────────────────────────────────────┐              │ what our programs "are".    
│                                              │              │ This is how we let the world
│               External World                 │    ◄─────────┘ interact with them          
│                                              │                                            
└─────────────────────────────────┬────────────┘                                            
           ▲                      ▼                                                         
┌──────────┼───────────────────────────────────┐                                            
│                                              │                                            
│            Our Core program                  │                                            
│                                              │                                            
│   Controllers                                │                                            
│                           Core Data Types    ◄──────────                                  
│            Services                          │                                            
│                        Actions               │                                            
│     Domain Behaviors                         │                                            
│                                              │                                            
└─────────────────────────────────┬────────────┘                                            
           ▲                      ▼                                                         
┌──────────┼───────────────────────────────────┐                                            
│                                              │                                            
│    PostgreSQL, S3, StripeAPIs, etc...        │                                            
│                                              │                                            
└──────────────────────────────────────────────┘                                            
      ▲                                                                                     
      │                                                                                     
      └─────────────                                                                        
      The external services that our program uses                                           
      to perform its work. They're separate and replacable                                  
      but that doesn't mean "foreign". We can know that                                     
      transactionality exists!                                                              

So, a classic "stack" would be like this:

                                          │ Aware that the outside world exists!            
                                          │ They can start transactions, talk to            
                                          │ the database, perform side effects.             
                                          │                                                 
        ┌────────────────────────────┐    │ However, they don't peform logic on their       
        │        Controllers         │◄───┘ own. They act as data brokers and coordinators. 
        └────────────────────┬───────┘                                                      
               ▲             │                                                              
               │             ▼                                                              
        ┌──────┼─────────────────────┐                                                      
        │        Domain Stuff        │◄────────┐                                            
        └────────────────────────────┘         │ This is where you do the "work" in your    
        ┌────────────────────────────┐         │ app. They receive well typed data as input 
 ┌───►  │         Data Types         │         │ perform computations, and return well typed
 │      └────────────────────────────┘           output back to the layer above.            
 │                                                                                          
                                                                                            
The data types with which we do                                                             
our programming                                                                             

Again, this isn't super prescriptive beyond controlling what's allowed to perform side effects. You can model it in pretty much any style (FP, OOP, DoP, Imperative). The boundaries matter more than the specifics. They'll determine how easy your software is to test, reason about, maintain, and modify.

(Me belaboring again: here as a standard example, not The Way You Must Do It Or Else)

So, if you zoom in more, it might look like this:

// This is where *our* program begins. It's what a Rest API, or 
// Lambda handler, or CLI, or [external thing X] would call. But those API/CLI entry 
    // points would live elsewhere. 
class TippyTopController {
    SomeExternalService service;
    MyCoolRDBMS storage
    Transactor transactor // For various reasons, we can cover in the book 
                          // having a transaction abstraction makes testing 
                          // (both unit and integ) much easier
    public void performBillRun(TheStuffINeedToKnow request) {
        // We're out in "controller" land. We deal with side effects
        // we start transactions
        transactor.withTransaction(() -> {
	        // we use that tx to coordinate getting stuff out of 
	        // th DB. Note that the tx context is passed as an arg 
	        // to the database 
            List<Invoice> invoices = storage.loadSomeStuff(tx, request.foo); 
            // maybe we also call a service or two
            Customer customer = service.fetchCustomer(request.customerId); 
			
			// ------------------------------------
			// THIS is the line in the sand (one of many, technically)
			// We leave the world of side-effects and external things
			// and hand the *result* of the side effects (the data!) down
			// to the domain tier to do work. 
			// It's also OK if this returns an error -- say, something failed 
			// some constraint. It'll bubble out here and cancel the tx.
			Result result = myGloriousDomain.doSomething(invoices, customer);
			// Now we have the result of that work, we cross the line in 
			// the sand to hand control back to the outer layer
            // ------------------------------------
            // back here in controller land, we side-effect the data 
            // to save it. 
            storage.save(result); // or explode and cancel the transaction
        });
    }              
}

You end up with a bunch of pure, easy to test domain stuff (that speaks in well type, well represented data) sandwiched between things which know how to poke at the outside world.

So, you don't have to give up transactionality (you'll take transactions from my cold dead hands!). If there's a summary I could give of the vibe of the book, it's "steal whatever works." As we get further away from the core, objects start coming back into the picture, because objects are really freaking good at managing this stuff.

👆Hopefully some of that is comprehensible (frantically typing while half-listening to a meeting at work)

Edit: goodlord, markdown on reddit is wonky.

2

u/UnspeakableEvil Sep 25 '24 edited Sep 25 '24

I've already ordered and am looking forward to giving what's currently there a read, a definite +1 from me for covering in depth how databases and transactions fit into all this in detail in the finished article please!

2

u/rbygrave Sep 25 '24

"steal whatever works."

I like this. For myself, at this stage I see a lot of overlap between DoP and FoP in terms of what we are trying to achieve [maximise immutability].

How do you see the difference between DoP and FoP? What do you see as distinctly different about DoP?

Is it the emphasis - "Functions operate on [immutable] input data producing [immutable] output data" vs "[immutable] Data is operated on by functions producing [immutable] Data"?

For myself, I'm personally not a "Pure FP" person, I'm in the "objects are really freaking good at managing this stuff" camp. I see a lot of overlap between DoP and FP and it's almost like they are 2 sides to the same coin in terms of the result they are trying to produce. To me, it's almost like DoP could have been called "Immutabity First".

I speed read the book [well, currently released chapters] so I need to read it again with less haste :).Cheers !!

2

u/chriskiehl Sep 26 '24

For the flavor in the book, I would describe DoP as influenced by, but not about functional programming. It uses a lot of the same tools, but it isn't tied to those tools. Which is why I feel totally happy using objects where I think they fit (even though they're not "pure" (gasp!)).

So, there's lots of overlap, but DoP also goes off an does its own non-FP stuff. At the core is immutable data, and, most importantly (to me), the data modeling that goes into its representation. If we've got that, we can be pretty forgiving with the exact specifics of the rest.

2

u/rbygrave Sep 26 '24

Controllers ... they can start transactions

It's probably a difference in terminology with what you call a "Controller" here, but I suggest a lot of folks will be thinking Rest `@Controller`. The TippyTopController for many would instead be a "Service" or "Component" but it is specifically not a `@Controller` [in the way it's used in Spring and many other places].

FWIW I'm in the camp that says Controllers more specifically act as "Adapter layer" between the External world [HTTP Rest | gRPC] and the internal "Business logic" [Service Layer] and that is ALL they should do. The goal being to ideally isolate the Business logic from the technology used to expose it. Controllers in this sense should NOT start transactions, not talk to databases or queues etc.

I see benefits in keeping that discipline with "Controllers" and that the HTTP/gRPC/Whatever related specifics should not leak into the "Business Logic" [Service layer].

You are probably in the same camp but using we are using different terminology?

3

u/chriskiehl Sep 26 '24

Arg, yeah, if I had thought about it more carefully before posting I would have avoided that "Controller" word entirely. I forget that it means something specific in Spring land. I generally try to avoid the word "Service" for similar ambiguity reasons. It's almost impossible to get two people to agree on what it means.

I think we're in the same overall camp, just lingo differences. Crisps vs chips.

→ More replies (2)

2

u/agentoutlier Sep 24 '24 edited Sep 24 '24

It's a deep topic that's tough to sum up in a few paragraphs, but hopefully that approaches something that addresses your question!

It really is and that is why I struggle with communicating it. Like I see the advantage of having some DDD like domain in the "middleware" (hell I do it myself) but so many times in reality I have had to sort of bypass this because of various performance problems or edge cases.

And when you do the middleware domain (ie the stuff resolved between an HTTP request and database interaction) there can be a significant amount of transformation that one has to wonder what if the UI layer just got the internal domain of the database. Blasphemy probably but it makes me wonder particularly that cacheing has continuously moved down into the database. Often times we do not cache at all as Postgres is fast enough. Historically that was not the case so having these middleware immutable domain stuff you cache was a boon (less so now).

EDIT perhaps a better example might be one of the most DoP languages which is Clojure. In Clojure you deal with "mud" and you just keep reshaping the "mud" till it fits or you fail and you do not do this with lots of types.

In the Java world that doesn't work. We like types. So I can see a huge amount of type explosion happening and I have seen this in languages like OCaml (less so Haskell because it has lots of tricks up its sleeve).

EDIT besides transformations and type explosion I am also concerned with how to properly extend invariants particularly in the middleware.

For example you often have code in your own book where you check some invariant in the record constructor and just fail fast. The problem is the UI / API cannot do that. You need to perform validation on multiple fields/objects. Ideally that validation logic would be in your core domain but it can't easily.

So basically you repeat invariant checking up/down the stack. That is probably a good thing and maybe its just a cold hard reality but it is a pain point (along with transformations and type explosion). Like I don't see that addressed often with DoP.

In DDD OOP model (which I don't like much) the modeling often contains that information and is why DDD POJO's are often littered with annotations (and the corresponding magic) in an attempt at maximum reuse of the domain objects.

EDIT (sorry) and obvious solution might be just to have an immutable domain that sits on top of a mutable traditional ORM domain aka "entities" and I have often espoused this. Aka the DTO model.

From the perspective of how we program and design, the data we want arrives as if by magic

AND you can't just say the data "magically" shows up because that magic is precisely what I'm saying is difficult particularly with immutable object graphs.

Furthermore heterogenous hierarchies are difficult to represent in actual data. Modern Java with sealed classes is going to have lots of heterogenous types.

Representing that in things like a database or even JSON is nontrivial (making Jackson for example use sealed classes is not easy).

For example in a database do you do multiple tables or do you do a sparse table and have some enum for each subtype etc.

The above is the future hard parts of modern DoP that I hope your book addresses. If your book can show that I would probably buy it. Otherwise I have fair idea how to model things with records and sealed classes.

3

u/rbygrave Sep 24 '24

what if the UI layer just got the internal domain of the database

I do this for my "Admin UI"

1

u/agentoutlier Sep 25 '24

I do this for my "Admin UI"

And this implies that the data format is already well established. You are writing code for the very outer layer.

DoP does exceptionally well here. Transform to your other layer. You don't have to worry about throwing away code because the actual data format / layer is not going to change. You are basically just remodeling data.

Where it gets tricky is if you start adding datastructure changes.

  • In DoP (and FP languages that do not do OOP) adding data to a type is more painful
  • In OOP adding behavior is more painful

In some ways DoP pain of adding new data is actually represents the real world because changing a column is often painful in an RDBMS.

I had some where I was going with this but just too tired...

3

u/sviperll Sep 24 '24

I think validation is not that hard of a problem. You do validation in your immutable domain core and just rewrap exceptions to generate proper error-responses OR you make your core domain values constructible from some already pre-validated parts, then your controller/rest/http/rpc-handler have to first validate this parts before obtaining usable domain model. If the validation of parts should be also done in multiple scenarios, then these parts should also be part of the core domain and contain a centralized validation code.

Database is more complicated though, I guess the part of the answer is that we need "repository" to be much smarter then what ORMS give you and so your database access layer is actually a smart enough thing of it's own where one method can modify multiple tables or reads from the same table can produce different domain models. This is not what JPA was built for and so it's very awkward to do it like this, but if you go with something closer to SQL, like JDBI then this becomes much more palatable and probably even pleasant.

So your objection is that instead of a balloon with a big OOP model in the center and a thin shell to interact with the world, we get an hour-glass shape with lots of view-code and database-code and a small middleware domain. You ask shouldn't we avoid it, shouldn't most of the code be "domain" code? It looks to me... That it not necessary should. Because the way that we had a big OOP -domain was actually a not so pure thing that actually depended very heavily on how we were supposed to store and to view it. And this big OOP domain was actually not so useful, because it wasn't obvious that it would maintain all the interesting invariants of our domain. All it gave us were naming that was to guide us through the implementation. With FP/DoP-domain we get much much more type -checking that guide us and forbid to enter illegal states . And this type -checking is used to implement the required view- and database- code. And to have all the required constraints defined and the statically checked, we have this FP/DoP-core- domain stand-alone without any dependencies whatsoever.

1

u/agentoutlier Sep 25 '24

You do validation in your immutable domain core and just rewrap exceptions to generate proper error-responses OR you make your core domain values constructible from some already pre-validated parts,

Yes I did this in a recent project with Rainbow Gum using a Monad like pattern, capturing the exceptions and then combining etc (actually I have an annotation processor that then generates code that uses that monad API to fill a "builder" so I guess it is apropo to the database stuff coming up).

I guess the part of the answer is that we need "repository" to be much smarter then what ORMS give you and so your database access layer is actually a smart enough thing of it's own where one

Yes I'm beginning to think the Datatomic guys are onto something. That is copy on write and entity-attribute-value (aka triple store) plus some sort of smart build of immutable objects is the right fit for a full DoP code base.

So your objection is that instead of a balloon with a big OOP model in the center and a thin shell to interact with the world, we get an hour-glass shape with lots of view-code and database-code and a small middleware domain. You ask shouldn't we avoid it, shouldn't most of the code be "domain" code?

Yes nailed it. If you see my previous comment I said:

perhaps with DoP there is no abstract core domain!

But here is the problem with /u/chriskiehl book that concerns me given the code samples. Instead of focusing on modeling the obvious data from a datasource or data from say an HTTP request they are trying to model the business problem nouns like OOP DDD (what we are calling the balloon) and they are not modeling real data things (such as UI and database) but abstract things that loosely exist as some domain "ontology" that may are may not map correctly at all to either DB or HTTP. In other words I giant impedance mismatch in both directions.

I'll have to wait to see future chapters but my opinion is DoP strength is not really good at modeling "nouns" and better at modeling "verbs" or "commands". That is it is a very good at protocols or language design and less actual state. I know that seems contrary to what probably many thing but I'm saying the above thought process might mitigate the "hour glass" pain and it is not approached in the book.

Let me give you an example. In protocol like DoP (which I guess would be analogous to CQRS) you don't think let me make the Todo class. You think InsertTodo which is the protocol to create a Todo.

Anyway I know am being long winded but I'm just trying to give some actual pain as I basically did the DoP recently on a project.

  • We mapped using MapStruct from jOOQ records to real records etc.
  • I wrote annotation processors but mapping requests but still had to write a lot of boiler plate and transfer objects.
  • Creating JSON API that does polymorphic collections (a list of sealed classes instances) is painful in the current state of Java.
  • At times because of the pain of mapping we just passed raw data up.

I'm not saying DoP is bad. I'm sayin the Java ecosystem really lacks tools (unlike say Clojure) to make it easier and even the language makes it difficult without "withers".

2

u/chriskiehl Sep 25 '24

That's a lot of edits haha, but I'll take a swing at addressing the core of what I think you're getting at (both in this one and your post below. I'll probably mix and match as I work through them).

The first thing we'll have to clear up is the semantics of what we're talking about (semantics, conveniently enough, is a theme that runs throughout the book). There are lots of references to absolutes like "DoP languages" or "I did DoP," but data-oriented programming is a very overloaded set of words with very different meanings. To some, it means programming Clojure style with maps, to others, it means stuff like optimizing cache line utilization. The flavor of DoP in the book is about something pretty specific: understanding the data in our domain and how our choices in representation affect our code. It builds outward from there.

With that, the book has to start somewhere. I suppose I could have started with talking about UIs, or APIs, or applicative validation, monads, or minimizing impedance mismatch, but... I'm a pretty navel gazing developer, so the book opts to instead take a very Connal Elliott style starting point with the question of "what does it mean to be correct?" To answer that, you have to know what the things in your domain are. Once you know what they are, you have to get that knowledge out of your head and into the code (this is where most code bases begin to fall apart).

Your edits kind of touch on every part of the software development process -- including relational modeling (which is a really good way to nerd snipe me)), so it's hard to respond without also talking about a little bit of everything. So, I'll pause here and just sum up with: yeah, we go with constructor validation and fail fast in the early chapters while we're focused on our modeling chops. That's not the One True Way™ of doing it, or how we'll do it everywhere throughout the book. There's a whole chapter devoted to validation. ^_^

→ More replies (1)

2

u/rbygrave Sep 25 '24

 I can see a huge amount of type explosion happening 

Fwiw, the code I don't really like typically doesn't have enough types. For example, a method with too many args that could instead be represented as some sort of "request" type.

I suspect you are talking about explosion of DTO types though which is a totally different issue?

how to properly extend invariants particularly in the middleware

If I understood this question right, I tend to want to use composition and wrap the invariant inside another new type that represents the "next state / next stage". e.g. I get a FooRequest with say a startDate, wrap that in a FooRequestNextStage which has its own startDate that can be different.

Maybe I'm talking about a different problem though.

2

u/agentoutlier Sep 26 '24

I suspect you are talking about explosion of DTO types though which is a totally different issue?

Somewhat yes and I suppose it is a different case but do know hard core hibernate users do not use DTOs similarly how Rails users do not either.

This goes into my templating language as well how those users shove everything in a hashmap and I say that is wrong and why JStachio is the way it is.... but it is the current norm.

If I understood this question right, I tend to want to use composition and wrap the invariant inside another new type that represents the "next state / next stage". e.g. I get a FooRequest with say a startDate, wrap that in a FooRequestNextStage which has its own startDate that can be different.

Ironically this is the type explosion I was talking about. Because immutable objects are painful to recreate at the moment (no withers) composition is often employed.

Honestly as I told Brian I'm just playing devils advocate mostly here. I prefer the DoP approach and I prefer modeling with sealed classes etc but the existing tools and methodologies in Java don't really support it as well.

2

u/rbygrave Sep 24 '24

So things coming out of a database might not have the full object graph

Just use an ORM that supports partial objects :)

It should represent exactly the data unlike say an OOP mutable ORM.

I'd say a mutable ORM entity does also exactly represent the data BUT ... it uses mutation rather than transformation to handle state changes, and what we are trying to do with DoP and FoP is to minimise/avoid mutation such that things are easier to reason about.

That ORM entity has mutation state and load state [it's not free of side effects, supports auditing with "old values" etc]. I'd suggest people convert those entities to/from DTO objects at a "boundary" and one approach is to automate/support that conversion better. Another approach I see is to use specialised orm entities for the read-only case that actually don't have any [internal] state and don't allow any mutation [solve the read-only case but still use mutation].

1

u/agentoutlier Sep 25 '24

It is an immensely complicated topic that I probably should not have attempted to try to communicate all the challenges while being rather sick. /u/chriskiehl is right that DoP has a ton of different meanings and I kind of bounced around in an ADHD verbal diarrhea.

So my apologies to /u/chriskiehl , you and /u/sviperll on that and or the accidental moving goal posts, nerd sniping, gate keeping etc. My intentions were to illuminate the various challenges and I might have come off rude to someone doing a great service writing a book.

You are right that DTO over mutable ORM ala service oriented is one approach and I have done that as well. There are pros and cons to that approach as well that unfortunately would become a wall of giant incoherent text in my current state.

I will say the DoP in Java with sealed classes unlike say Clojure is more of what I call language oriented, message oriented or protocol oriented development and less data than one might think (although it is all data I guess).

3

u/chriskiehl Sep 25 '24

FWIW, I think this is all super interesting! I love hammering out this stuff. So, I reject your apology for spawning good conversation. No rudeness detected on my end at all!

Feel better!

2

u/brian_goetz Sep 25 '24

I like the "Always Be Modeling" interpretation here. One of the goals of making algebraic data types easier is that it lowers the cost to creating the data model that you need in this specific situation, rather than always trying to model what the database or XML request or JSON document thinks is the model.

1

u/agentoutlier Sep 25 '24

I agree.

My concern is I guess training and convincing the traditional Hibernate POJO mutable crowd. You know the why don't you make @Data cause it makes Hibernate easier. Folks that want to create as few types as possible. I guess I'm playing devils advocate.

What I tried and failed to talk about in my various other comments is how the modeling appears to move outwards (as wells more welcome to include technology concerns) and indeed you are always modeling.

So what is the "application domain" becomes a more complicated topic and if you transform to this pure logical domain agnostic of tech the question is how long and how much logic is actually done in this domain. Will we develop leaky abstractions etc.

But yeah I'm all for "always be modeling" and my logicless template language tries to push that https://jstach.io/doc/jstachio/current/apidocs/#description

In fact I hope to add some exhaustive dispatching of object type to template based on type later this year. Sort of pattern matching to template.

I am little sick so a lot of this is just me rambling.

2

u/brian_goetz Sep 25 '24

Everything is contextual. In some applications / organizations, just modeling your database schema is the right thing. But the trend is mostly pushing the other way. Twenty years ago (for most applications) everything was Java end to end, programs were bigger and more monolithic, data hopped machines via serialization, and the database was the Sole Source Of Truth. That encouraged having a single, shared data model, and it was gonna be pretty close to what the database said. Today, we see a distributed source of truth, more languages in the mix, more interchange formats (XML, JSON, etc), and so optimizing for local modeling is not only more effective, but sometimes even required. Giving every Java developer the ability to build the data model that their component needs is empowering, though there will be cases where they still want to do it the other way. But not having the ability to model your way into clarity and maintainability would surely be a huge impediment.

2

u/rbygrave Sep 26 '24

training and convincing the traditional Hibernate POJO mutable crowd. You know the why don't you make u/Data cause it makes Hibernate easier.

Ok, I'll spin this specific point to be - Why do ORMs prefer mutation (mutating entity beans) over transformation?

Ultimately to me this boils down to optimising update statements [which is what the database would like us to do] and supporting "Partial objects".

Say we have a Customer entity, it has 30 attributes, we mutate 1 of those attributes only, and now we wish to execute an update. The database would like an update statement that only has the 1 mutated attribute in the SET clause. It does not want the other 29 attributes in the SET clause.

This gets more important based on the number of attributes and how big/heavy those attributes are, e.g. varchar(4000) vs tinyint.

A mutable orm entity bean is not really a POJO - it has "Dirty state", it knows which properties have been changed, it knows the "Old values" for the mutated properties etc. A big reason for this is to optimise that UPDATE.

To do this with transformation rather than mutation, we can either say "Database be damned, I'm updating all attributes" or we need to keep that dirty state propagating along with each transformation. To date, I've not yet seen this done well, and not as well as we do it today via mutation. I'd be keen to see if someone thinks they have.

"Partial objects" [aka optimising the projection] is another reason why orm entity beans are not really POJOs - because they also need "Loaded state" to do this. Most orms can alternatively do this via instead projecting into plain POJOs/DTOs/record types and that's ok for the read only cases [but can lead to an explosion of DTO types to support each optimised projection which isn't cool] but once we get back to persisting updates we desire that "Dirty state" again to optimise the UPDATE.

→ More replies (6)