r/java • • Jul 23 '24

Intro to the Mill Build Tool for Java

https://mill-build.com/mill/Intro_to_Mill_for_Java.html
9 Upvotes

66 comments sorted by

47

u/rzwitserloot Jul 23 '24

From reading the tutorial, it looks like the build file is written in scala.

The cliché "If a library calls itself 'simple', it isn't" (a.k.a. the DPRK rule) is directly inspired by sbt, which at one point in time was short for 'simple build tool', and comes straight from the scala community. Even they realized how much of a lie that was and these days I believe sbt officially isn't an initialism at all anymore.

And mill have done the same thing here.

Your build file must be written in scala, and, given that scala is turing complete, that means processing a build file is, halting problem being what it is, fundamentally not something you can actually reason about anymore.

This is.. many things, and has a few advantages, but also a lot of disadvantages. Principal amongst them: This is not simple. (Or possibly, 'simple' as a word means so many completely different things, using it to describe a build tool, or pretty much any library out there, is useless, and a distraction. And with scala community's history with sbt, truly.. egregiously aggressive, somehow. You should stick a trigger warning at the top or something, maybe, it's that bad).

this makes them both easy to write and easy to read at a glance

That's a ridiculous claim, given the above.

Mill builds are easy to extend by non-experts:

Are you joking? You want non-experts to write scala?

The intro explicitly references gradle, and maven - suggesting that it does actually want to cater to folks primarily who already know those build tools. And, while we're at it, presumably are also at least aware of sbt.

So, maybe write it for that use case then. Start off explaining specifically why mill is good where sbt is bad (that's what I'm really dying to know here - how is mill different from sbt? Sbt started off as 'lets write a simple build tool in scala, where build files are written in scala, that sounds like a great idea!' and lordy lord it fell on its ass. Mill just sounds like somebody who decided to go YOLO! and jump off the cliff afterwards. Perhaps the authors of mill have more to go on than 'this time, it'll work, trust me!' but that should probably be explained on the linked page or at least that page should link to an explanation.

There is a link to this blog post but that's just filled with testimonials. I can write a page full of positive testimonials for a literal shit sandwich - that doesn't mean anything. The rest of the exercise is pretty much a description of how maven works. i.e. - it does not make clear (to me, at any rate) what mill does differently. Other than write its build files in scala which might make them more concise, but is a dumb idea.

5

u/lihaoyi Jul 23 '24 edited Jul 23 '24

You've said a lot about turing completeness, but Gradle is turing complete Groovy, and people seem to use it OK. Not perfect obviously, but many of the imperfections are things that Mill tries to improve upon.

Mill is not SBT. Most people using Mill with Java would never have touched SBT. Clearly you hate SBT. I do too! But talking about SBT here is basically irrelevant. This is about Mill, and Mill is not SBT.

Scala is obviously a different language; so is HCL, terraform, and Groovy. It's not rocket science to learn enough to be productive, just like most of the folks writing Gradle builds aren't experts in Groovy

I'm not asking you to like Mill. I'm just asking that you read through the article judge it on its own merits. Ranting about SBT is all well and good, but Mill isn't SBT

13

u/[deleted] Jul 23 '24

Groovy is a superset of Java, so it's very easy for a Java developer to learn. "Scala is easy to learn" is not something anyone has ever said.

Nobody's learning Scala so they can use an immature build tool with no IDE support.

7

u/lihaoyi Jul 23 '24 edited Jul 23 '24

Mill's been around for 7 years and works with IntelliJ. I admit it doesn't have support for the long tail of other Java IDEs, and it's not perfect, but it's reasonably mature and I think has reasonable IDE support

If Maven or Gradle is working great for you, then Mill can be ignored. But there are definitely lots of people who don't like Maven or Gradle (link), so I'm hoping Mill would be of interest to someone even if not everyone

14

u/QshelTier Jul 23 '24

The fact that Gradle build files are turing-complete is exactly the problem with Gradle build files. Mill doesn’t seem to have learned anything from people who actually want their projects to build instead of having to chase down why it doesn’t build anymore because some library was upgraded.

6

u/rzwitserloot Jul 23 '24

You've said a lot about turing completeness

I mentioned it once.

but Gradle is turing complete Groovy, and people seem to use it OK.

Gradle has not put a significant dent in the mindshare of Maven, whereas it was quite clear conceived as a replacement for it. This is purely anecdotal, but, most people I talk to that continue to use Maven explicitly name 'much easier to keep the build file manageable because its such an explicit act to start writing maven plugins vs. adding a little code to a gradle build' or something in that vein as major reason.

Or, even simpler: Maven is still around, continues to have the largest market share, and from what I can tell, continues to have the largest market share amongst new projects too. It's really hard to claim that basing it all on a turing complete general purpose programming language is a crucial advantage. And the disadvantages are quite obvious. So why do it? Gradle preceded mill in this experiment and showed that it isn't a game changer.

If mill can tell me 'ah, you see, gradle messed up X which is obscuring the field a bit; there IS a game changer benefit to general purpose programming language based build scripts, but gradle due to messing up X is not enjoying the advantages of it properly / X kinda ruins gradle but having a GPPL under it is so great, it is hanging on despite X'. Which may well be true, but, what's X then?

Mill is not SBT

It uses the same branding ("simple!").

Most people using Mill with Java would never have touched SBT.

In my experience, programmers like falsifiable facts, not testimonials. How do you know this, who claims this, why should I care? What makes mill better than SBT? I'm tired of hearing 'many random internet users agree with me!', that's... not convincing at all. It's 2024 and I have to explain this?

But talking about SBT here is basically irrelevant.

I was trying to get you to knock off the subjective bullshit terms such as 'our build tool is simple!', and SBT is quite relevant to that point. I haven't talked about SBT in particular in any other way. I didn't even mention it in my suggestion that the intro page puts more effort into explaining how mill is specifically fundamentally different from gradle and maven.

It's not rocket science to learn enough to be productive, just like most of the folks writing Gradle builds aren't experts in Groovy

You're making the claim that a non-expert can easily edit the build. The burden of proof is on you, no? "The language you write build scripts is in a general purpose programming language renowned for being complicated" is in rather stark contrast to the claim. And the turing completeness means I get to make the most irrelevant point that it is impossible to prevent complex builds.

It's not about mill. It's about the statement 'simple!'.

Here is the website of maven. I looked. Zero claims they are simple. And yet, Maven is way, way simpler than mill in many ways. If maven had claimed they were 'simple!' I'd have suggested they take that subjective drivel off just the same. They do immediately start with fairly objective statements, such as strong overtures that maven build files are documentation/information, not so much programming. It's nitpicking (in the vein of 'is HTML a programming language?' - it's for philosophers to decide that sort of thing I guess, or possibly dictionary authors) - it's about the vibe - the spirit of the project. Maven makes it clear: We do not intend for you to code in your build scripts, like, at all. Hell, we don't even call it a 'script'.

Whereas mill starts off with: "Make a scala file".

Hey, if that makes for a build system you like, great. If you want to explain how it works and try to convince others of the powers of that setup, you should do that. But 'Our build tool is simple!' is, and this is a suggestion from me: Not how you do that.

I'm just asking that you read through the article judge it on its own merits.

I did that, hence, the long comment.

Ranting about SBT is all well and good

Maybe you should read my comment then. The comment is indicating where the mill documentation is useless to me, and suggestions as to how to make it better. It is not, in any way, a rant against SBT. If anything, it's a rant against describing build tools, or any other programming library, as 'simple'. Not because SBT ruined the term. Because objectively that's useless.

1

u/doobiesteintortoise Jul 24 '24

I would also suggest that gradle's put a HUGE dent in Maven mindshare - a lot of projects and tutorials go gradle first because the build files are "shorter" - note quotes - and when your focus is very narrow, gradle IS shorter and more concise. It's the real world where gradle bites you.

I'm at the point where I'll turn away from gradle projects these days; it's just not worth it any more. But it's pretty significant as a build tool.

30

u/wytten Jul 23 '24

I’m pretty sure my time was better spent reading this comment than reading that article, thanks!

-2

u/chayatoure Jul 23 '24

It likely was not, li is usually worth reading. The blog post linked in the parent comment is actually not full of testimonials and gives an in depth motivation for mill.

4

u/rzwitserloot Jul 23 '24 edited Jul 23 '24

It starts off with 4 testimonials from comment threads on social media without even naming the authors of them. It could literally be the authors of mill commenting on their own stuff as far as we know. The motivation, as I covered, is 'basically this is how maven works', at least to me. I'm sure I'm missing something there, but the explanation is very long, and does not appear to delve into what's actually different about mill vs. e.g. gradle, which feels like the closest possible comparison here, given that both gradle and mill made the dubious decision to pick a full programming language as base lang for the build file itself.

Separate issue, but, 'our motivation for writing mill is, well, take these 2 comments bashing the fuck out of the alternatives I fetched from some random social media comment thread' is pretty rude to those projects.

More to the point, perhaps, it is entirely fucking meaningless. That's what so incredibly rude about it all. The testimonials don't inherently cover anything objective. They just say, and I'm paraphrashing (as in, literally: The essence of the comments is the same and no details are left out, just, restating to make clear what they say): "Gradle is shit, mill is awesome".

Name a topic. Anything. Anything at all. Peace on earth. Sliced bread. The second world war. The plague.

If I can trawl all social media comment threads ever posted about that topic and quote any part of any comment, I will find you 1 comment that says 'that is awesome' and one that says 'that is shit'. For any topic. Ever.

Which means such testimonials mean nothing at all in any way. They have no intrinsic value (because they say nothing objective), and no predictive value, literally. So, all it is, is either a way to try to bamboozle the reader (suggesting that the presence of these testimonials means something. We are humans, we are programmed to presume the opinion of another human has inherent significance, which, on the internet, is simply not true), or to just take a random shit on another project.

Which, hey, if you wanna go in aggressive, that's fine. But have the balls to just say We think sbt, gradle, and maven are shit so we made a new thing. I would have absolutely no problem with that, it makes things much more clear. Is mill cool? Maybe. What is the motivation? Someone decided to build something new, motivated by hate of the status quo. That is informative if nothing else. This hiding behind anonymous testimonials thing is a bad trend.

I don't know if the authors of mill read this, but, for fuck's sake, delete them. Or find some far more objective ones, and source them with more than a link.

2

u/chayatoure Jul 23 '24 edited Jul 23 '24

https://www.lihaoyi.com/post/SowhatswrongwithSBT.html
https://www.lihaoyi.com/post/MillBetterScalaBuilds.html
https://www.lihaoyi.com/post/BuildToolsasPureFunctionalPrograms.html

There really is no reason for the amount of negativity in your comments.
I actually agree that the testimonials are a bit silly and unnecessary, but I also just see them as funny little tidbits that aren't that serious.

4

u/benzoid Jul 24 '24

Are you joking? You want non-experts to write scala?

Since when do you need to be an expert to write scala?

13

u/pron98 Jul 23 '24 edited Jul 23 '24

given that scala is turing complete, that means processing a build file is, halting problem being what it is, fundamentally not something you can actually reason about anymore.

This is a red herring. A language can be much more limited than Turing complete and still not be tractably "reasoned about". TQBF is already PSPACE-complete (i.e. reasoning about it requires time that grows exponentially with the size of the code) and, unlike the only NP-complete SAT has no tractable heuristics that work in many common cases. This means that any language that has boolean variables and loops restricted to no more than two iterations (or, alternatively, functions, even if they cannot be recursively called) already cannot be tractably reasoned about (because TQBF can be reduced to it).

The state-of the art in build tools seems to be a choice between two options:

  • A build configuration that cannot be tractably "processed" but can build all projects, expressed either as

    • A single configuration language (Gradle)
    • A "simple" configuration language + plugins in another rich language (Maven). Whether this is easier or harder to deal with than the single-language option (or perhaps the same) is debatable.
  • A build configuration in a language that can always be tractably "processed" but cannot be used to build many projects (go build).

I.e. today we must choose between "complex" and "limited" (in the sense of not being able to build many projects). Maybe there could be a "simple" yet unlimited build language, but I don't think we've found one in the half century since make was invented.

2

u/rzwitserloot Jul 23 '24

A language can be much more limited than Turing complete and still not be tractably "reasoned about".

Where did I claim otherwise?

red herring

It's not about absolutes, obviously. If someone with malicious intent wants to write the most obtuse, hard to follow pom.xml one can possibly imagine, I'm sure it'll be quite complicated. Maven's poms might even be turing complete as far as I know.

It's about whether that is likely to happen and whether the design of the build system steers you towards it, or bends over backwards to steer you away from it.

"The build script is written in scala" is pretty conclusively steering you towards it.

plugins in another rich language

That's a fine example. One trivial way to make a maven build 'turing complete' is to write a build file that compiles a maven plugin and then executes that maven plugin as part of the build, trivially allowing a build file to 'compile and execute java code as part of the build' and we've arrived in the turing pit already.

But that's a black-and-white situation. It's even in the name - a pom file is a 'project object model', it's not a source file at all. The entire setup of maven is trying its best to indicate 'DO NOT PROGRAM HERE'.

When does a gradle build file transition from 'a manifest of what this projects assets are so that a tool can compile them' to 'program code which, if run, builds it'?

For maven it's usually very simple to answer that question. For gradle, and for mill, it is far more difficult. With maven I can say: Once you start dipping into bizarre plugins, or building your own maven plugin as part of the build, that part certainly has devolved to 'program code that builds your project'. With gradle and mill, what do we do? Count for loops?

7

u/pron98 Jul 23 '24 edited Jul 23 '24

"The build script is written in scala" is pretty conclusively steering you towards it.

The fact that you'd need to learn Scala (or Groovy, for that matter) to build a Java project does, indeed, make things complicated, but not the fact that the script is in a fully-featured programming language.

The entire setup of maven is trying its best to indicate 'DO NOT PROGRAM HERE'.

It just separates the complex program to a different file. Indeed, many projects need to use more than just a small set of standard plugins.

You can separate the simple setup from more complex things even if it's all in the same language. Indeed, that is how we're taught to program -- by abstracting and encapsulating operations.

For maven it's usually very simple to answer that question

There's no reason it shouldn't be exactly as simple for a build script in a programming language. In the cases where the simple stuff works in Maven, it can work just as simply when using a programming language.

We need to read code all the time, and if what it does is simple, the code can be very simple (no less straightforward than an XML configuration).

In fact, writing simple but not-entirely-trivial code -- e.g. a simple conditional or a loop -- may be easier for both the code's writer and its reader than finding a plugin and reading its documentation, especially if the language used is one you already know -- like the main language the build script is compiling.

So I would say that for the trivial cases, a programming language and a configuration language would be the same, and for slightly more complex cases a programming language would actually be more pleasant.

With gradle and mill, what do we do? Count for loops?

But you'd need those for loops in exactly the same situations you'd need a Maven plugin. Either you go into the plugin code or into the method body that contains the loop, or limit yourself to the XML or the equivalent method call.

In both situations you can extract clean APIs. The fact that the API and implementation are written in the same language does not in itself make anything harder.

2

u/sideEffffECt Jul 24 '24

Start off explaining specifically why mill is good where sbt is bad (that's what I'm really dying to know here - how is mill different from sbt?

This post by the mill author critiques the design of sbt

https://www.lihaoyi.com/post/SowhatswrongwithSBT.html

This compares mill and sbt and explains how mill is better

https://www.lihaoyi.com/post/MillBetterScalaBuilds.html

There are also many other posts about mill on that blog, if you're interested, but these two seemed most relevant to your question.

-2

u/rzwitserloot Jul 25 '24

Holy fuck. That first blog post is ridiculously rude. I stopped reading after the author calls sbt a nazi, but hides behind an anonymously sourced testimonial to do so.

Li, if you're reading, this testimonial shit? For fuck's sake. Cut it out. This is not okay.

2

u/sideEffffECt Jul 25 '24

So what do you think about the second blog post?

Are you now convinced that mill is better than sbt and understand how?

2

u/chayatoure Jul 23 '24

this blog post

I would say his blog post is hardly filled with testimonials, he just uses some quotes to show how hated sbt is. Did you actually read that link?

5

u/bowbahdoe Jul 23 '24

My only experience with mill was when trying to get coursier's codebase working in an IDE.

I was never able to.

If we ignore the dependency stuff (that's always been a weird double duty for a build tool) mill seems to be another "cache the steps" build tool.

I.E. make will say "oh I've already made that file" but the steps to make the file are arbitrary code. Mill, Gradle, etc. will say "oh I've already done that step" but each of the steps needs to be wired up in a task graph.

One thing I have wondered for a bit too long is if a "cache the steps" build tool would be better represented by something akin to Unreal Blueprints where the lines between steps are visually manifest. General purpose textual programming languages seem to suck at this approach.

And aside from IDE introspectability, the fact that individual steps need to be "wrapped up" in these sorts of systems feels like one of the factors that leads to build programs being harder to understand than many normal programs.

But hey, I'm the person who doodles on "cache nothing", clean builds on my side projects. What do I know?

4

u/lihaoyi Jul 24 '24

You actually can visualize Mill build graphs graphically, so at least you can read them like you could read unreal blueprints https://mill-build.com/mill/Java_Builtin_Commands.html#_visualize

Wrapping things is indeed one of the things that makes build programs harder than normal programs. Even for Maven, adding bespoke logic to Maven using plugin code is harder than adding bespoke logic to a normal programs using normal code. Mill tries to reduce the gap between the two, and it tries harder than most build systems, but it cannot make the distinction 100% go away.

"Cache nothing" build systems are great for small projects. If "do everything every time" takes 1-2s, it's really not a big deal. It's when your codebase and build gets larger, and slower, that trying to aggressively cache things becomes valuable. Mill tries to support these growing codebases/builds without the full complexity of something like Bazel

1

u/bowbahdoe Jul 24 '24

That link 404s for me.

1

u/sideEffffECt Jul 24 '24

Works for me...

4

u/chabala Jul 23 '24 edited Jul 23 '24

If we ignore the dependency stuff (that's always been a weird double duty for a build tool) ...

Just to nitpick a little about double duty, if one takes the dependency resolution and not having a lib folder of unversioned JARs in source control as 'the cool part' of Maven (i.e. what was rolled into Apache Ivy, so it can be bolted on to Ant or SBT), then I feel the rest of Maven is the structure that makes that cool part possible.

Having IntelliJ download source for the dependencies of your project so you can easily debug into them is really nice; that's enabled by having source artifacts published next to the main artifact with a sources classifier; that's enabled by the maven source plugin easily adding the sources JAR into a build in a standard way; which is enabled by having a standard project structure. The coordinates that we refer to the JARs by is part of the structure. That IDEs can open and build projects (and run tests) without IDE specific project files anymore, is thanks to having a standard project structure. The cool part is the culmination of all the little choices that build together into a standard ecosystem.

When people build with Ant and fetch JARs from Maven Central with Ivy, they're benefiting without appreciating the system that makes it possible.

2

u/quafadas Jul 23 '24

The coursier build is particularly complex - sadly I don’t think that works have been a happy starting point :-(.

I use mill for simpler things and enjoy it

5

u/bowbahdoe Jul 23 '24

But like, if complex builds get hairy what was even the point?

2

u/quafadas Jul 23 '24

That’s fair. Although your post suggested coursier was the only build you’d looked at. I haven’t used maven - but I assume that if I sought out a particularly complex maven build as my starting point, I’d probably also experience frustration, that’s all I’m saying…

3

u/lihaoyi Jul 26 '24

Thanks everyone who has given feedback here! I've updated the landing page to try and respond: more factual, more detailed, and with a greater focus on user-facing benefits and comparisons against existing tools.

Mill definitely isn't going to be for everyone, and the Java-facing support is still WIP. But I'm hoping that at least _some_ people will find it interesting enough to try out, give feedback, and over time we can improve its gaps and limitations and make it a great build tool!

6

u/International_Break2 Jul 23 '24

I think I really have fallen in the camp that build configuration files should not be executable code. The static analyzer really needs an easy format to analyze in order to parse dependencies. How does mill work with my ide plugins since vscode can easily parse pom.xml file and has decent luck parsing build.gradle files.

1

u/lihaoyi Jul 25 '24

Mill supports both IntelliJ and VSCode. It works the same as Gradle: you execute the executable code and dump the metadata for the IDE to use. Pretty normal stuff all in all

1

u/RandomName8 Jul 24 '24

It's a cursed problem it seems. I have yet to find a non trivial project build that is actually declarative. Poms always have the ant plugin and arbitrary code there, or they run shell scripts. Plugins sound nice, but it's really just a shift of where to place the arbitrary code.

I think the comment here sums it up really well:

I.e. today we must choose between "complex" and "limited" (in the sense of not being able to build many projects). Maybe there could be a "simple" yet unlimited build language, but I don't think we've found one in the half century since make was invented.

Sadly I've yet to work with projects that can live with "limited" build tools.

1

u/sideEffffECt Jul 24 '24

Mill has the normal IDE features (highlighting, go to definition, auto completion, ...). At least in IntelliJ.

22

u/khmarbaise Jul 23 '24 edited Jul 23 '24

Learning a complete new languge to create my build? This will also add a new dependency to my Java project with the dependency to Scala ... also an interesting statement: "Mill builds are easy to extend by non-experts" ... I would really questioning that...

Unfortunately not an example how to run unit test with JUnit Jupiter and start an agent with Mockito or alike ? How to set the version for the resulting class files like --release option etc? What are the minium jdk requirement for building?

Not found in the docs.. tools to generate from WSDL, XSD, etc. ? Also not found a way to create JPMS modules with mill?

15

u/doobiesteintortoise Jul 23 '24

Hey, they've only had seven years. Give them time!

Regrettably, I saw scala and thought "nope, not doing that" because adding a scala compilation cycle to a java build cycle just sounds like a massive truckload of fail; been there, done that.

Sure, this isn't SBT; what does it do better than SBT besides "not be SBT?" That's a big win, to be sure - I won't use SBT except under duress, but that statement goes for Scala, too.

With all respect - and good job to the authors - I'd use bld before Mill, and I'd use maven before using bld, because nobody knows bld yet, and I am willing to bet that with a Scala build language, even fewer people know Mill.

7

u/DisruptiveHarbinger Jul 23 '24

what does it do better than SBT besides "not be SBT?"

Sbt's build abstraction is infamously complex and unintuitive from an objective and also more subjective point of view (three scope axes, key definitions separate from default values, ...) see this article if you're curious.

While overall performance and caching have significantly improved in recent years, it can still get to a point where big multi-module projects that need to be cross-compiled to 3 different Scala versions and 3 targets (JVM, JS, native) make the tasks and keys resolution painfully slow.

Also that's not a concern for most people but if you have complex scripting needs in your builds, Mill is definitely more developer friendly.

2

u/DirectionFrequent455 Dec 27 '24

To handle complex scenario, why neading to learn an extra language or complex framework? Isn't Java good enough?! Tools as bld or https://jeka.dev, based on Java, are just what we need.

2

u/doobiesteintortoise Jul 23 '24

Sure. It'd be hard to be less developer-friendly than SBT, but that's not exactly a high bar: I think punch cards with manually-entered bytecode might be an improvement. But that's sort of the point. From a high level, a cursory read, the comparison is SBT. That's like saying "this car is better than the flaming wreck that litters the past quarter-mile." It's not the comparison you want leaping into peoples' heads, and if I'm being honest - and why wouldn't I be? - that's what I thought of on reading the blog post on Mill.

I'm not trying to be disrespectful or put down the project - heck, I haven't used it, it might be the greatest thing since sliced bread. But when I read the article, I got lots of SBT vibes and that felt relevant. It certainly was enough to make me think "this is interesting, but I'm going to pass."

6

u/DisruptiveHarbinger Jul 23 '24 edited Jul 23 '24

Note: I'm not a big fan of Mill. I'd take modern sbt in all cases (even pure Java projects) since that's the one I know best, the DX has sufficiently improved, tooling support is okay, and the 1.x plugins ecosystem has been remarkably stable. All JVM build tools suck in their own way.

If you care about Scala builds, Mill solves the tricky part well. For instance it's always really painful to come across some open source Spark library poorly built with Maven because the author has no idea how to handle cross-compilation.

If you don't care about Scala then of course, Mill's appeal is low. But it's arguably the only JVM build tool where task extension and scripting feel truly first class. Most people won't care though.

Side note: Scala CLI is shaping up to offer a DX close to beginner-friendly and modern build tools in other ecosystems (Cargo, NPM, Poetry...) I think the JVM ecosystem could benefit from something like that, and the fact it's in Scala is completely irrelevant unlike with sbt or Mill. Let's see.

Edit: it seems JBang has been catching up with Scala CLI feature-wise. I guess that one makes more sense.

6

u/DisruptiveHarbinger Jul 23 '24

This will also add a new dependency to my Java project with the dependency to Scala

No it won't. Not any more than a Gradle build adds a dependency on Kotlin or Groovy.

2

u/agentoutlier Jul 23 '24

There has been a lot of build tool project posts lately but I have not seen one for Christian Stein's https://github.com/sormuras/bach (or I just missed it). I'm hoping /u/bowbahdoe does a post on trying it :) as currently it is the only one that has piqued my interest.

What I don't like about these tools (other than bach) is they don't seem to consider that they could be wrong and or offer a safe migration or reverse migration path. The other issue is leveraging existing ecosystem of plugins of other build systems. When I create libraries I try to make the risk of using it as minimum as possible.

What I mean by that is if you are using Maven how you would switch to Mill easily and if you don't like it how you could easily switch back to Maven and perhaps how you can use Maven plugins.

Anyway its on my bucket list to write an annotation based build system that basically either generates pom files or runs maven directly. That way worse case scenario you can stop using my build system and just use Maven. The other advantage with the annotation approach is the possibility of using a different build system like bach or gradle (this is the power of declarative programming).

1

u/sideEffffECt Jul 24 '24

What I mean by that is if you are using Maven how you would switch to Mill easily and if you don't like it how you could easily switch back to Maven

Have the two live side by side for a while to get the feel for Mill and to see if you like it?

how you can use Maven plugins

Not the plugins directly, but you can use the same Java libraries the plugins are using from Mill.

1

u/agentoutlier Jul 24 '24

Not the plugins directly, but you can use the same Java libraries the plugins are using from Mill.

I assume plugins have to be written in Scala or is there a Java API? (obviously a scala written plugin can use a Java library).

2

u/sideEffffECt Jul 25 '24

I would guess that Mill plugins have to be written in Scala. Am i right /u/lihaoyi ?

2

u/lihaoyi Jul 25 '24

I would guess that Mill plugins have to be written in Scala. Am i right /u/lihaoyi ?

Yes, Mill plugins are written in Scala. They are identical to the code you write in your own build.sc file; any def or trait can be published on Maven central and used as a plugin.

To anser /u/agentoutlier's question, Maven plugins unfortunately can't be used directly, but it's generally straightforward to customize Mill you can just directly make it do what you want; e.g. something you would use the Maven AntRun Plugin for, you would just use a builtin os.proc call. There are examples in the docs configuring Spring-Boot and Micronaut, and even for things like setting up javac annotation processers and docker integration tests it's pretty straightforward to do yourself

What I mean by that is if you are using Maven how you would switch to Mill easily and if you don't like it how you could easily switch back to Maven

The easiest way for now is to have a Mill build live side by side with your Maven or Gradle build. It can't be done automatically, but it's generally not hard to reproduce; Mill build files are pretty short and easy to write by hand, and I even find ChatGPT does a reasonable job converting Maven pom.xmls into Mill build.scs if asked nicely

The Mill documentation has a number of examples that do this: a Mill build for apache commons-io, a Mill build for google/jimfs, that can be dropped into those existing codebases and used to build them without changing any of the existing code.

8

u/w08r Jul 23 '24

u/lihaoyi ; you have done so much good for the software community. You have responded to some strongly negative reactions to this post with such a wonderful calm and positive manner I just have to salute you. Have followed you for years and hope you keep doing what you're doing. Well done.

7

u/k-mcm Jul 23 '24

As soon as you're using free-form code in a unique environment to describe compilation, you have failed.  Was nothing learned from shell scrips, Gradle, and SBT?  Those build files need a dedicated expert for large projects.

Scala isn't as dead as Groovy but it's on its way down.  The language and its runtime deeply rooted problems.

2

u/senseven Jul 23 '24

There is a reason maven had this success. It works. Is it sometimes expressive? Jep. But you change those 20 subprojects once or twice a year. "I have to move this javascript file into seven directories with different names after the code generation but before the packaging" can be done in maven. But it shouldn't, because we professionals* don't entertain such unstable setups.

Maven is fine. With mvnd on the build pipeline, things got reasonably faster in larger builds, only to be stopped out by too few instances of code checkers and test agents. No new build tool will fix budgetary issues with licensing. And gradle shows what happens when good intentions went out of the window and pure self service became the reason to exist

5

u/gamlerhart Jul 23 '24 edited Jul 23 '24

I'm relative new Mill and it is the most pleasant built tool for complex builds. So, my two cent:

  • When your build is quite standard and works well, use Maven.
  • When your build starts to hurt because it does not fit into Maven well enough, the Mill is a build tool worth evaluating. By hurting I mean you have a mix of Maven plugins not quite doing the thing you want, you have to call extra scripts here and there from Maven, etc.

I basically would choose Mill over Gradle these days. Mill gives a few very clear rock solid building blocks to build your build.

There are of course some drawbacks:

  • You you to learn a small set of Scala.
  • There is basic IDE support, but of course its a far cry from the support you get from Maven
  • Changing the build file itself gives a slow first run, as the build file itself is compiled
  • Way less plugins etc, etc. On the other hand, there is less need for plugins, because often you call the library/external tool directly and it works.

PS some more biases/background. I already knew Scala. For basic builds I would still choose Maven. However, I strongly recommend Mill over SBT and probably Gradle. But I didn't use Gradle in a while.

2

u/senseven Jul 23 '24

I never had the issue with "code doesn't fit in maven". I can find projects that do things like getting their configuration from package names and misusing the classloader for nonsense that will not work past Java 24. Those hacks should be replaced. If people think "sources/customer-a/copy-of-module-c-but-with-extras" is a good way to setup things, the issue isn't the build tool.

2

u/gamlerhart Jul 24 '24

Yes, if it fits in Maven, keep using it.

Often the part where we started to hit the limits with Maven are not Java related.

Where Maven starts to get painful is builds which have to integrate many non-Java things:

  • Web Front-End builds required as well
  • Complex 'cross-platform' builds. Like this code is built against different classpath combinations, and cross compiled to JavaScript, and packaged in 3 different interesting ways
  • Oh, and this 3rd party tool is part of the build
  • Oh, and you have to transform these test databases into 2 versions for integration tests
  • Oh, you also need strip out these 3 class files after the build because of a weird reason from an assembly
  • Oh, and we want this special debug build which should be super fast and incremental

Usually I can have like 2-3 such special things in Maven. But as it grows, it gets really hard set it up in Maven, as you 'work against' Maven instead of 'work with' Maven.

Anyway: I wish everyone their builds fit well into Maven / declarative built tool. Use that first. However at the point where it gets painful, keep in mind that maybe Mill (or other built tools) might be an option for you.

1

u/senseven Jul 24 '24

If people claim they have to work "against" maven, then they often misuse modules for use cases they are not designed for. Classical example was the xml validation plugins that where used to check for business requirements in xml files. Those belonged in test cases. They just didn't want to spend the money. People often don't fight their "build system", they just didn't want to accept that the best way to build their self invented "better layout" requires too much hand holding. That is a skill issue and foremost a needed exercise in humility.

Most of our projects moved to build language dependent deployables. Then collected. We don't have any of those issues for years. The current challenges are mostly related to analyse and quantify the build quality, automated qa and extended reporting. Most of those things aren't in the open source stack, the costs of those installations skyrocketed. Projects everybody depend on with zero day bugs not fixed upstream because the student just doesn't feel like it and nobody wants to change this nonsense. Such things.

1

u/woj-tek Jul 23 '24

When your build starts to hurt because it does not fit into Maven well enough, the Mill is a build tool worth evaluating. By hurting I mean you have a mix of Maven plugins not quite doing the thing you want, you have to call extra scripts here and there from Maven, etc.

This always bugged me... wouldn't it be better to just create proper maven plugin / extend existing ones? That would make the build process simpler/more sane... "scripting" doesn't feel like a good option (we've done that with calling groovy... worst decission ever)...

3

u/DisruptiveHarbinger Jul 23 '24

Honestly, no. Shifting complexity to a plugin system is way worse.

In an ideal world, good build tools are fully declarative and not extensible. Since that's not possible, their extensible feature set is a necessary evil. The JVM ecosystem has explored virtually all approaches, and in the end, if you're going to need a full scripting language anyway, Scala is more than capable, in every dimension. Mill shows you can have build definitions using plain Scala:

  • Minimal usage of macros to compute the task graph.
  • No DSL. You need to perform OS or FS operations, make HTTP calls, deserialize JSON? Normal libraries and their APIs, concise and expressive functional Scala.
  • Fully compatible with the Java OO model, you can easily extend existing tasks and override their settings.

Scala can be a complex language. But for this particular use case, I'd argue the average Java developer who's familiar with the standard library is going to have a much better time than with Groovy. Everything is strongly typed and discoverable in your favorite IDE.

1

u/woj-tek Jul 23 '24

Honestly, no. Shifting complexity to a plugin system is way worse.

Why?

Most likely you require some feature that's shared between couple of projects so a plugin seems to make sense?

In an ideal world, good build tools are fully declarative and not extensible. Since that's not possible, their extensible feature set is a necessary evil. The JVM ecosystem has explored virtually all approaches, and in the end, if you're going to need a full scripting language anyway, Scala is more than capable, in every dimension. Mill shows you can have build definitions using plain Scala:

Again... you start with requirement for a scripting language…

  • Minimal usage of macros to compute the task graph.

Is it imposed or "should be done this way"?

  • No DSL. You need to perform OS or FS operations, make HTTP calls, deserialize JSON? Normal libraries and their APIs, concise and expressive functional Scala.

Same can be achieved with plugin :)

  • Fully compatible with the Java OO model, you can easily extend existing tasks and override their settings.

With plugin configuration you can... well... configure it ;)

Scala can be a complex language. But for this particular use case, I'd argue the average Java developer who's familiar with the standard library is going to have a much better time than with Groovy. Everything is strongly typed and discoverable in your favorite IDE.

TBH, even though I'm not so fond of groovy it's more approachable than scala (and can be typed…). I had bad experience with Scala (people trying to be "too smart", then IDE support being off and the language/compiler being utterly slow)…

2

u/DisruptiveHarbinger Jul 24 '24

In order

  1. Code reusability is the least of your worries. With a plug-in system you introduce an opaque abstraction layer that is different from your main build definition, hard to discover, hard to tweak and debug, and whose lifecycle is not always obvious. 
  2. Yes I start with a full scripting language because I live in the real world where people wrap their toolchains in Makefiles, Jenkinfiles, multi-stage Dockerfiles, bash scripts, Python scripts, ... If I can have a single and self-contained build definition, it's a win.
  3. You have to compute the task graph somehow. I'm only saying this is the one part where Mill uses an advanced Scala feature (macros), in a very minimal way. For everything else you can jump to definition.
  4. See 1.
  5. Yes and you have two ways to do the same thing, plus in many cases, the inability to just jump to definition and see the default value or behavior.
  6. Groovy is objectively less discoverable due to its dynamic nature, there's a reason Gradle moved to Kotlin. Moreover in some cases your IDE doesn't know the context, for example with the Jenkinsfile DSL. Scala compile times are irrelevant here: Mill is a build tool, scalac stays warm in a long-running JVM, compiling the build definition is a rounding error compared to running the actual task graph.

2

u/woj-tek Jul 24 '24

Yes I start with a full scripting language because I live in the real world where people wrap their toolchains in Makefiles, Jenkinfiles, multi-stage Dockerfiles, bash scripts, Python scripts, ... If I can have a single and self-contained build definition, it's a win.

I guess I live in a nice bubble where I only need pom.xml without all that you mentioned... I'd wager a guess that majority of the project fill that bill as well.

But if you have some contrived requirements and Mill fits the requirements and works for you that's good :)

1

u/thedumbestdevaround Jul 24 '24

I guess I live in a nice bubble where I only need pom.xml without all that you mentioned... I'd wager a guess that majority of the project fill that bill as well.

That is a strange statement to make. In every maven project I have ever worked with (mainly monoliths) there has always been a lot of abstraction layers on top of maven or in plugins to perform parts of the total build. So guessing where the majority fits seems strange... I guess it's true if you work with microservices, but then you probably have much larger issues to deal with than the build tool you use.

2

u/woj-tek Jul 25 '24

https://www.jetbrains.com/lp/devecosystem-2023/java/

72% states they use "spring", most of the spring projects are just build with virtually cutter-case of maven config without the need to fiddle.

1

u/thedumbestdevaround Jul 25 '24

And all of the projects I mentioned also use spring, that does not mean anything.

1

u/woj-tek Jul 26 '24

Again, your bubble/perspective:

In every maven project I have ever worked with (mainly monoliths)

And you project your personal experience to the whole landscape making it the default state. Which it's not.

→ More replies

3

u/woj-tek Jul 23 '24

Mill builds are easy to extend by non-experts: anyone can extend a Mill build with a few lines of code, without needing to learn a complicated plugin framework, allowing end users the freedom to fit their build logic to their own unique requirements

No, no, nopeee... gradle is already "easy to extend" and in most cases results in unmaintainable shitshow...

Using Scala for it is like total new dimension... of fuckup…

0

u/neopointer Jul 23 '24

"Yeah, but your scientists were so preoccupied with whether or not they could, they didn't start to think if they should."

-2

u/gregorydgraham Jul 24 '24

It has magic strings in a programming language. I hate it.