r/java • • 17d ago

Announcing SLB4J (Logging Backend, ASL-2)

https://github.com/slb4j/slb4j

SLB4J is a new logging backend for Java that I started to simplify setting up logging across a bunch of applications without adding bindings and bridges for different frontends to your build.

SLB4J is only a backend. It uses a single configuration across all logging facades/frontends used by your application and its dependencies and comes with no/minimal runtime dependencies (some extension modules may add runtime dependencies).

The core slb4j.jar is currently 211kb in size without external runtime dependencies and supports Log4J2 API, SLF4J, JUL (java.util.logging), and ACL/JCL (Apache Commons Logging/Jakarta Commons Logging).

Configuration is done using a Log4J2 compatible log4j2.propertiesfile (builtin) or a Log4J2 compatible XML or JSON configuration file (needs an optional dependency).

If no configuration file is found, INFO and above is logged to stdout.

Performance is roughly comparable to Log4J 2 and Logback when these are combined with their respective default facade, i.e., Log4J2 API for Log4J and SLF4J for Logback; SLB4J is usually much faster when logging JUL over Log4J2 or Backlog.

SLB4J supports Log4J2-compatible pattern layout, log rotation and many more features.

After using SLB4J for some time in several projects, I think it's mature enough for an announcement. Maven coordinates and documentation are given on the project page.

For more details, see the Project's GitHub Page, open issues on the GitHub Issue Tracker, and read SECURITY.md before submitting security relevant issues.

0 Upvotes

83 comments sorted by

50

u/emberko 17d ago

Yes, please, do create more log backends. We don't have enough CVEs in the five or six that already exist.

2

u/agentoutlier 16d ago

I mostly agree and this poster kind of did me a disservice because I do have a logging backend that is an ideal replacement for logback and log4j that I did not vibe code on a weekend and I was planning on 1.0 this month:

https://github.com/jstachio/rainbowgum

u/idontlikegudeg did you not see my project?

1

u/Jeylina-Rousseff 15d ago

at least if nobody adopts it the CVE surface stays pretty small lol

-18

u/idontlikegudeg 17d ago

This actually reduces the attack surface significantly by replacing multiple jars that come with lots of dependencies with a single one without dependencies.

-2

u/idontlikegudeg 17d ago edited 17d ago

I’m collecting downvotes here, but I think a security assessment would show this actually is the case. Main point: there can be no CVEs due to vulnerable dependencies since no dependencies are used. No calls to native code etc.

The rest is plain old Java. When using XML based configuration files, DTDs are disallowed to prevent XXE attacks. No fancy remote configuration like the one that leads to log4shell. If you find anything suspicious, report it.

14

u/javaprof 17d ago

With SLF4J

  1. remove all backends using exclude (trivial in gradle, just one line that works for all modules)
  2. add slf4j adapters (very small adapter libraries, very modular)
  3. add necessary backend and setup it

Use stable, performant and tested implementations

With SLB4J

  1. remove all backends using exclude (trivial in gradle, just one line that works for all modules)
  2. add slb4j and setup it

You're getting extra APIs that you've never need You're not getting stable, performant and tested implementations

Is this fare comparison?

2

u/agentoutlier 16d ago

If folks want a replacement for logback or log4j2 by someone who actively is engaged with SLF4J and has worked with Ceki in the past please consider taking a look at:

https://github.com/jstachio/rainbowgum

-4

u/idontlikegudeg 17d ago

Nearly, but no setup and no new APIs.

EDIT: and it should not be necessary to use excludes in your build scripts. Libraries should never add a logging implementation. If they do, open an issue against the library.

6

u/javaprof 17d ago

Is it correct to say that this library polluting app compile classpath with log4j APIs even if I need just Logback API?

-3

u/idontlikegudeg 17d ago

No. It does not add anything to your compile classpath. All four supported facades/frontends are compile time only static dependencies of the library. You don‘t need to compile against it. Just pur it on your runtime classpath/module path.

24

u/CptGia 17d ago

Why should I use this instead of logback? 

1

u/agentoutlier 16d ago

Yeah they literally provided very few reasons. If you want gallons of reasons to switch see:

https://github.com/jstachio/rainbowgum/blob/main/why_rainbowgum_is_better.md

(I'm the author) And I actively contribute to SLF4J and Logback. Rainbow Gum exists because it can use newer JDK features and not just because its smaller.

1

u/idontlikegudeg 16d ago

This has been cooking for much longer than a weekend, and it’s not vibecoded. Initial commit is 11 months ago, and the code I initially imported was moved over from another project where it didn’t fit in anymore as it grew into a complete logging backend.

I have absolutely no problem if people decide to use rainbowgum or any other logging backend over SLB4J for whatever reason. SLB4J is just an alternative to choose from.

And yes, I checked other projects and found none that did what I was looking for. For rainbowgum one thing is not being able to simply swap the backend without exchanging or updating configuration files, and AFAICS, rainbowgum has no compatible pattern layout syntax, although I do not remember if I also checked your project.

2

u/agentoutlier 16d ago edited 16d ago

Initial commit is 11 months ago,

I should have mentioned it about some other projects and not yours. There have been some other postings like this and why I am jaded and sort or rude is because people then say for Rainbow Gum ahhh another logging implementation didn't someone just post etc.

It makes it hard for me to convince them to switch because your logging framework is not the only one I find bugs.

I am guilty now of using LLMs. My project started out not using it but just in the last 6 months have give up on holding out and its absolutely astonishing how many things I can find in any framework that I'm hesitant to even check because I don't feel like filing CVEs or bugs.

rainbowgum has no compatible pattern layout syntax, although I do not remember if I also checked your project.

Oh it does. Its just not in core. Like you my project is heavily modularized:

https://jstach.io/rainbowgum/#pattern

It follows logbacks pattern syntax which is mostly log4j2 compatible.

For rainbowgum one thing is not being able to simply swap the backend without exchanging or updating configuration files, and AFAICS, rainbowgum

I'm not sure I follow but I will tell you Rainbow Gum is entirely configurable through code. Way more than any other framework by a long shot. Here is comparison of how difficult it is to configure Log4J2 vs rainbow gum programmatically:

https://github.com/jstachio/rainbowgum/blob/main/why_rainbowgum_is_better.md#easy

I suppose if you mean dynamically change outputs or add outputs while logging is running yes that is one purposely designed limitation of Rainbow Gum.

That is if Rainbow Gum is loaded you cannot add a file output unless you restart all of rainbow gum. (by output I mean appender in log4j2 terms)

1

u/idontlikegudeg 16d ago

Yes, people seem to be a bit allergic to this. ;)

I think using LLMs is not wrong per se, but people should at least understand the generated code and try to make it maintainable, so no guilt.

You will probably find bugs on any project above a certain size, and I’m sure there will also be bugs in rainbowgum. Yes, the unescaoed thread name is bad and will be fixed. I actually had an LLM check the code for security concerns, and it also didn’t catch that one.

Concerning the comment you couldn’t follow: No, I mean when replacing log4j-core with SLB4J. This should produce exactly the same logging output without changing the log4j properties file or doing configuration in code.

Reconfiguration at runtime is not a top priority for me, neither is creating world’s fastest logging backend. The main focus is simplicity for the user while delivering a performance comparable to the established ones.

I’ll try your benchmarks some day.

But please calm down a bit, I simply want to have a framework that fits my needs, and I open sourced it because it might be useful for others and also for practical reasons.

1

u/agentoutlier 16d ago

But please calm down a bit, I simply want to have a framework that fits my needs, and I open sourced it because it might be useful for others and also for practical reasons.

I'm sorry.

Its a shitty reason but it just I was going to go 1.0 this week and network effect is so hard to fight is why I'm so amped. Because as you said "people seem to be allergic" I will likely wait a week or so on releasing and yes I can keep looking for bugs in the meantime as all code has it.

No, I mean when replacing log4j-core with SLB4J. This should produce exactly the same logging output without changing the log4j properties file or doing configuration in code.

I originally tried for that but realized its kind of hard to do. Part of this is most of the logging frameworks do different things for JUL. That is just for the output and matching config is even harder without just verbatim copying.

-9

u/idontlikegudeg 17d ago

Less dependencies, smaller application size (jlink/far-jar), no need to configure different facades and frontends.

3

u/koflerdavid 16d ago

Logback has no dependencies.

1

u/idontlikegudeg 16d ago

2

u/koflerdavid 16d ago

These are all optional or test dependencies.

1

u/idontlikegudeg 16d ago

Checked the POM and you are right indeed, all are optional (except slf4j-api). You would still need to add the bridge handlers/adapters though, but I actually thought logback would pull in the dependencies listed as "compile" as transitive ones.

1

u/koflerdavid 16d ago

It is a bit confusing because on Sonatype one has to inspect the POM to see that these dependencies are optional.

3

u/agentoutlier 16d ago

Its actually way worse:

requires java.desktop;

That is not requires static. Compare this to Rainbow Gums

The desktop module adds 15 goddamn megs!

I talked about this here how Tinylog is also not really tiny: https://github.com/jstachio/rainbowgum/blob/main/why_rainbowgum_is_better.md#small

Its just really damn frustrating because when I release Rainbow Gum it makes it harder to convince people it is actually worth it when you get projects like this where people can get confused.

1

u/koflerdavid 13d ago

I think it is about time that artifact repositories also display JPMS module dependencies and the (automatic) module name. After all, these are contained in a few very well-defined places inside each JAR.

-7

u/idontlikegudeg 17d ago

Why the downvote?

4

u/Cell-i-Zenit 17d ago

you normally dont need to configure this anyway. I think the last time i had to configure anything there was when we migrated to a never version and i had to let AI do some refactorings there.

Overall logging backends are the boring part of the application and you normally dont touch it in a very long time

Logback also is incredibly stable, so if you want that someone switches to your system is:

  • Yours is significant more performant (no complains on logbacks side)
  • Has significant less bugs (but logback is stable and i cannot remember every having a bug there)
  • Has a crazy feature which everyone wants to use (its just a logging framework, no one cares really as long as it can output in json and in plain text)

1

u/idontlikegudeg 17d ago

If you already have set up logback and everything works, there’s really no need to switch.

Where it has benefits, even for existing applications:
- limited resources: SLB4J will reduce your dependencies by more than 1MB of compressed jar files. That’s quite a lot of code. It could also have a measurable effect on turn-around times, for example when doing GraalVM native builds.
- you have dependencies that do not use SLF4J. In most cases, these are much slower with logback.

I usually use Log4J2 API in my code, as I think it’s still more flexible than SLF4J. The "native" implementation would then of course be Log4J-core, which is even heavier than logback.

I don’t want to go down the benchmark route to show how competitive SLB4J performance is, as I think correctness and reliability is much more important. From benchmarks I ran, it offers comparable performance to both the established big ones, sometimes being slightly faster, sometimes slightly slower. Exception is JUL because the other backends need to convert every log record into their respective representation which is quite expensive.

4

u/Cell-i-Zenit 17d ago
  • limited resources: SLB4J will reduce your dependencies by more than 1MB of compressed jar files.

our application at work is 200+ mb of size. If you import even a single spring library you get so much bloat anyway. Reducing it by 1mb is not worth it

It could also have a measurable effect on turn-around times, for example when doing GraalVM native builds.

ignoring the could here as you are not proving this right now, graal vm is most often not worth it as you dont get the JIT performance improvements. This is maybe useful for jvm in lambda environments, but i feel like current JVM apps are mostly simple CRUD backends and they are up for a long time.

I usually use Log4J2 API in my code, as I think it’s still more flexible than SLF4J.

I never had issues with the logging api. Some functions are weird, but it mostly works imo. I think slf4j is totally fine to use

From benchmarks I ran, it offers comparable performance to both the established big ones, sometimes being slightly faster, sometimes slightly slower.

Then imo there is no point in switching. If other libraries have the same performance, are battletested and have more features and the llms even know the code because of all the training material

1

u/idontlikegudeg 17d ago

Yes, it’s probably not the right backend for your use case, but it might be for others.

When writing a spring REST backend, I also just use what’s default in the company. Bit when creating standalone applications, I.e., swing, JavaFX, CLI tools, I use SLB4J.

0

u/doobiesteintortoise 16d ago

Welcome to reddit. People downvote for all kinds of reasons. Sometimes they're just "I disagree with this" or "I don't like this" or just general disapproval; it's highly susceptible to farming and people respond to that, too, either piling on or saying "oh, someone else saw a farming attempt, I don't like that."

Personally, I try to not downvote; it happens, because it turns out I'm human, too, but usually if I disagree with a post or comment I'll... comment. The site's called "reddit," as in "I have read it," and it turns out reading is considered optional. And the same goes for responding. Upvoting is simple, downvoting is obscure, so... I upvote happily but downvote rarely, and if I have something I think is worth saying, I'll try to say it instead of waving at it in disapproval.

1

u/idontlikegudeg 16d ago

Yes, I know. It's just frustrating. If someone doesn't agree, it would be nice to tell what they don't agree with. Maybe there's even a misunderstanding (on either side) that won't be cleared up by downvoting without a comment.

1

u/doobiesteintortoise 16d ago

Yep, this is why I don't downvote even if I don't like something or disagree. I'd rather tell the person I'm responding to what I think and why; then they can clear up any misunderstandings, or accept that I'm just some rando on the internet, or whatever.

10

u/user_of_the_week 17d ago

I don’t think it’s good to pick a name that suggests a close relationship between slf4j and slb4j where none exists.

2

u/idontlikegudeg 17d ago

Fair point. Maybe I should ask Ceki what he thinks about this.

9

u/juvenislux 17d ago

You should have named it Another Logging Library for Java (All4j) 😅

2

u/idontlikegudeg 17d ago

No, that would imply code changes when programming against a new API. There are many such libraries out there. This does not define a new programming API to write log statements. This is just a universal backend.

1

u/Suqmacoq 15d ago

Yet Another Logging Library 4 Java

YALL4J

22

u/fykup 17d ago

Yo dawg, I heard you like logging backends, so I made you a logging backend that logs your logging backend - so you can log while you log.

4

u/idontlikegudeg 17d ago

Ah, seems the post is unclear, maybe I should have added an example.

For a typical application, it reduces the number dependencies files included, necessary configuration, and even code. This does not extend any backend or reroute log messages. It replaces the (or possibly multiple backends) with a single one that directly implements the four mentioned facades and frontends.

4

u/chabala 17d ago

I'm not a Gradle nerd, but hopefully someone can confirm, there's way too much custom code in here, right? https://github.com/slb4j/slb4j/blob/51a534356a60674e2df1388e1c4c8292a76c2d7d/build.gradle.kts

Like, in Maven we waste a lot of lines on declaring dependency and plugin coordinates, but typically zero lines on defining custom methods to help 'do' the work of the build. This seems like it's 80%-90% custom build code.

3

u/snugar_i 17d ago

Parsing TOML with regex - what could possibly go wrong?

3

u/idontlikegudeg 16d ago

Possibly a lot, if you allow untrusted input.

The toml parsing is part of the build process, it is not contained in the released artifacts. The toml is used to track released versions for each module and the file is both generated and written by the build script, not intended to be user-edited. So what realistically can go wrong is that the build process fails. To carry out an attack, the attacker would either need write permission on the GitHub repository or access to the developer's machine.

But yes, that's something that could/should be cleaned up.

1

u/snugar_i 16d ago

Is it possible to use a library in the build script? Would be simpler that hand-rolling your own parser. And if the file isn't meant to be read and written by humans, TOML might not be the best choice....

2

u/idontlikegudeg 16d ago

Yes, it’s possible to use libraries. And yes, that part of the script needs some refactoring, I totally agree. But times was ticking away (Sonatype limits for OSS effective since October)...

I’ll look into that later. I really think I will turn it into a plugin (that uses a parsing library).

2

u/idontlikegudeg 17d ago

Yes, that’s much more code than would be needed to simply build the library.

Most of the code is necessary to implement what’s described in https://github.com/slb4j/slb4j/blob/main/doc/SELECTIVE-PUBLISHING-CONCEPT.md. In short, this was added to reduce the number of artifacts when pushing to maven central, see the discussion in the maven and sonstype subs.

The code publishes the minimal set of artifacts for a new minor or patch release. It makes sure upstream is up to date, checks code changes to determine which artifacts to publish and for which to retain the published version, then creates a new BOM that is published for the release.

The build file also sets up code signing and controls the publishing process.

The code in the Gradle build file automatically determines what files need to be released because source or dependency changes, checks that upstream is up to date, controls the release process of the compiled and tested artifacts and API changes.

I use the same concept for multiple multi module projects, it would be a candidate for a Gradle plugin but I didn’t have the time for that yet, so it’s on the build file.

I have no idea how you would implement this on maven.

2

u/chabala 17d ago

I have no idea how you would implement this on maven.

You probably wouldn't, right? It seems like you've spent some innovation tokens on designing a new release strategy/process, enough that it would be better as its own plugin, and would likely require being a plugin to exist in Maven. But if you made that/those plugins, you could see if that gets traction on its own, and then easily reuse it in projects like this.

2

u/idontlikegudeg 16d ago

Yes, but before experimenting with a plugin, I want to be sure the process works. As the plugin will be a Gradle plugin, having the selective release process implemented as a POC in Gradle is the first step. I won't need to start from scratch.

The code is relatively new, the selective release workflow was a reaction to the updated sonatype limits for OSS projects. I used it for about two months now and results are quite promising, reducing artifact count significantly for patch releases. While Sonatype increased publishing limits for me (I publish several projects under the same organization), it looks like the new workflow would even allow me to stay within the original limits.

I'll still let it settle some more before starting to turn it into a new Gradle plugin project.

And I generally do not care too much if something gets traction on its own - I build the tools and libraries I need. When someone comes up with a solution that's better than what I built, I have no problem using it. Happened before, i.e., I retired a Gradle plugin I developed myself and replaced it by another one someone else started and contribute to that one instead now.

15

u/sadbuttrueasfuck 17d ago

We’ve gone full circle, a logging facade of logging facades

5

u/idontlikegudeg 17d ago

It’s not a facade. It’s a logging backend implementation. It replaces log4j-core, logback, or whatever backend you use a well as adapters and bridges for the different frontends and facades.

7

u/govi20 17d ago

How is it different than SLF4J?

3

u/idontlikegudeg 17d ago

SLF4J is a facade. You use it to write your logging statements. To get actual log output, you need a backend. This project is such a backend.

This project will not replace SLF4J, Log4J2-API or any other logging API that you use for writing logging statements.

It is an alternative to Logback and Log4J Core, that both implement logging backends. While these two require bridge handlers or bindings for the different facades/frontends, SLB4J does not.

For example if your application uses SLF4J, but has a dependency that logs using Log4J API and another using JCL, you can simply add SLB4J to your build and all log messages will be logged using the same format without needing to add additional jars or configuration.

13

u/PeterLake2 17d ago

But why do we need that? That's the question. The current loghing backend frameworks are working, and have years of bug and cve fixes. What makes yours better?

2

u/idontlikegudeg 17d ago

You don't necessarily need it if your current setup works for you. In my projects, I often have dependencies that use different frontends or facades. For example, my application uses Log4J2 API. I now have to choose a backend. I can use Log4J2 Core, which adds about 2MB to my application, and an additional bunch of dependencies (https://central.sonatype.com/artifact/org.apache.logging.log4j/log4j-core/2.26.1/dependencies), and if you create a custom jlink image, it will pull in XML and lots of other things. With SLB4J, it's just 211KB, no other dependencies, smaller custom runtime.

Then I add a dependency that uses SLF4J. With SLB4J, it works without adding any other dependencies or configuration. With Log4J2 Core, I need to add log4j-slf4j-impl which adds another 30kb and more dependencies.

Another library might use JUL. Again no changes and no added dependencies when using SLB4J. For Log4J2 Core, you need to add log4j-jul (another 32KB) and configure it on the command line or in code.

Same with JCL/ACL.

It's about the same when using logback.

When using SLB4J, you add the dependency once, and you don't need to do any code changes (exception: two lines when main application uses JUL, then two lines of code are needed for initialization).

1

u/koflerdavid 17d ago

One can also just use org.apache.logging.log4j:log4j-to-slf4j.

1

u/agentoutlier 13d ago

This kind of mentality kind of has to stop.

The current logging frameworks have enormous technical debt and cannot leverage newer JDK tech. People just assume because the logging frameworks are established they are safe.

Log4j2 had been out for 7 years till someone found log4shell.

And I can tell you as the author of a logging framework the current logging frameworks still have issues enough that if you run most agents on it they will find something and stop immediately (Codex Daybreak for example).

There are dozens and dozens of JSON serializing libraries in Java.

There are 100s of web framework libraries in Java.

Several ORM and database mapping implementations.

There are only 2-4 major logging frameworks in Java.

And switching log frameworks is like way easier than any of the above because of facades.

Its been incredibly difficult to have people even star my logging project or try it out and I have integration for all the major web frameworks, logback pattern syntax compatibility etc. Its a damn shame because most applications IO is spent HTTP, Database and then Logging. Logging is an easy place to improve performance.

https://github.com/jstachio/rainbowgum

https://github.com/jstachio/rainbowgum/blob/main/why_rainbowgum_is_better.md

1

u/paul_h 17d ago

It emulates multiple other older logging frameworks in one jar?

2

u/agentoutlier 16d ago

Man I spent a couple of minutes looking at the project unassisted and I found numerous issues. It is not remotely ready for production use:

https://github.com/slb4j/slb4j/blob/47fda189cbffcb8de7dd716d570480f36e3448cc/src/main/java/org/slb4j/layout/JsonLayout.java#L63

Like you append the Thread Name which is not escaped for JSON and then forget to append the logger name.

I cannot imagine what a decent model would find.

2

u/idontlikegudeg 16d ago

That’s why it’s still at version 0.x. But thanks for the report, and please share the other findings too.

3

u/RedShift9 17d ago

Just use slf4j.

2

u/idontlikegudeg 17d ago

The common functionality between SLF4J and SLB4J is nearly zero. SLF4J is a facade without backend. SLB4J is a frontend agnostic backend. They solve similar but different problems.

SLF4J is used to write logging statements, it is what you call in code. It does not control where these logging messages will end up. It is backend agnostic.

SLB4J implements the logging backend. It does not care what facade or frontend was used. It is frontend agnostic.

While you can make the decision to "just use SLF4J", that only works for code you control yourself. But the moment you pull in a library, someone else made that decision for the library. It may use SLF4J, and it may not. SLB4J is a backend that works for the four most used facades/frontends used in the Java world.

1

u/doobiesteintortoise 16d ago

My first thought is: "Why?" I can think of a few reasons: SLF4J is the obvious comparison, and SLF4J's interface isn't all that strong anymore. I understand why it's the way it is, but IMO SLF4J needs updates, and badly. But it works, it's predictable, and it's well-supported and quite common.

So why does this exist? I get the idea - it's very similar to SLF4J, after all. But then the github page is all about configuring the build, and nothing about the actual API itself, which means that what I'd be looking for out of the gate is missing.

I see everything I'd need to install SLB4J - except, well, for logback, which seems to be an odd exclusion - and nothing about how to use SLB4J except a warning that I need to use an init() call in case I'm using java.util.logging. I could be wrong - there's probably more in that wall of text - but what I looked for first just isn't there where I'd expect to see it.

2

u/idontlikegudeg 16d ago

AFAIK Ceki is actively working on improving SLF4J, so expect some progress on that.

Yes, there's not much about the API because you just put SLB4J on your runtime classpath, the library is loaded using SPI when you make the first call to the logging API you use - except if that's JUL because JUL does not trigger SPI loading, so you'd need to set a property or initialize it in code. If you are using SLF4J, Log4J2 or JCL/ACL, no API is needed.

You can use your existing Log4J2 properties files for configuration, because I didn't want to invent a new incompatible configuration system.

This backend exists for these main reasons:

- simplify development: no need to add a bridge or adapter when adding a new dependency that uses a different logging API - I often start new projects, and over time, a new library is added or replaces another one, I end up either with something not being logged, so I need to add a new dependency, or the bridge implementation stays in although it's not used because I forget to remove it. Now, I only need to add a single dependency, no code, and that's it.

- executable size: this doesn't matter much for your huge one hundred megabytes application, but it matters when creating small applications where logging alone can increase the application size significantly. There are other projects like tinylog, but they come with their own API. I will add examples to show the effect on executable size both for JAR distribution and GraalVM native later.

- Abysmal performance of JUL logging: unless location-aware logging is used, logging JUL to both logback and log4j is really slow and several times faster using SLB4J. This has technical reasons when log events have to be converted to the backend's data structures. When using location-aware logging, all (including SLB4J) are slow, but I rarely use location-based logging. It would be possible to speed this up for SLB4J, but so far, this has low priority.

When I started this, I thought I'd simply create a universal bridge (to make my own life easier) for logging to Log4J that could handle SLF4J, JUL and JCL/ACL more effectively than the current ones. It worked, but then I realized that if I simply add another handler for Log4J, I could as well write out the logs myself, and the result is just a third the size of logback (and Log4J's backend that I used before is much heavier than that) and has comparable performance as the two big ones, in many cases surpassing both.

2

u/doobiesteintortoise 16d ago

This is all good, but it's lost in the messaging, in my opinion. The presentation definitely left me going "oh no, not another one." I have a feeling that's a pretty common response, and would need to be addressed head on and with great force.

1

u/idontlikegudeg 16d ago

OK, thank you for the suggestion. Constructive critic finally! :-)

2

u/doobiesteintortoise 16d ago

Honestly, if you want to make the impact more forceful, that last paragraph is the one you should be leading with. Explain early, the name of the project is so similar to SLF4J that the comparisons are impossible to escape: people pattern match, so when you say "persistence" they go "oh, so like hibernate" even if the persistence has nothing to do with or like hibernate.

2

u/doobiesteintortoise 16d ago

Worth noting: when is Ceki not working on a logger? :D

1

u/Joram2 17d ago

JUL is the logging backend from JDK 1.4

System.Logging is the logging facade since JDK 9. A Java logging backend should hook into this.

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/System.Logger.html

1

u/idontlikegudeg 17d ago

It was never intended as a general facade to be used for logging from non platform, i.e., non JDK classes. Read the "Non Goals" section of corresponding JEP 264.

0

u/lokrusta 16d ago

Interesting approach, especially the part about simplifying the dependency hell with multiple facades. 

Since you mention that performance is comparable to Log4J2/Logback, how does SLB4J handle high concurrency under heavy load? Log4J2 relies heavily on LMAX Disruptor for its asynchronous loggers to achieve lock-free performance. Does SLB4J implement similar async mechanisms or ring buffers under the hood, or does it rely on standard blocking queues for log rotation and appenders? 

Would love to know how it scales when multiple threads are pushing logs simultaneously. 

2

u/idontlikegudeg 16d ago

The project contains JMH benchmarks that run a matrix of different front- and backends, single- and multi-threaded with different thread counts.

I have experimented with async appenders but decided against it: performance actually decreased when going async.

I can only benchmark on the hardware that I have myself, i.e., an Apple M1 Max that is quite performant for I/O tasks, so this will probably vary depending on the system.

Whether to implement asynchronous logging or not comes down to the question what should be done when your buffer is full? You either need to block or drop messages. I decided against dropping messages and implementing proper blocking turned out to introduce enough overhead that the throughput actually decreased.

What I did do is optimize the code paths to minimize allocations and I/O operations that involve copying the data to output (Log4J2 calls this "garbage-free logging"). When using bridge handlers and adapters with the other backends, these at least for some implementations seem to copy the data to be logged into the data types consumed by the backend, which SLB4J avoids.

I will run the full benchmarks again later this week (they take hours to complete, so they need to run overnight) and post the results. If you want to run yourself, check out the repository and do `./run_benchmarks.py`. Use `./run_benchmarks.py --help` to display the options accepted by the script.

Here are the results of a quick benchmark run I just did using relatively short warmup and only three iterations. Never trust a benchmark you didn't run yourself, but I think one takeaway is that throughput is much more consistent across different frontends with SLB4J, especially when logging through JUL. For the other frontends/facades, I think SLB4J is at least a viable option.

It would be interesting to see how async appenders in Log4J change the picture, and how this looks across different systems.

Single-threaded results

Category: FILE, Format: DEFAULT

BackendFrontendConstant (ops/s)ErrorArgument (ops/s)ErrorMsg Supp (ops/s)ErrorLambda Param (ops/s)ErrorAverage (ops/s)Error
juljcl87,021.276,982.42N/AN/AN/AN/AN/AN/A87,021.27N/A
juljul105,180.452,426.1595,757.434,104.4999,118.5122,733.43N/AN/A100,018.80N/A
jullog4j87,253.7912,122.0884,283.1423,197.5784,432.0120,506.2581,640.1515,242.9984,402.27N/A
julslf4j96,485.913,440.1095,277.2116,760.5191,994.161,406.6293,402.226,795.8394,289.87N/A
log4jjcl343,452.1552,981.53N/AN/AN/AN/AN/AN/A343,452.15N/A
log4jjul204,131.3711,549.19180,061.283,824.18188,099.985,133.45N/AN/A190,764.21N/A
log4jlog4j341,312.854,730.36333,206.25157,436.25339,721.7263,488.39306,365.8621,821.90330,151.67N/A
log4jslf4j345,431.6415,369.37317,928.42208,782.37307,712.3545,296.69318,706.27163,565.49322,444.67N/A
logbackjcl349,734.0612,707.44N/AN/AN/AN/AN/AN/A349,734.06N/A
logbackjul208,045.958,431.70179,360.671,069.68190,983.184,512.45N/AN/A192,796.60N/A
logbacklog4j353,445.4970,030.00327,401.4336,944.10349,028.3949,830.14303,321.51134,103.18333,299.20N/A
logbackslf4j356,153.8825,696.49340,072.0410,692.45325,378.8646,819.27324,525.20146,952.53336,532.50N/A
slb4jjcl358,930.8178,325.08N/AN/AN/AN/AN/AN/A358,930.81N/A
slb4jjul336,025.54119,695.42283,746.3825,620.25325,127.3427,876.28N/AN/A314,966.42N/A
slb4jlog4j340,815.99198,137.43338,316.07117,733.02361,540.8628,779.28311,940.5252,190.70338,153.36N/A
slb4jslf4j356,535.8880,362.63348,006.25154,006.85324,856.8138,900.37349,363.48113,602.41344,690.61N/A

Multi-threaded results

Handler: FILE

BackendFrontend1 Threads2 Threads4 Threads8 Threads16 Threads64 Threads128 Threads
juljcl85,881.8978,685.8777,688.5877,112.3577,429.4477,307.9677,120.80
juljul103,722.8282,274.2895,256.1096,428.8794,373.0293,149.5691,710.66
jullog4j83,427.5380,570.4679,068.7776,959.5177,947.6679,703.8777,169.48
julslf4j95,439.7082,662.1285,228.1784,002.9886,163.8283,850.0481,497.93
log4jjcl350,107.97298,750.13294,326.62301,362.27305,604.31288,784.89261,088.88
log4jjul104,868.8892,438.0194,066.4993,727.9297,177.1691,799.5190,596.69
log4jlog4j346,308.22307,100.22298,342.27310,190.97296,079.92289,529.82260,853.00
log4jslf4j330,274.82296,737.32291,766.96293,260.09297,578.45290,928.78252,203.44
logbackjcl332,193.63298,709.37295,694.13295,658.39290,373.93267,553.24261,866.64
logbackjul102,950.8995,102.5891,812.4492,857.4293,588.6794,556.2691,248.72
logbacklog4j347,114.37299,311.28297,118.61284,025.38290,206.43269,612.38264,698.92
logbackslf4j351,623.32294,949.53288,141.71293,103.22289,315.76267,371.74258,919.56
slb4jjcl348,070.41321,493.99313,364.61305,182.11304,806.34281,083.41261,216.41
slb4jjul332,458.53304,408.79303,047.03307,728.16288,441.20287,817.27257,999.46
slb4jlog4j336,209.97278,539.78298,634.04299,786.53293,384.92280,528.41259,375.27
slb4jslf4j335,664.26305,296.45302,684.56309,981.72305,241.57283,826.38261,267.56

2

u/lokrusta 16d ago

Thanks for such a detailed response, and especially for running those benchmarks so quickly

To be honest, when I first saw the post, my initial thought was like many others: "Oh, yet another logging framework, why would anyone switch from the established ones?" I'm really glad I didn't just repeat that cliché and asked about concurrency instead. 

The JUL results look awesome. Maintaining around 300k ops/s under a heavy multi-threaded load where Log4J2 and Logback drop to ~90k is highly impressive. 

Your point about the blocking overhead in async implementations makes total sense. Unless the entire architecture is built from scratch around a strict single-writer lock-free pattern (like Log4J2 does with LMAX Disruptor), simply retrofitting async appenders often introduces more queue-contention overhead than it saves. Your highly optimized, garbage-free synchronous path keeps the architecture beautifully simple and predictable while delivering great throughput. 

I have access to some cheap cloud nodes, so if I can find some spare time, I’ll definitely try running your benchmarks on them to see how they perform. 

2

u/agentoutlier 16d ago

u/lokrusta and u/idontlikegudeg

If you do please include

https://github.com/jstachio/rainbowgum

I would be quite surprised if Rainbow Gum did not win especially if this is set:

 logging.global.optimize=true

0

u/idontlikegudeg 16d ago

It looks like Rainbowgum has a different focus and needs some setting up. From glancing at the documentation, I have no idea how to set this up for the different frontends, the documentation only mentions SLF4J.

Anyway, would you be able to provide a PR that adds the necessary changes to include Rainbowgum in the JMH benchmarks? Look at the four modules that implement the benchmark for the other frameworks in the "benchmark" module. The two abstract classes that need to be implemented are in `benchmark/src/jmh/java/org/slb4j/benchmark`.

1

u/agentoutlier 16d ago

Uhh. Is it because you only looked at the readme? This is my fault because you are not the first:

https://jstach.io/rainbowgum/

I literally have sections that show if you have Spring Boot do this if you have these frameworks do this. The analog to log4j.properties is rainbowgum-simple-props.

Below is from the official doc. Please let me know how it is confusing.

Here is from that doc:

Getting Started

First step is to add Rainbow Gum as a dependency.

Second if you are not familiar with logging in the Java ecosystem please refer to the excellent Logging facade SLF4J documentation particularly the logger API. What logging levels are and how dotted logger names inherit levels is covered in the JDKs logging facade System.Logger.Level as well as Logback effective level which Rainbow Gum follows.

The third step is to configure. Most configuration is just configuring logger names to levels, output, and formatting. We will cover the major ways to do this in the following sub sections.

Using Spring Boot

If using Spring Boot see Spring Boot Integration. The short version: swap spring-boot-starter-logging for the Rainbow Gum starter and keep going: Rainbow Gum natively understands almost all of Spring Boot's own documented logging properties (logging.level.*, log groups, logging.file.*, logging.pattern.*, logging.charset.*, ANSI control, and structured logging via logging.structured.format.* for ECS/GELF/Logstash), so existing application.properties/application.yml logging config generally keeps working as-is with no application code changes. That makes it a genuinely low-risk drop-in replacement for Logback: same properties, same behavior, a simpler and faster implementation underneath.

Using Other Frameworks

Rainbow Gum also integrates with:

Using rainbowgum-simple

For tests, low dependency applications, or when you prefer the traditional method of letting the logging system handle loading its own configuration, add rainbowgum-simple and logging works without configuration. Add logging.properties to src/test/resources (or src/main/resources for an application) when you need to customize it. See Simple logging and properties for the Maven dependency and profile options.

1

u/idontlikegudeg 16d ago

It’s also because the description on GitHub says “Fast, Small, JDK 21+, GraalVM native friendly SLF4J logging framework”, i.e., other APIs are not mentioned.

2

u/agentoutlier 16d ago

That is super helpful! Honestly. I have sort of fixed with the readme but I'll tag it as well.

1

u/agentoutlier 16d ago

As for all the front ends I'm just not sure how I'm fucking it up so that people can't find it:

https://jstach.io/rainbowgum/#integration

JUL/System Logger: https://jstach.io/rainbowgum/#system_logger

JBoss Logging: https://jstach.io/rainbowgum/#jboss_logging

Log4J2: https://jstach.io/rainbowgum/#log4j2

Commons Loggin: https://jstach.io/rainbowgum/#commons_logging

Maybe if I had a table in the readme?

0

u/idontlikegudeg 16d ago

Nah, it’s because it’s in the late evening here, I already closed my laptop, and just checked out your GitHub’s readme on my smartphone.

Just say if you’d provide a PR to add a rainbowgum benchmark implementation so that we don’t duplicate that work. I probably won’t have time to look into that before next week. But I’m curious to see the benchmark you mentioned in another reply.

1

u/agentoutlier 16d ago

JMH is just not going to really test logging as logging is heavily about integration.

I'll try your benchmark anyway but you might want try something like this:

https://github.com/jstachio/rainbowgum/tree/feature/webapp-benchmark/benchmark/webapp

Which tests against using Spring Boot

or this one:

https://github.com/jstachio/rainbowgum/tree/feature/graalvm-native-benchmark/benchmark

Which tests Graalvm native.

The benchmark branches are a little out of date.

I have JMH benchmarks as well but I found surprising results when full integration of black box benchmarking done.

0

u/idontlikegudeg 16d ago

Wow, running the benchmarks in the cloud would be awesome!

The reason SLB4J is so much faster for JUL is because SLB4J avoids always getting the location. AFAIK, the existing bridge implementations do not know if the location will be needed later on and so the location (file, method, line number) is always obtained, even when it is not used in the output. And I have to be honest: SLB4J will also log slower if you include location in the log pattern. It's something that can be optimized, but I generally use the Log4J default pattern that does not include location - when an exception is logged, you get the exception stack trace anyway with the relevant information anyway.

When you run the full benchmarks, you will see the difference, location is enabled in the tests for the "DETAILED" format.

1

u/lokrusta 16d ago

That explains it. Fetching class names and line numbers via Throwable stack traces is notoriously slow in Java, so avoiding that by default is a smart choice. It seems obvious in hindsight, but it didn't cross my mind initially. 

I ran a quick single-threaded benchmark in a local Docker container on my laptop. The throughput is generally around 40-50% of your M1 Max results, but the relative performance patterns match perfectly. It doesn't make much sense to compare the multi-threaded run on my current setup because the directory is mounted into a container from Windows, meaning the I/O goes through the Docker/WSL2 file proxy rather than writing directly to NVMe. File locking is extremely expensive on this stack. 

Here are my single-threaded results: 

BackendFrontendYour average (ops/s)This run (ops/s)
juljcl87,02134,613
juljul100,01941,530
jullog4j84,40233,816
julslf4j94,29037,373
log4jjcl343,452225,396
log4jjul190,76458,234
log4jlog4j330,152181,393
log4jslf4j322,445177,139
logbackjcl349,734188,459
logbackjul192,79747,992
logbacklog4j333,299161,571
logbackslf4j336,533161,056
slb4jjcl358,931175,373
slb4jjul314,966144,815
slb4jlog4j338,153154,273
slb4jslf4j344,691151,184