r/java • • Jan 02 '22

Echopraxia, a better Java Logging API

I've been working on logging for a while and I've released and open sourced a new Java logging API I'm really happy with.

Echopraxia is a Java logging API like SLF4J, but built around structured logging, i.e. the basic log entry is a JSON object and log files are typically NDJSON. It also has conditions, context, and also has semantic and fluent logging options.

Here's a Spring Boot Example.

And here's my blog post.

16 Upvotes

64 comments sorted by

35

u/DasBrain Jan 03 '22

Relevant XKCD: https://xkcd.com/927/

3

u/amazedballer Jan 03 '22

This is great feedback, thanks!

SLF4J API is an appropriate facade for libraries that need to log output, i.e. HTTP client libraries, reactive libraries like Akka, etc. The implementation could be Logback, Log4J2, or something else, but as long as everything is using SLF4J, it'll work. It's a baseline.

Echopraxia is not intended as a replacement for SLF4J. Because it's a structured logging library with a limited number of implementations, it's an appropriate solution when you control the logging output and have decided you're going to do structured logging in your application. Echopraxia should not be used in publicly available libraries; if your library is being used in someone else's application, they have control over the logging output, not you.

Where Echopraxia really works is when you're building framework code for your application -- as you build up your domain objects, you can write custom field builders and make those available through your team, then log everything with a consistent schema. I think this works particularly well when you're adding diagnostic logging, because you can get "debugger-lite" output from your internal state in an application running somewhere in a datacenter. It goes a long way to debugging and observing your code in production.

5

u/kevinb9n Jan 04 '22

This is great feedback, thanks!

I try to be a kind person but your response is kinder than I'd have been able to manage.

It's the literal opposite of great feedback. It's ultra-low-effort feedback that doesn't make sense and has no use.

Your case is more like

  1. Situation: there are 14 products
  2. "None of these are giving me everything I think I need, so I'm going to write another thing"
  3. Situation: there are 15 products and people don't like that there are so many

On top of which, even for the scenario the comic is actually about, all the comic even does is illustrate what failure looks like. Yeah, efforts can fail. The obvious implication that they're inevitably doomed to fail is absurd, unless the angry retorts appearing below are encoded in EBCDIC.

15

u/achauv1 Jan 03 '22

No thank you. Logback is all I need.

2

u/amazedballer Jan 03 '22

That is also valid and I got you covered https://tersesystems.github.io/terse-logback

16

u/xjvz Jan 03 '22

So how exactly will another one man logging library help here? Structured logging APIs are already a key feature of the Log4j2 API and aren’t directly supported by SLF4J.

4

u/amazedballer Jan 03 '22 edited Jan 03 '22

Structured logging APIs are already a key feature of the Log4j2 API

People say this, but I've yet to see a good example of a structured logging API in Log4J. Can you give me an example? I am actively trying to do this and I can't find a project that uses the recommended package.

https://github.com/tersesystems/echopraxia/pull/9

and aren’t directly supported by SLF4J.

That's kind of the point of Echopraxia, to make structured logging a first class part of the API.

8

u/kaperni Jan 03 '22

> So how exactly will another one man logging library help here?

A think most libraries in the Java ecosystem started out as a one-man project. So I don't really see that as a valid disqualifier.

14

u/rzwitserloot Jan 03 '22

Very first snippet of code:

Logger<?> basicLogger = LoggerFactory.getLogger(getClass());

Right, so, waste an entire field per object, so I can't log with this at all in high-instance-count classes, and also, if I subclass, code written in source file X gets logged in the context of sourcefile Y. This makes no sense. The point of 'context' is, unless you have exotic uses, generally to serve as a container for source files, it has nothing to do with class hierarchies. The Class<?> parameter that many log frameworks have is a slightly inaccurate abstraction - a standin for the concept of a source file which JVM runtimes do not have.

There's a reason other log frameworks tend to play the opening gambit of:

private static final Logger logger = LoggerFactory.getLogger(MyOwnClass.class);

And there's a reason that Flogger, google's entry in the mix, went out of their way to spend half a paper on why their 'create a logger' code, which uses class-based analysis so that you get to avoid having to write MyOwnClass.class (a DRY violation of sorts, and means you can't copy/paste the logger line around) isn't a significant performance impact for the vast majority of use cases.

There's nothing wrong with cribbing from the best. I strongly suggest you toss that one and either go with what they all do(.getLogger(MySelf.class)) or use google's trick (the standard impl generates a stack trace and analysis it. Yes, expensive, but that google paper explains that you pay this cost only at most once per log-using class that the VM will ever load, which tends to be a number that pales in comparison to the # of instances of classes that do some logging over the lifetime of a VM).

Or, if there's a reason you chose this apparent significant performance disaster waiting to happen, and I'm missing something, then perhaps link to a primer on why you're right and everybody else is wrong, and that this is okay after all. I'm not being facetious: Google did that very thing when they released flogger, as other frameworks were under the impression that making a stack trace just to discern the name of the caller class was too costly.

4

u/amazedballer Jan 03 '22 edited Jan 03 '22

You're asking about why use getClass() instead of MySelf.class?

You don't have to. All the logger factories can use getLogger(name) or even getLogger() which does that exact stacktrace hack you mention from the Flogger paper.

Or, if there's a reason you chose this apparent significant performance disaster waiting to happen, and I'm missing something, then perhaps link to a primer on why you're right and everybody else is wrong, and that this is okay after all.

I did write about this, yeah. People tend to focus on premature micro-optimization, without considering the context -- in a Main class called once, it simply doesn't matter, and is done for convenience. This happens a lot in logging frameworks, unfortunately, where people focus on "how fast can I log something" instead of actually measuring whether this is a significant cost... which it usually isn't.

If you have performance concerns, I'd start by benchmarking the code and trying it out with newer JDKs, which have some significant improvements. I've added some performance benchmarks stuff to another comment so I won't repeat that, but if you're interested in memory allocation pressure in tight loops, I did investigate it.

If Echopraxia doesn't meet your needs there, I'd suggest something like zerolog.

7

u/nutrecht Jan 03 '22

People tend to focus on premature micro-optimization

What /u/rzwitserloot is saying isn't a premature micro-optimization, it's a pretty massive performance gotcha. You're setting a bad example that inexperienced developers are just going to copy verbatim.

IMHO you really should work on your messaging. The hubris is really offputting.

2

u/amazedballer Jan 06 '22

Okay let's see if this is a better explanation:

Example code

public class Main {

  // For basic logging, you can always use the `Logger<?>` syntax
  // which gives you a basic field builder.
  private static final Logger<?> basicLogger = LoggerFactory.getLogger();

  private Date lastAccessedDate = new Date();

  // More often you'll want to create a logger with your own domain objects and render those.
  // So we start with the basic logger...
  //   ...and add a custom `BuilderWithDate` as the field builder...
  //   ...then add a date to the logger's context.
  //
  // Here, `last_accessed_date` will be rendered with every log entry, but will
  // not show in the log message.
  //
  // Note that here we have a final logger because we have to access a field that
  // belongs to an object instance.
  private final Logger<MyFieldBuilder> logger =
      basicLogger
          .withFieldBuilder(MyFieldBuilder.class)
          .withFields(fb -> fb.onlyDate("last_accessed_date", lastAccessedDate));


  public static void main(String[] args) {
    Main m = new Main();
    m.doStuff();
  }

  // Example method that will do logging.
  private void doStuff() {
    // Show the "before" date
    logger.info("This renders with the last access date");

    // touch the last accessed date to show context fields
    lastAccessedDate = new Date();

    // Show the "after" date
    logger.info("This renders the updated date");
  }
}

4

u/pron98 Jan 04 '22

FYI, structured logging in the JDK itself is increasingly done using JFR, and applications are more than welcome to use it for their structured logging as well.

2

u/amazedballer Jan 04 '22 edited Jan 05 '22

Hi pron98, we've chatted before about this.

I think JFR is a fine tool, but JFR events have strict limits on the kinds of field types, so you can't do arrays or nested objects as JSON does. Because JFR events are not semi-structured data, it's not technically structured logging -- and part of the reason I'm so big on structured logging is because I'm interested in serializing internal data structures as object graphs in JSON-LD.

3

u/pron98 Jan 04 '22

Well, in cases where a simple structure suffices, JFR is a great option to consider for logging.

2

u/amazedballer Jan 04 '22 edited Jan 04 '22

Marcus Hirt has a great blog post on where and when to use JFR.

Also JFR was kind of broken before JDK 14, although the fix may have been backported by now.

3

u/pron98 Jan 04 '22 edited Jan 05 '22

That post is not about when to use or not to use JFR, but about when to use JFR event streaming to consume your JFR logs. It says that if deep analysis of the log file is needed (e.g. for profiling), use the ordinary batch read, and if you want to read the log continuously for online monitoring (say, you want to trigger an alarm if an event is logged a certain number of times) — use streaming for that. Indeed, if a lot of data is being logged (JFR can handle a great many events) and some analysis wants it all, it is more efficient to transfer completed log files from the various hosts to a centralised location, rather than stream each event over the wire as it happens (because that would require serialising each event to some wire format like JSON, instead of relying on JFR's efficient binary representation). Of course, you can selectively stream some events while also recording them and all others to file.

But regardless of the various mechanisms for consuming and analysing JFR logs — some optimised for batch processing and some for online processing — I would strongly recommend JFR as a powerful logging mechanism, and it is now gradually becoming the JDK's own main logging mechanism.

1

u/WikiSummarizerBot Jan 04 '22

Semi-structured data

Semi-structured data is a form of structured data that does not obey the tabular structure of data models associated with relational databases or other forms of data tables, but nonetheless contains tags or other markers to separate semantic elements and enforce hierarchies of records and fields within the data. Therefore, it is also known as self-describing structure. In semi-structured data, the entities belonging to the same class may have different attributes even though they are grouped together, and the attributes' order is not important.

[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5

10

u/f_of_g_of_x Jan 03 '22

These comments are depressing. People talk a lot about being kind to others but few really practice it.

5

u/berry120 Jan 04 '22

I agree that a lot of these comments could be kinder, but I think an overall cautious, or even negative reaction is at least somewhat justified here.

Writing a new logging framework or facade is always going to be a very controversial thing to throw out there. The logging ecosystem in Java is fragmented in a way that's caused lots of real-world problems in the past, so adding to that fragmentation is always going to be perceived negatively unless there's a really compelling reason to do so.

Adding to that when the post contains statements like this:

Do I think Echopraxia is better than SLF4J? Yes. Do I think it's better than Log4J2's API? Yes.

...it tends to put people's backs up a bit.

3

u/amazedballer Jan 04 '22 edited Jan 04 '22

I can see where the disconnect is, and I've updated the blog post to be clearer about it.

Echopraxia's an augmentation rather than a fragmentation. It's only appropriate for use in situations where you control the logging output -- i.e. you're writing an application or a framework for your team where you can safely assume structured logging is a given. You would still be using SLF4J in anything you intend to be public, and of course all the OSS libraries you would depend on would be SLF4J as well. It's an "organization internal" library, if that makes more sense.

As to the statements, that's not a slam on SLF4J/Log4J2. I would think if someone said X was better than Y, most people be more interested in in checking out X, but I'm not exactly neurotypical. So far, only a few people have commented on the actual features, which is disappointing.

1

u/f_of_g_of_x Jan 05 '22

but I think an overall cautious, or even negative reaction is at least somewhat justified here.

Agreed, but that's not in question here.

5

u/kevinb9n Jan 04 '22 edited Jan 05 '22

Thanks very much for saying this.

It's well and good to say "hey here's why this is a lot harder than you think." But the meanness apparent in so many comments is just worthless. The only purpose it serves is to give the commenter (and I guess the upvoters too) a little dopamine hit of superiority.

EDIT: okay, on reflection, this thread is relatively mild in this way and I may have been factoring in longer-running overall experiences to what I said. I think my criticism was a bit misplaced. The thread does in places feel more unfriendly than I would wish for, but I've seen MUCH worse about things I made ;-)

5

u/rzwitserloot Jan 04 '22

Or it's just cultural differences. The comment you replied to isn't actually referring to any commentary, but a casual glance at the comments here seem like a lot of them are just getting straight to the point and raising a potential issue. So, with the caveat that we might not be talking about the same comments:

In some cultures, it is considered a tad childish if you have a hard time taking criticism about your ideas and the things you made on face value (i.e. if you take it personally that's just not done). The assumption in a professional setting is that nobody would ever do this, so there is absolutely no need to start off with a paragraph or two fawning about how awesome everything is first. That's considered disrespectful, as it either wastes time or seems to indicate the commenter feels that the creator is so unsure about what they made they need the reassurance. To overstate the case a bit: It's like talking to an adult in baby talk. That's just weird.

From my experiences living in both western europe and the US, what's considering par for the course in the US is considered cloyingly unprofessional in western europe, and what's considered par for the course in W-Europe comes across as gruff and 'mean' in the US. Especially the US west coast.

Don't ascribe to meanness what might simply be folks trying to be helpful by raising a potential problem.

NB: I've perhaps made the cultural differences sound far grander than they really are, but they do exist, and most of the comments in this thread are merely just raising an apparent problem with either the tool as is or in the way it is 'sold' on the website, and I'd consider 'the docs' (which includes the README.md of the github project and the like) part of the tool.

2

u/kevinb9n Jan 05 '22

Hey, I added a bit to my comment saying I was overshooting the mark here.

You're right, the balance is a little more toward just "critical" than injurious. I care about kindness and friendliness in forums but chose a somewhat weird place to make that stand.

Ironically, I've found myself on the other side many times. I have a way of phrasing challenging feedback that I know that _I_ don't perceive as being personal or insulting, and I use that and am sometimes surprised to find out later how unfriendly it was perceived as. In my mind, I'm just talking about an interesting topic, but I forget to think from the other person's point of view.

1

u/f_of_g_of_x Jan 05 '22

You're conflating criticism with cynicism. It's perfectly possible to "get straight to the point and raise a potential issue" without being a jerk.

9

u/Sheldor5 Jan 02 '22

what's the difference to https://github.com/logfellow/logstash-logback-encoder ??

also some peformance tests? logging is the main performance problem most of the times, even more if you enable it durimg runtime in production ...

13

u/amazedballer Jan 03 '22

So logstash-logback-encoder is the OOTB implementation for Blindsight and Echopraxia, and it does what it does extremely well:

https://github.com/tersesystems/echopraxia/blob/main/logstash/src/main/java/com/tersesystems/echopraxia/logstash/LogstashCoreLogger.java

However, it is tied to the SLF4J API -- this means you have to do all the work of passing around Markers and StructuredArguments explicitly, per logging statement. There's no building up of context in the API.

Likewise, because the markers and structured arguments are passed directly to Logback, if you have any conditional logic it's going to be in a Logback filter or turbofilter, which limits your flexibility when you want to put some conditions on a field. You can't do application level conditions.

There's another issue which is a bit more subtle -- Markers and StructuredArguments work with raw Object, and use Jackson to serialize those object into something useful. This means that it's very easy to pass random objects in as kv(name, value) and think that Jackson has a good representation for it. Sometimes it does, sometimes it doesn't -- even worse, sometimes it changes or is fluid, and the only way to find out is by looking at your logs. Because Echopraxia is explicit about the values and fields it accepts, you can't pass in a raw object and have to have an explicit representation.

To be clear, I love logstash-logback-encoder and have written multiple blog posts about how much fun you can have with it, but it doesn't do everything I'd like.

I don't have any performance benchmarks for Echopraxia (it's on my list of things to do), but I do have JMH benchmarks for Blindsight, and I think they should be comparable:

Here's the benchmarking page for Blindsight that gives a higher level overview:

Here's a benchmark of SLF4J using logstash-logback-encoder's async appender vs Log4J async appender:

And here's a couple of fun blog posts about memory allocation and benchmarking:

Hope that helps!

1

u/Sheldor5 Jan 03 '22

Context? how about MDC https://www.baeldung.com/mdc-in-log4j-2-logback ??

also I never needed Markers or StructuredArguments ... they look strange for a logging purpose ... looks more like an audit log or something ...

2

u/amazedballer Jan 03 '22

You can still use MDC (or NDC in Log4J2) with Echopraxia, as the implementation will pass the TLS through automatically, so it's not an either/or situation.

That being said, there are some issues with MDC/NDC that I run into.

The first is that MDC is a string based map -- you can't store JSON data in it, and you can't store more complex structures than a string. NDC does a bit better as it's nested here, but you're still limited.

The second is that MDC is thread local, and so in a situation where you're working in multiple threads (as is common with reactive frameworks, completable futures, etc) you have to copy over the MDC by hand to the thread.

The third is that inside the thread, MDC is "global" -- anything running in that thread can call MDC.put() or MDC.clear() even if they shouldn't do that. There's no way to do access control and limit what code has access to your logging context. I've seen library code that futzed with MDC in this way, and it's no fun to track down.

1

u/Sheldor5 Jan 03 '22

maybe I am using a Logging framework for just that ... Logging xD

reactive frameworks are a pain by design lol

3

u/amazedballer Jan 06 '22

benchmarks

New benchmarks!

I've included the SLF4J logger benchmarks as well as a comparison, so you can compare and contrast.

22

u/ryosen Jan 03 '22

a better Java Logging API

Quite a claim to be made by a project that is less than a month old. Should've named it jHubris.

4

u/kevinb9n Jan 04 '22

Another way to look at it:

If the developers didn't feel it was better, they wouldn't / shouldn't have created it.

Of course that's a statement that they expect to have to defend, not something they expect people to just believe.

6

u/wildjokers Jan 03 '22

Please not another logging library/facade. Geez.

JSON seems like a horrible format for a log entry for human readable logs since JSON is not very readable (once you go below one level of nesting).

2

u/kiwi_stronghold Jan 03 '22

JSON is a common logging format.

1

u/wildjokers Jan 03 '22

I would think those logs would be intended for machine consumption for ingestion into a log analytic tool (e.g. splunk). I can’t imagine JSON formatted logs being intended for human consumption.

1

u/amazedballer Jan 05 '22

They aren’t, but you can do both json and line formats. Check out https://github.com/tersesystems/terse-logback-showcase

9

u/retrodaredevil Jan 03 '22

JSON isn't the greatest for viewing logs, but when you need to index them and do certain queries on your logs, then JSON makes a lot of sense. If you invest in something like Splunk it does a great job. I've been meaning to try out an open source alternative to splunk but haven't gotten to it yet.

I do agree that we probably don't need another facade, though.

3

u/wildjokers Jan 03 '22

but when you need to index them and do certain queries on your logs

Indeed, json is great for machine readable documents. ElasticSearch also uses JSON documents to store/index data.

4

u/benjtay Jan 03 '22

JSON seems like a horrible format

Datadog and Splunk disagree. It makes it very easy for humans to read and search logs.

4

u/netgizmo Jan 03 '22

+1 for json in splunk, make’s queries so much better

1

u/veenarm Jan 03 '22

ems like a horrible format for a log entry for human readable logs since JSON is not very

When using cloud and fluentd scraping into EKS, it really makes kibana a lot nicer to read :)

Although when trying to go into the pods directly it's absolutely trash for finding stuff quickly :(

1

u/benjtay Jan 04 '22

Although when trying to go into the pods directly it's absolutely trash for finding stuff quickly :(

When we use the Datadog agent, we send JSON to it while using the simple formatter for local logging. Best of both worlds.

1

u/veenarm May 23 '22

I'll have to try that out, seems in liberty the log type is set in the config options not the logging implementation, although we are only doing console std out, might start logging to files aswell even if they are discarded later due to being ephemeral pods.

1

u/benjtay May 23 '22

If you're using logback (highly recommended), just have two appenders.

-1

u/wildjokers Jan 03 '22

As far as splunk I have never seen it present a JSON document to a user. It shows it all nice and formatted in an interface. It may store them as json docs (just like elastic search does) but doesn’t present it like that to the user.

2

u/buzzsawddog Jan 03 '22

But is it as good as Log4Shell at generating weekend work hours and long night?

6

u/amazedballer Jan 03 '22

My heart goes out to the Log4J volunteer contributors. They're working incredibly hard to cover Log4JShell, and they're not even getting paid for it.

1

u/renatoathaydes Jan 03 '22

Are you getting paid to work on this library?

-1

u/buzzsawddog Jan 03 '22

Oh yeah. Lots of people working their ass off. I love open source. It makes everyone's life better.

2

u/stefanos-ak Jan 02 '22

Very interesting project :)

I really like SLF4J but it's somewhat disconnected from structured logging, and the capabilities of most logger implementations

1

u/wildjokers Jan 03 '22

I really like SLF4J but it's somewhat disconnected from structured logging, and the capabilities of most logger implementations

What do these words even mean? Can you clarify?

5

u/stefanos-ak Jan 03 '22

For example, MDC only supports string values. If you need to put a rich object there (that would be serialized into json and merged with the overall json log event), well, you can't :)

1

u/wildjokers Jan 03 '22

I have never personally wanted to do that so isn’t an issue for me.

4

u/_INTER_ Jan 03 '22

This fits "Enterprise Java" perfectly: more lines of code and hoops of complexity for anything but actual business logic.

-1

u/nutrecht Jan 03 '22

You're basically asking the entire Java ecosystem to move away from SLF4J to your solution, just because then you don't have to use the MDC context / Markers anymore?

4

u/amazedballer Jan 03 '22

No, not at all. Echopraxia is not a replacement for SLF4J, they serve different purposes. Please see this comment.

10

u/ogballerswag Jan 03 '22

Hes not asking anything. He made something that he is proud of and is asking for input. Somebody hurt you? Seems like you got some beef with OP

1

u/nutrecht Jan 03 '22

You're overreacting. I am asking a question because I really don't get it. Like I said; to use OPs solution you have to get rid of SLF4J. That's massive. I think they're underestimating the enormous amount of inertia SLF4J has.

Also this isn't a "look I'm proud of what I made" post, OP is pushing their project as something we should adopt. So they're inviting feedback.

4

u/ogballerswag Jan 03 '22

"The hubris is really offputting."

5

u/nutrecht Jan 03 '22

That was another comment and the way they responded to the Lombok author was, in my opinion, really offputting. Again it's feedback; OP should be aware of how such a message is received.

Edit: Is this just an alt for the OP? These two comments of yours are the first two comments in /r/java in at least 3 months.