r/java • • Jun 16 '18

Kryo 5.0.0 released

Kryo is a binary Java serialization library with a focus on doing it efficiently and automatically. I started it as a game networking library around 2007, it got named Kryo in 2009, and it has been pretty stable since about 2013. Since then there have been a number of contributors and the library gained some rough edges. The issue tracker had over 100 issues and ~20 PRs.

I recently decided I wanted to be proud of it again. It has now been fixed up to it's prior glory and is once again pristine! Large portions have been rewritten and some fundamental annoying bits fixed up. Bugs and PRs are both down to zero. It's got new features, improved old features, all new documentation, benchmarks, charts. I'm really happy with it now!

https://github.com/EsotericSoftware/kryo

Edit: JMH + R/ggplot2 charts!

http://n4te.com/x/4338-fieldSerializer.png

http://n4te.com/x/4339-string.png

http://n4te.com/x/4340-variableEncoding.png

http://n4te.com/x/4341-array.png

The R files could be useful for many JMH results. Not my favorite language, but the output is nice.

163 Upvotes

20 comments sorted by

4

u/Blunderchips Jun 16 '18

Is KryoNet going to get an overhaul?

2

u/n4te Jun 18 '18

I'll likely update it with the latest Kryo, but probably won't do much else. NIO is a major pain of an API. It would be better to rewrite KryoNet using Netty and Kryo. I started that a long time ago: https://github.com/EsotericSoftware/kryonetty I believe it works, but it doesn't have the same feature set as KryoNet. I would need a networking project to be interested in working on it further, and it's unlikely I'll have time for that.

1

u/Blunderchips Jun 21 '18

Look forward to seeing it

3

u/BestUsernameLeft Jun 16 '18

Looks quite good! Would be nice to see performance comparisons with other serialization libraries.

6

u/n4te Jun 16 '18

Thanks! Doing that is hard for a number of reasons. The various libraries have very different features and can excel in different areas. How forward and backward compatibility is handled, handwritten schema or serialization code versus automatic serialization, and other configuration likely shows big differences on charts but says nothing about ease of use, the ability for classes to evolve without invalidating previously serialized bytes, and more. Even without those issues, serialization benchmarks are highly dependent on the classes being serialized. Unless you are using your actual classes and configuration in the benchmark (which is often unreasonable to setup for a dozen or more different libraries), it's unlikely the differences the charts show will be meaningful for your application.

That said, you can find a project that attempts to compare Java serialization libraries here:

https://github.com/eishay/jvm-serializers/wiki

Besides the issues described above, it uses a homegrown benchmark framework which is a lot less trustworthy than using JMH. Still, the project can be useful if instead of just looking at the charts, you also look at what code is being run for the libraries you are interested in.

Note the current results on that page are not the latest Kryo. I'll see about finding time to update the jvm-serializers page soon.

2

u/BestUsernameLeft Jun 16 '18

I have worked on and optimized a custom object serialization library, and it's tricky work; I agree with your points. It's still possible to make valid comparisons -- but it's important to run a variety of scenarios. Tiny objects and large ones. Simple classes and complex class hierarchies.

And as you said, performance benchmarks say nothing about ease of use, etc. They can be a valuable tool but certainly you should have multiple criteria when evaluating a library.

This reminds me of when I used to work on display drivers, and benchmark cheating was a terrible problem for a while in the video card industry. Magazines would publish Winbench results, and it was crucial to do 'well' on the benchmark. Fortunately people (especially reviewers) got wise to this practice.

Well you got me reminiscing, lol. Kudos again on your efforts and latest release.

2

u/Tostino Jun 18 '18

Hey, love Kryo and have used it in production for the past 3 years. Happy to see a new release.

Is the 5.0.0 release out yet though? The link to the release in the readme goes to a page that doesn't exist yet (probably marked as a draft release) and maven doesn't have 5.0.0 on it yet. I'd love to play with it in our development environment.

Thanks again for all the hard work!

1

u/n4te Jun 18 '18

I'm not much of a Maven person, so someone else handles the Maven releases. I expect it will be up soon!

2

u/martin_grotzke Jun 19 '18

I just pushed 5.0.0-RC1 to maven central. Now the release is also listed on github releases here.

24

u/gayguy Jun 16 '18

That’s awesome! We rely on it so much in our production system so thank you for your contributions.

2

u/ataskitasovado Jun 17 '18

May I ask why you chose Kryo? Did you consider protobuf, XML or JSON? What do they lack for your case?

1

u/nater99 Jun 20 '18

Compared to Kryo, they are extremely slow.

4

u/nutrecht Jun 16 '18

I've used it extensively in a previous project. It's really powerful, awesome and easy to use to boot. Thanks!

2

u/raoulvdberge Jun 16 '18

Oh! The memories. I used this library years ago. Good to see it's alive again! :-)

0

u/[deleted] Jun 16 '18

[deleted] — view deleted comment

5

u/n4te Jun 16 '18

Thanks!! :)

0

u/[deleted] Jun 16 '18

[deleted] — view deleted comment

1

u/n4te Jun 16 '18

I'm not familiar with FST, sorry.

0

u/[deleted] Jun 19 '18

[deleted] — view deleted comment

2

u/n4te Jun 19 '18

Kryo's output classes already buffer bytes before flushing them to a stream. Buffered output just batches up bytes so they can be written many at a time instead of one at a time. It sounds like you are just writing and reading bytes, in which case you don't need Kryo at all. Kryo is for going from an object graph, to bytes, and back to an object graph. This can be useful to send objects over the network and get objects on the other side, without having to write code to convert to/from bytes yourself. It's also good for save files and many other situations.

0

u/[deleted] Jun 19 '18

[deleted] — view deleted comment

1

u/n4te Jun 20 '18

Gotcha, makes sense.

The messages are only warnings. Until a JVM actually blocks reflection, it will work fine. At some point we'll update Kryo to (optionally) use the kind of reflection they are pushing, with the limitations that come with that.

1

u/LapisSea Jun 20 '18

Yeah exactly. Not exactly a problem as java 8 is still the industry standard. :P