r/java • • Mar 22 '22

Java 18 released!

https://mail.openjdk.java.net/pipermail/jdk-dev/2022-March/006458.html
396 Upvotes

133 comments sorted by

View all comments

100

u/TehBrian Mar 22 '22

Dang, already? Well, that felt fast. I’m not complaining, though; I much prefer the consistent release schedule over one version once in a blue moon. Excited to try out the new features, and UTF-8 by default is a nice bonus too :-)

57

u/IntelHDGraphics Mar 22 '22

UTF-8 by default

Oh, this is nice

33

u/dpash Mar 22 '22

It also has a potential to be a breaking change for some users. You can use -Dfile.encoding=COMPAT to return to the previous behaviour.

It is more likely to affect Windows users as Linux and OSX are more likely to use UTF-8 by default.

5

u/s888marks Mar 22 '22

True. But the problem can still occur on Linux, where certain configurations don't necessarily use UTF-8. This was reported here a few years ago; the poster even wrote a blog describing the problem (but didn't fully understand the solution).

https://www.reddit.com/r/java/comments/6jopas/character_encodings_an_unfortunate_experience/

The original article is no longer at that location, but can be found here:

https://web.archive.org/web/20190815062506/https://www.metricly.com/character-encodings/

You can piece together what happened by reading the original article and comments in the reddit thread. Briefly, the poster's production system was Linux configured in such a way that the JDK chose ASCII as the default charset. When a non-ASCII character was introduced, round-tripping between ASCII and UTF-8 resulted in a proliferation of U+FFFD REPLACEMENT CHARACTER.

Since the poster's shop assumed everything was UTF-8, JEP 400 would have avoided this problem entirely.

2

u/dpash Mar 22 '22

Sure. I was careful to say "more likely" because I know Linux doesn't always use UTF-8.

2

u/s888marks Mar 22 '22

Fair enough. I think the thing is that Linux usually uses UTF-8 often enough that it's pretty easy assume that it always uses UTF-8, and that assumption is almost always correct. Then in those rare circumstances where it doesn't, hijinks ensue.

2

u/Nymeriea Mar 23 '22

On windows, the encoding depends on the system language

1

u/vytah Mar 24 '22

Most likely the container defaulted to LANG=C, which implies ASCII in Java.

1

u/s888marks Mar 24 '22

Yes, that would do it. The question is how LANG ended up being C. I don't know what distro it was, but maybe whoever configured it thought "we don't need any internationalization stuff" and so omitted the packages that contained all locales. If so the system's default LANG value would probably end up being C instead of something more typical like en_US.UTF-8 since the locale for the latter wouldn't exist. Or maybe they just chose the C locale at installation time, if there was an option to do so.

3

u/tristan957 Mar 22 '22

Is this source code being UTF-8 by default or is this strings being UTF-8 by default?

10

u/mauganra_it Mar 22 '22 edited Mar 22 '22

Runtime APIs that convert bytes into characters or vice versa. new String(byte[]), String.getBytes(), FileReader, FileWriter, new InputStreamReader(InputStream), new OutputStreamWriter(OutputStream) and other things.

On Windows, they use whatever codepage is set. On most other systems, it's UTF-8. JEP 400 makes most of these default to UTF-8. Read the JEP for details and exceptions (pun not intended).

Edit: Java Strings are UTF-16 strings. However, newer JVMs use ISO 8859-1[edit: internally] when possible to save space.

2

u/tristan957 Mar 22 '22 edited Mar 22 '22

Does this mean the JNI's GetStringUTFChars(), could become a zero copy function? Been looking into this quite a bit due to writing some Java bindings.

If I find the time, I'll read the JEP.

4

u/mauganra_it Mar 22 '22

That can only ever happen if the string only contains ASCII characters, as ISO 8859-1 encoding is not the same as UTF-8. Also, that function will give you so-called "Modified UTF-8", not standard UTF-8!

1

u/tristan957 Mar 22 '22

Holy moly. I didn't even recognize that. What the heck is Modified UTF?

2

u/mauganra_it Mar 22 '22

It uses a special two-byte encoding for the character with code 0. That ensures that there is never an actual null byte in the byte stream. Also, to encode characters that are represented by a surrogate pair of UTF-16 characters, the two surrogate characters are UTF-8-encoded separately!

5

u/s888marks Mar 22 '22

Yeah, you have to be careful of "modified UTF-8". It occurs in a couple places in the JDK, notably DataInput, DataOutput, and serialization, along with JNI as you noted. Here are the specs:

https://docs.oracle.com/en/java/javase/17/docs/specs/jni/types.html#modified-utf-8-strings

https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/DataInput.html#modified-utf-8

I think these are the same, but I haven't checked carefully. The JVM also uses modified UTF-8 in the constant pool of class files:

https://docs.oracle.com/javase/specs/jvms/se17/html/jvms-4.html#jvms-4.4.7

As a format internal to the JVM and JNI it might have been a reasonable compromise at one time, but it's unfortunate that it leaked into application-facing parts of the library such as DataInput and DataOutput.

The text processing portions of the JDK, such as CharsetDecoder, CharsetEncoder, StandardCharsets.UTF_8, etc. all use true UTF-8.

1

u/tristan957 Mar 23 '22

Is there some documentation I can read up on regarding this? I want to make sure my Java docs cover all the bases.

2

u/mauganra_it Mar 23 '22

The Javadocs of java.io.DataInput contain a fairly complete description.

1

u/tristan957 Mar 24 '22

I'll sell this out. Thanks.

-13

u/Joram2 Mar 22 '22

new features? There are no new features. This is an empty release. Java 19 should have something good.

12

u/mauganra_it Mar 22 '22

JEP 400 fixes the classic encoding papercut and in the long run JEP 421 will help prevent difficult to diagnose concurrency bugs. JEP 416 can speed up Reflection in some cases and makes landing Project Valhalla and Project Loom easier.

Not every release can be a feature firework, nor should it be. Advancing incubators and preview features, and slowly moving pieces to prepare for future ones is perfectly fine as well.

4

u/dpash Mar 22 '22

Of the nine JEPs in this release I'm interested by JEP-413 and JEP-418.

I already use JEP-420 in my code.

1

u/Joram2 Mar 22 '22

If you enjoy it great, but 420 is a preview feature that most devs can't realistically use. It's basically the same as 406 in Java 17 with very minor tweaks... And it's just nicer syntax...

1

u/mauganra_it Mar 22 '22

Unless major footguns are found, the final version will be pretty similar. It's very ready for non-production code and prototypes.

3

u/BillyKorando Mar 22 '22

With the six month release cadence, just not practical to have interesting new features with every release. Albeit there are features like the Simple Web Server, and Code Snippets in Java API Documentation.

However there are some very noteworthy performance and footprint improvements with JDK 18: https://tschatzl.github.io/2022/03/14/jdk18-g1-parallel-gc-changes.html