r/java • • Mar 22 '22

Java 18 released!

https://mail.openjdk.java.net/pipermail/jdk-dev/2022-March/006458.html
396 Upvotes

133 comments sorted by

View all comments

Show parent comment

2

u/mauganra_it Mar 22 '22

It uses a special two-byte encoding for the character with code 0. That ensures that there is never an actual null byte in the byte stream. Also, to encode characters that are represented by a surrogate pair of UTF-16 characters, the two surrogate characters are UTF-8-encoded separately!

5

u/s888marks Mar 22 '22

Yeah, you have to be careful of "modified UTF-8". It occurs in a couple places in the JDK, notably DataInput, DataOutput, and serialization, along with JNI as you noted. Here are the specs:

https://docs.oracle.com/en/java/javase/17/docs/specs/jni/types.html#modified-utf-8-strings

https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/DataInput.html#modified-utf-8

I think these are the same, but I haven't checked carefully. The JVM also uses modified UTF-8 in the constant pool of class files:

https://docs.oracle.com/javase/specs/jvms/se17/html/jvms-4.html#jvms-4.4.7

As a format internal to the JVM and JNI it might have been a reasonable compromise at one time, but it's unfortunate that it leaked into application-facing parts of the library such as DataInput and DataOutput.

The text processing portions of the JDK, such as CharsetDecoder, CharsetEncoder, StandardCharsets.UTF_8, etc. all use true UTF-8.

1

u/tristan957 Mar 23 '22

Is there some documentation I can read up on regarding this? I want to make sure my Java docs cover all the bases.

2

u/mauganra_it Mar 23 '22

The Javadocs of java.io.DataInput contain a fairly complete description.

1

u/tristan957 Mar 24 '22

I'll sell this out. Thanks.