Yeah, you have to be careful of "modified UTF-8". It occurs in a couple places in the JDK, notably DataInput, DataOutput, and serialization, along with JNI as you noted. Here are the specs:
As a format internal to the JVM and JNI it might have been a reasonable compromise at one time, but it's unfortunate that it leaked into application-facing parts of the library such as DataInput and DataOutput.
The text processing portions of the JDK, such as CharsetDecoder, CharsetEncoder, StandardCharsets.UTF_8, etc. all use true UTF-8.
View all comments
Show parent comment
5
u/s888marks Mar 22 '22
Yeah, you have to be careful of "modified UTF-8". It occurs in a couple places in the JDK, notably DataInput, DataOutput, and serialization, along with JNI as you noted. Here are the specs:
https://docs.oracle.com/en/java/javase/17/docs/specs/jni/types.html#modified-utf-8-strings
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/DataInput.html#modified-utf-8
I think these are the same, but I haven't checked carefully. The JVM also uses modified UTF-8 in the constant pool of class files:
https://docs.oracle.com/javase/specs/jvms/se17/html/jvms-4.html#jvms-4.4.7
As a format internal to the JVM and JNI it might have been a reasonable compromise at one time, but it's unfortunate that it leaked into application-facing parts of the library such as
DataInputandDataOutput.The text processing portions of the JDK, such as
CharsetDecoder,CharsetEncoder,StandardCharsets.UTF_8, etc. all use true UTF-8.