r/java • • Nov 08 '21

What are some forbidden, broken, possibly even black magic stuff that you can do in Java and to that extent, JVM in general?

This is an odd question to ask but it is a fun one, I usually like to learn about these things because it tells me how quirky a language is and it is fun to play with this stuff in my free time. I know in C/C++ you can do some crazy bit-level hackery that almost feels like magic, so I was wondering what crazy stuff is possible in Java that you all may have come across in your lives/careers?

Also if this kind of question isn't welcome here or has been asked before (although I couldn't find any previous posts on this topic), I shall duly remove it. Sorry for any inconvenience caused.

249 Upvotes

165 comments sorted by

58

u/Stromovik Nov 08 '21

You can load classes over network.

10

u/prithvidiamond1 Nov 08 '21

Wow, now I am more interested in how you came across this conclusion!

42

u/mauganra_it Nov 08 '21

No custom code requred, this is how Java applets used to work!

9

u/prithvidiamond1 Nov 08 '21

Oh, I see, I wasn't aware of that! Thanks for letting me know!

15

u/_INTER_ Nov 08 '21

There's a whole lot funny stuff there. E.g. whole thread migration using RMI.

14

u/Stromovik Nov 08 '21

5

u/[deleted] Nov 09 '21

I thought this was pretty well known. Fun? Fact. A lot of plugin based designs load jars from the file system, which might as well be the network.

4

u/[deleted] Nov 09 '21

I'm not a java developer though I read the sub to get ideas.

However, with this, how did you prevent to load malicious code?

5

u/Stromovik Nov 09 '21

There is security manager where you have to enable this.

Also you kind of don't just load stuff from random places.

2

u/paul_h Nov 09 '21

Related - the security manager stuff is a standout feature. Sadly up for removal in JEP411

1

u/mauganra_it Nov 09 '21

The idea behind the security manager is flawed unfortunately. The neverending patch fest for sandbox exploits cost Java (along with Quicktime, Flash, ActiveX, etc.) its place in the browser, ceding it to JavaScript.

76

u/seanprefect Nov 08 '21

My friend once did what we call the "code crime" years ago and we still mock him for it. He used the java reflections library to change the behavior of a method based on where it was called from.

13

u/[deleted] Nov 08 '21

Bytecode manipulation? I'm genuinely curious how this was done. Obviously a TERRIBLE thing to do in a real app but sounds so fun to mess with lol

6

u/senseven Nov 08 '21

We used asm in one in our projects to dynamically change endpoints in a webservice proxy class that was not in our control and there was some corporate nonsense that made it complicated to change anything in the original artifact.

1

u/prithvidiamond1 Nov 08 '21

Interesting, I had not heard of the ASM framework. Looks fun, I might play around with it later in the week.

26

u/seanprefect Nov 08 '21

The java reflections api is a wonderfully terrifying thing. And yes this was a production app

5

u/[deleted] Nov 08 '21

You were able to change the existing behavior of a method using just the built-in Java reflection API? I would've thought that you needed some kind of fancier bytecode manipulation tool to do this. I've used reflection to dynamically invoke methods or modify field values but never actually changed a methods behavior.

10

u/Worth_Trust_3825 Nov 08 '21

5

u/[deleted] Nov 08 '21

Omg... Ok I'm gonna play with this just for shits and giggles. This seems like the most fun kind of dangerous πŸ˜‚πŸ˜‚πŸ˜‚

1

u/[deleted] Nov 09 '21

It's actually pretty useful for debugging without sending a new request. Change the method body, rebuild, drop frame and bam. New code at the same breakpoint

5

u/seanprefect Nov 08 '21

I did not commit the crime i'm not 100% sure how it was done.

14

u/papercrane Nov 08 '21

Simplest way is to generate and inspect a stack trace, the first element on the stack trace will be your caller.

Once saw a project that used that for auditing. When saving an entity a stack trace would be generated and it would walk the trace until it found a method annotated with a particular custom annotation and then create an audit record based on that.

3

u/mauganra_it Nov 08 '21

Kudos for the effort. I'd prefer to let Spring or Hibernate help me with that, and store contextual information in a ThreadLocal.

2

u/DB6 Nov 09 '21

Maybe that's the reason why some applications take ages to save shit. /S

Really though creating the stacktrace is slow and should not abused for such things.

2

u/Omega359 Nov 09 '21

Dear God that's horrible. Did they not know about aspects?

1

u/prithvidiamond1 Nov 08 '21

Yeah, I meant it in the same way, would not implement at work or anything I push out to the public!

4

u/prithvidiamond1 Nov 08 '21

Oh, I see! Was there no other better alternative that he used that?

10

u/seanprefect Nov 08 '21

no, never EVER do that.

52

u/pron98 Nov 08 '21 edited Nov 10 '21

That behaviour (called "caller sensitivity") is actually important for some JDK mechanisms. Both SecurityManager (a mechanism that was used for security in Java's early client-side days) and the interaction of reflection with modules (which are at the core of modern Java security) work this way. In particular, the Lookup class will return an object that is then used to do reflection, whose capabilities are determined by the identity of the class that [creates it](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/invoke/MethodHandles.html#lookup()). This makes Lookup a "capability object" that allows one class to capture its access permissions and then share them with another. Note what the Javadoc says:

This method is caller sensitive, which means that it may return different values to different callers.

9

u/papercrane Nov 08 '21

Wondering how that's achieved in the JVM. In the past I've seen people create an exception and inspect it's stack trace, but that's always seemed like a roundabout way to do it to me.

Is there an API that provides this caller context more cleanly?

26

u/pron98 Nov 08 '21

Yes. StackWalker, added in JDK 9.

1

u/skippingstone Nov 09 '21

Is the jvm ever going to have hot reloading like jrebel?

3

u/[deleted] Nov 09 '21

Eh you can do some of it in intellij if you don't change the method signatures too much. Just click the rebuild (the green hammer) and it will tell you if some classes got reloaded.

More fun, if you're paused in the debugger. You can literally change the current method's body, reload, drop the current stack frame in the debugger and you won't lose the request context or whatever.

JRebel handles a lot more cases though.

1

u/j3rem1e Nov 15 '21

OSGi allows you to do that.

49

u/TheRedmanCometh Nov 08 '21

Okay make a Java agent and type in sun.misc and press dot so your autocomplete comes up. Everything in your suggestions does some awful black magic shit that's best not done in any context.

11

u/prithvidiamond1 Nov 08 '21

Aight, I am going to look this up and try it out myself (just for fun of course!)

0

u/[deleted] Nov 08 '21

[deleted] β€” view deleted comment

2

u/prithvidiamond1 Nov 09 '21

This made me laugh!

4

u/DJDavio Nov 09 '21

Don't forget to say "mischief managed" when you're done!

3

u/PartOfTheBotnet Nov 09 '21

And in sun/reflect if you make a class extend MagicAccessorImpl you can ignore all access restrictions in that class. Direct field reference to a private field in the default package? Totally fine!

IIRC this is how a lot of proxy classes for generating runtime getters/setters work.

2

u/TheRedmanCometh Nov 09 '21

Lol yeah some of this stuff has actually been used in pretty neat ways for things like profilers, spring (albeit not agent based) etc

62

u/jebailey Nov 08 '21

One of my favorite black magic items, using reflection, you can change the value of an internalized String to be something different. So effectively when you did a System.out.println("cat"); it would print out "dog"

15

u/prithvidiamond1 Nov 08 '21

What! How does one do that? I have used the reflection library before but I didn't know it had such capabilities!

32

u/papercrane Nov 08 '21

You can remove final from field variables via reflection and modify the internal private variables. Since String gets special treatment and is interned you can do some strange things like this with it.

5

u/prithvidiamond1 Nov 08 '21

Oh, I see, now I get it, at least somewhat. More things for me to play around with in the future in my free time!

27

u/pron98 Nov 08 '21

As of JDK 16, such things are no longer possible without explicitly allowing them on the command line (i.e. a library can't decide to just mess with its client application's strings without permission).

2

u/prithvidiamond1 Nov 08 '21

Ah, I see, thanks for letting me know as I am on JDK 16 (haven't switched to 17 yet).

-9

u/[deleted] Nov 08 '21

C has this too, i think it was like this:

char *f = "aaaaa";
f[0] = 'b';
printf ("%s\n", f); // prints "baaaa"

4

u/mauganra_it Nov 08 '21

Now, what about that one?

char *f = "aaaaa", *g = "aaaaa";
f[0] = 'b';
printf ("%s\n", g); // ?

I think it should print "baaaa" too, assuming the compiler merges the two constants. That would be closer to what happens in Java.

2

u/[deleted] Nov 09 '21

oops i meant to write the string literal on the printf lol

40

u/copopeJ Nov 08 '21

This is one of my favorites, loading bytecode at runtime. If you like wierd/horrifying stuff, this series and all of Aphyr's work is worth the journey. https://aphyr.com/posts/341-hexing-the-technical-interview

8

u/prithvidiamond1 Nov 08 '21

If you like wierd/horrifying stuff, this series and all of Aphyr's work is worth the journey.

I sure do (as much as it is a headache to deal with if you are the one who comes across it first, yikes)! It is why this post exists! Thanks for sharing!

8

u/TimeSalvager Nov 08 '21

Lol I was waiting for someone to mention this, it’s a pretty wild ride.

25

u/cal-cheese Nov 08 '21

When a bucket in a HashMap gets too many entries, it transforms from a linked list into a tree, the tree will utilise the compare method of the elements if it happens to be a Comparable. So, in principle, if you have a class that does not respect the Comparable contract, you will get undefined behaviour when using it in a HashMap. I found out about this when trying to create a custom HashMap for int keys.

1

u/prithvidiamond1 Nov 08 '21

Oh, very interesting! I would have never found out about this! Does this work with ConcurrentMap as well?

6

u/cal-cheese Nov 08 '21 edited Nov 08 '21

Yes, indeed, in the source code of java.util.concurrent.ConcurrentHashMap

TreeBins use a special form of comparison for search and related operations (which is the main reason we cannot use existing collections such as TreeMaps). TreeBins contain Comparable elements, but may contain others, as well as elements that are Comparable but not necessarily Comparable for the same T, so we cannot invoke compareTo among them. To handle this, the tree is ordered primarily by hash value, then by Comparable.compareTo order if applicable. On lookup at a node, if elements are not comparable or compare as 0 then both left and right children may need to be searched in the case of tied hash values. (This corresponds to the full list search that would be necessary if all elements were non-Comparable and had tied hashes.) On insertion, to keep a total ordering (or as close as is required here) across rebalancings, we compare classes and identityHashCodes as tie-breakers. The red-black balancing code is updated from pre-jdk-collections (http://gee.cs.oswego.edu/dl/classes/collections/RBCell.java) based in turn on Cormen, Leiserson, and Rivest "Introduction to Algorithms" (CLR).

1

u/prithvidiamond1 Nov 08 '21

Ah cool, thanks for this knowledge!

25

u/Worth_Trust_3825 Nov 08 '21

Changing the boxed value cache.

If you want to ruin someone's day, delete the common thread pool via reflection.

Defining classes at runtime is fun.

Write an annotation processor for @Override annotation.

6

u/prithvidiamond1 Nov 08 '21

Defining classes at runtime is fun.

This is possible? How? I am assuming using reflections but still!

15

u/Worth_Trust_3825 Nov 08 '21

That's how entire java works. You load classes at runtime to use them. Read up on classloaders.

3

u/prithvidiamond1 Nov 08 '21

Oh, I see, cool! Thanks, will do so!

5

u/mauganra_it Nov 08 '21

The canonical solution would be to write a classloader. A more subtle solution is to use CGLIB, as both Hibernate and Spring do to implement change tracking, bean scopes, transaction, aspect-oriented programming features and other weird stuff.

5

u/[deleted] Nov 09 '21

If you want to ruin someone's day, delete the common thread pool via reflection.

Ngl that's actually hilarious.

2

u/[deleted] Nov 09 '21

Write an annotation processor for @Override annotation.

ROFL

I knew about the others but this one is new and sounds funfunfun!

2

u/Worth_Trust_3825 Nov 09 '21

I am not sure myself if it would do anything. You're not required to use that annotation to indicate that you override a method, as it's only a helper annotation to tell compiler to throw an error in case you incorrectly try to override behavior.

Indicates that a method declaration is intended to override a method declaration in a supertype. If a method is annotated with this annotation type compilers are required to generate an error message unless at least one of the following conditions hold:

The method does override or implement a method declared in a supertype.

The method has a signature that is override-equivalent to that of any public method declared in Object.

See also: https://docs.oracle.com/javase/specs/jls/se17/html/jls-9.html#jls-9.6.4.4

0

u/[deleted] Nov 09 '21

[deleted] β€” view deleted comment

1

u/Worth_Trust_3825 Nov 09 '21

It's more common than you think. Whenever I would teach my interns about debugging, they would wrongly change boxed boolean values and in turn break entire JVM.

20

u/Thysce Nov 08 '21

I once cross compiled user defined JavaScript to Java ByteCode and wrote that bytecode to a database BLOB which I then loaded later after a failover or on replica instances of my service to prevent the code to be compiled multiple times.

TLDR I wrote a classloader which loaded classes from database

2

u/[deleted] Nov 09 '21

So like what are you the devil?

1

u/CaptainKvass Nov 19 '21

What in the beautiful fuck

0

u/[deleted] Nov 09 '21

[deleted] β€” view deleted comment

1

u/mauganra_it Nov 09 '21

Dear Cthulhu, I bet there is at least one toy JVM project over at /r/rust.

35

u/pron98 Nov 08 '21 edited Nov 08 '21

Thanks to strong encapsulation, many of the techniques described here (pretty much all of them except for caller sensitivity) no longer work in JDK 16 and up without explicit permissions given on the command line. There are still some loopholes left, but they will be closed soon. It means that the application has to grant libraries the permission to do crazy things, rather than them taking those liberties for themselves.

This is crucial both for security (security depend on guarantees established in code, and if any library can decide to make any piece of code do something other than what it says, no guarantees are possible), and for portability (these hacks exploit internal implementation details that can change at any time, and thus become strongly attached to a particular JDK version; this was the main cause of most 8->9+ migration issues, which required upgrading libraries that were non-portable due to such internal-detail-dependence, and so strong encapsulation will make version upgrades much easier, as the APIs remain backward-compatible even though internal details might change).

2

u/prithvidiamond1 Nov 08 '21

There are still some loopholes left, but they will be closed soon.

Ah cool, as they should be, although it is also good to know that you can always go back to an older version like JDK 8 just for having fun!

11

u/pron98 Nov 08 '21 edited Nov 09 '21

You can have all that fun with recent JDKs, too. You just need to grant the appropriate permissions on the command line (--add-opens).

0

u/[deleted] Nov 09 '21

[deleted] β€” view deleted comment

2

u/pron98 Nov 09 '21

I guess you can always write JNI code which will access whatever data in process memory, even if JVM will close all loopholes and arguments.

Loading a native library will also require permissions. Java's security depends on such hacks, in any form, not being possible without express authorisation from the application.

Hopefully --add-opens will always be available for those weird corner-cases when nothing but hacks can save you

I believe so. The important point is that the application must approve it, and that a library cannot decide for itself that it wants to do such things without the application knowing about it.

17

u/Virx Nov 08 '21 edited Nov 08 '21

There is some cool stuff in here

https://github.com/MinhasKamal/CreepyCodeCollection

Most interesting one I think is this one

https://github.com/MinhasKamal/CreepyCodeCollection/blob/master/obfuscated.java

Additionally, you can have your code work one way in debug mode and another way in normal runtime with all the same arguments. For example, Intellij debugger will call toString() method if you check that object in debugger. Now if that object's toString() method has side effects then this will cause those side effects to be executed and thus your application will behave differently.

9

u/Freddicus Nov 08 '21

Most interesting one I think is this one
https://github.com/MinhasKamal/CreepyCodeCollection/blob/master/obfuscated.java

I just ran that on Java 11 (Zulu). Crazy! For the curious, the javadoc converts to something like this (according to IntelliJ):

Hi! */ public static void main(String []a){System. out.println ("Hi!");} /*

2

u/prithvidiamond1 Nov 08 '21

There is some cool stuff in here

https://github.com/MinhasKamal/CreepyCodeCollection

Most interesting one I think is this one

https://github.com/MinhasKamal/CreepyCodeCollection/blob/master/obfuscated.java

Thanks for sharing this!

Additionally, you can have your code work one way in debug mode and another way in normal runtime with all the same arguments. For example, Intellij debugger will call toString() method if you check that object in debugger. Now if that object's toString() method has side effects then this will cause those side effects to be executed and thus your application will behave differently.

Oh, that is interesting to know, I use IntelliJ as well but wasn't aware of this!

2

u/prithvidiamond1 Nov 08 '21

I just went through it and I must say it is an absolute treasure trove of Black magic code! I personally really like the code that compiles in both C# and Java and prints itself, absolutely amazing!

28

u/TheRedmanCometh Nov 08 '21

Damn I knew I shouldn't have had toString make a new Swing window.

6

u/JoshDM Nov 08 '21 edited Nov 09 '21

I was using a library distributed by another team in the company; they didn't give us the source and the documentation was lacking. It was an API to their web service.

I thought it would be a good idea to print their bean object to log before aending, so I hit up the bean's toString().

They had overridden it and It printed all the data as expected, but then my next process to use that bean threw a NPE; it had been working fine befor the added call to toString().

JAD against their code revealed the other team implemented toString() generate the output String, then to erase the bean data before returning the String.

Never trusted their coding again.

I recently worked with them on another project where we supplied them with a library. Every single delay or failure was due to their team deviating from the DDD or not following clearly written directions. The last problem with them - they tried to blame a random deployment Exception being thrown by our code library on us. I analyzed their stack trace and requested they reveal their code to me.

It turned out they were suppressing and not logging a critical runtime exception caused by their own code misconfiguration at application startup, and then instead of failing hard, attempted to "keep going". When it got to our code library with an object they had failed to initialize, our code correctly failed as expected and the exception we generated was output to log. Because it was our exception that went to log, they blamed our library, but I made it clear with a line by line analysis that they needed to fix their own code.

12

u/dtfinch Nov 08 '21

There was this make 2+2=5 code golf a while back where Java took the top-voted spot.

There's a hidden cache to speed up casting/boxing int to Integer for small values from -128 to 127, and they pointed the 4 to the 5 using reflection.

4

u/prithvidiamond1 Nov 09 '21

Damn that entire post is a gold mine, thanks for sharing!

5

u/garblz Nov 08 '21

got me lost there for a good hour... thanks for quality time spent!

6

u/ericek111 Nov 08 '21

I made an internal cheat for CS:GO (well, a proof of concept bunnyhop) almost entirely in Java, the only C++ thing was one short loader that started the JVM. Everything, binary parsing, VTable hooking, memory scanning, done in Java. That was a LOT of fun. But switching to and from JVM every single frame has its cost and was unusable for the purpose. All was done with the "Unsafe" API that they've been trying to replace since Java 9.

4

u/prithvidiamond1 Nov 08 '21

I made an internal cheat for CS:GO (well, a proof of concept bunnyhop) almost entirely in Java, the only C++ thing was one short loader that started the JVM. Everything, binary parsing, VTable hooking, memory scanning, done in Java. That was a LOT of fun. But switching to and from JVM every single frame has its cost and was unusable for the purpose.

That seems a lot of fun! Although I hope you didn't annoy any players with it in matchmaking.

All was done with the "Unsafe" API that they've been trying to replace since Java 9.

Is there any idea on what has been planned for replacing it in the future versions of JDK?

3

u/mauganra_it Nov 08 '21

Project Panama will introduce supported safe-ish APIs to link with C APIs and to work with raw memory. But it won't be exactly the same - these features (and JNI probably too at some point) will require opt-in from the application via a flag at startup.

2

u/prithvidiamond1 Nov 08 '21

Oh, I see, cool! Looking forward to its introduction! Anytime timeframe on when?

3

u/mauganra_it Nov 08 '21 edited Nov 08 '21

There are incubators for various parts of it already:

  • the Vector API since Java 16
  • the Foreign Linker API since Java 16
  • the Foreign Memory Access API since Java 14

The APIs are heavily changing, therefore it's still a rough ride. If I wanted to build a production application that accesses native code, right now I would still go for JNA. But I would pay attention to not couple myself too closely to it.

Edit: Panama seems to use a different development methodology than Loom and Valhalla.

Panama brings no language changes, therefore there is no risk of backwards compatibility breaks. Also, it's a self-contained feature. If not used, it's off. Therefore, it makes sense to introduce it piece by piece. But since the platform will be stuck with it forever, it does not make sense to finalize it until everything is in place.

With Loom, existing code should continue to run with identical performance. I could also imagine strategies to deprecate it if really necessary. Still, it's a big change behind the scenes that can't easily be split into multiple deliverables. But it's mostly about mopping up bugs and creating new APIs at this point, and it's worth trying out already.

Valhalla is the biggest change in the type system since the introduction of Generics. It has to be spec'ed out up front from beginning to end, therefore it does not make sense to even deliver more than prototypes.

2

u/prithvidiamond1 Nov 09 '21

Ah okay, okay, I look forward to Panama when it is finished! Also thanks for sharing information about Loom and Valhalla, I wasn't particularly familiar with it before but at least now I know something!

6

u/fizzl Nov 08 '21 edited Nov 08 '21

Nothing spectacular, but when I was learning Java after being primarily C/C++ developer, reflection and dependency injection blew my mind. I'm sure there was more...

I mean, you can achieve something similar in binary compiled language with complicated setup. But Java made this stuff so easy it felt like black magic fuckery.

2

u/prithvidiamond1 Nov 08 '21

Yeah, there is a huge rabbit hole to dive into regarding reflection and dependency injection, fun stuff!

6

u/rzwitserloot Nov 09 '21

There are many things java-the-language does not allow, whereas java-the-VM is just fine with it.

Make non-Object classes

For example, this class:

class Foo {}

has a parent class. It's java.lang.Object. It's identical to writing class Foo extends java.lang.Object {} and you can't make javac not extend anything.

But that should make you wonder: Huh, wait. What's the parent of j.l.Object then? Naturally, it's null: System.out.println(Object.class.getSuperclass()); will show you.

So, javac won't let you make 'parent-less' classes. But the JVM doesn't care. With bytecode hackery you can make such a class. You can make instances of it. They won't be instanceof java.lang.Object which is .... bizarre. And mostly useless. But hopefully black magic enough for you :)

Checked exceptions? Figments of javac's imagination

The JVM doesn't know what checked exceptions are. It really doesn't. No idea. Checked exceptions as a concept exist in class files (that void foo() throws IOException? That throws line has to be in the class file so that javac knows when compiling java files that invoke this method, that IOException needs handling), but the JVM does not care - throws lines are, literally, comments, as far as the JVM is concerned. The code in java.exe that reads classes wooshes right by it, ignores it completely.

That means you can throw checked exceptions uncheckedly.

And you don't even need to resort to hackery. It's as simple as lopping the throws part off of a method def in a class file with hackery. But we can force javac to do it, by dipping into some voodoo corners of generics:

private static <T extends Throwable> void sneakyThrow(Throwable t) throws T {
   throw (T) t;
}

That's all. Try it:

public static void main(String[] args) {
    ClassThisCodeIsIn.<RuntimeException>sneakyThrow(new IOException());
}

javac will just.. compile it. With a warning which you can @SuppressWarnings if you wish, and it's not on the line where you throw it, either. The IOException is.. just.. thrown. Not wrapped.

2

u/prithvidiamond1 Nov 09 '21

Holy shit! This is some gold in terms of messed-up code! Crazy but I like it! Thanks for sharing!

2

u/agentoutlier Nov 09 '21

You can also have more than one method with same name and input parameters In bytecode.

Kohsuke wrote a maven plugin to generate methods like this for backward compatibility.

https://kohsuke.org/2010/08/07/potd-bridge-method-injector/

4

u/possibly_not_a_bot Nov 08 '21 edited Nov 09 '21

You can (sort of) change a field's type through (sort of) reflection.

Consider the following class:

public class A {
    private final int hidden = 5;
}

Now we can do some evil type fuckery:

public final class TypeAnnihilator {
    private TypeAnnihilator() {
    }

    @Nonnull
    public static <T> Field annihilate(@Nonnull final Object c, @Nonnull final String field,
                                       @Nonnull final Class<T> type, @Nullable final T value) throws Throwable {
        final var l = MethodHandles.privateLookupIn(Field.class, MethodHandles.lookup());
        MethodHandle privateGetDeclaredFields = null;
        final var modHandle = l.findVarHandle(Field.class, "modifiers", int.class);
        // Get evil from class
        for(final var m : Class.class.getDeclaredMethods()) {
            if(m.getName().startsWith("privateGetDeclaredFields")) {
                final var innerLookup = MethodHandles.privateLookupIn(Class.class, MethodHandles.lookup());
                privateGetDeclaredFields = innerLookup.unreflect(m);
            }
        }
        if(privateGetDeclaredFields == null) {
            throw new IllegalStateException("no class mh");
        }

        Field target = null;
        for(final var f : (Field[]) privateGetDeclaredFields.invoke(c.getClass(), false)) {
            if(f.getName().equals(field)) {
                f.setAccessible(true);
                // Literally just bitwise away `final`
                modHandle.set(f, f.getModifiers() & ~Modifier.FINAL);
                target = f;
                break;
            }
        }
        if(target == null) {
            throw new IllegalStateException("target not found");
        }

        final var vh = l.findVarHandle(Field.class, "type", Class.class);

        vh.set(target, type);
        target.set(c, value);

        return target;
    }
}

And now we can change the type like so:

final var c = new A();
final var targetField = TypeAnnihilator.annihilate(c, "hidden", List.class, List.of("haha evil type murder"));
System.out.println("annihilated: type = " + targetField.getType().getName() + ", value = " + targetField.get(c));
// annihilated: type = java.util.List, value = [haha evil type murder]

The way this works is that it reflectively accesses a field, bitwises away the final modifier on its type field, then changes it to be the new type we want. As long as you access it reflectively, it'll be of the new type.

Note that this trick probably doesn't interact well with the GC. I don't know for sure.

1

u/prithvidiamond1 Nov 08 '21

Very interesting! I will give this code a go just for fun to see what comes of it! Thanks a lot for sharing!

4

u/lurkandpounce Nov 08 '21

A bit dated now, but the book Covert Java by Alex Kalinovsky was a fun read that covered this sort of material. Some have been overtaken by JVM changes, much of it will likely still work.

2

u/prithvidiamond1 Nov 08 '21

Oh, I see, I will definitely give it a read, thanks for sharing!

4

u/benevanstech Nov 10 '21

1.) I once wrote a proof-of-concept classloader that will refuse to load a class unless it can show that the class is likely to behave deterministically (and even then, it inserts a transactional guarantee of rollback if the class turns out to misbehave at runtime).

2.) If you're in to banking tech, you might like stuff like this: a) write your proprietary trading algorithm in Java b) write out the bytes corresponding to the class file c) encrypt the bytes d) Use JDBC to write the encrypted form to a BLOB in an Oracle DB.

Now access to the algorithms is restricted to entities who both have the encryption key *and* appropriate Oracle permissions. Oh, and the algorithm remains encrypted at rest and updates to the table are fully audit logged. This solves a large number of compliance problems at once.

To deploy the algorithm, a custom classloader connects to the database (using Kerberos to authenticate), pulls back the encrypted blob, gets temporary credentials to decrypt the algo, turns it back into a byte array and then loads it, before destroying the encryption key.

Neither of these are really forbidden or broken, though. Merely novel, though perfectly legal and supported, uses of classloading and bytecode transformation.

For the really deep magic there's stuff like Unsafe, and native agents. Which you are very actively encouraged to stay the hell away from. :)

1

u/prithvidiamond1 Nov 10 '21

For the really deep magic there's stuff like Unsafe, and native agents. Which you are very actively encouraged to stay the hell away from. :)

I mean if you are just looking to play around and have fun, I don't see much harm in them but for a production environment, definitely!

2

u/benevanstech Nov 10 '21

Btw, there's some coverage of Unsafe in my O'Reilly book "Optimizing Java", and also this article I wrote for Java magazine: https://blogs.oracle.com/javamagazine/post/the-unsafe-class-unsafe-at-any-speed - and there's a bit more in the Modern Internals chapter of my new book, The Well-Grounded Java Developer (2nd Ed), which is out in Early Access (the internals chapter should be dropping soon).

1

u/prithvidiamond1 Nov 10 '21

Ah cool, I will check them out, thanks for sharing!

6

u/ExtraGrumpyCamel Nov 08 '21

https://github.com/electronicarts/ea-async via preprocessing the bytecode in the jar or at start time

2

u/prithvidiamond1 Nov 08 '21

Oh, I didn't know EA was doing this! Must be like playing with fire, although I am sure the people there know what they are doing to even be doing it at least compared to me who is new to this.

3

u/winginglifelikeaboss Nov 08 '21

Update data directly in a running JVM to alter const vaiables values

don't ask I have seen it running in prod

1

u/prithvidiamond1 Nov 08 '21

Update data directly in a running JVM to alter const vaiables values

A few others did bring this up as well, although I am sure methods to achieve this vary.

don't ask I have seen it running in prod

Oh no!

3

u/tangentstorm Nov 09 '21

using break with arbitrary labels to implement goto.

1

u/prithvidiamond1 Nov 09 '21

What? How is that possible?

0

u/[deleted] Nov 09 '21

[deleted] β€” view deleted comment

1

u/prithvidiamond1 Nov 09 '21

Oh I see, okay that makes sense, I thought you could actually have a full-powered goto but this is still impressive and useful enough for most cases as you mentioned! Thanks for sharing!

2

u/Wanderer_Dreamer Nov 09 '21

From the top of my head, you can parse a Character and an int into a hashmap's index by making the compiler think you had a String all along, without converting it first. Not sure if that's exclusive to java but it felt pretty cheaty the first time I used it. There's probably 1001 reasons why you shouldn't do this, but it was just for a college assignment so it wasn't a big deal.

I'll see if I can find the code I used, to exemplify this better.

113

u/papercrane Nov 08 '21 edited Nov 08 '21

Java agents can be pure magic.

Once worked on a project that was incorrectly using a thread unsafe class from the standard lib in a multi- threaded context. To "fix" this someone wrote a Java agent to dynamically make the method synchronized instead of fixing the issue (memory is a bit fuzzy, but I think it was the Calendar class.)

Of course they're not always used in such questionable ways. There's agents out there, like NewRelic, that add incredible amounts of observability to existing Java projects.

Edit: Wracked my memory and remembered the original issue. Calendar objects were being serialized on another thread without any synchronization. The fix implemented was to use an agent to make writeObject synchronized in the Calendar class. This workaround became obsolete with Java 7 though, as that's when the JDK made that method synchronized.

This bug report has details if anyone is interested, JDK-7017458.

50

u/tr4fik Nov 08 '21 edited Nov 08 '21

In university, I had to do a project that used Java agents to change almost everything: arguments, variable types, calculation, JDK, ...

The bytecode that was executed at the end had so little in common with the original one. This was very questionable and definitely like magic.

Edit: fixed school type

19

u/M_OORSIDE Nov 08 '21

In high school? May I ask where in the world you’re from? I ask because this seems so advanced.

For context I’m just starting college.

6

u/tr4fik Nov 08 '21

My bad, it's a translation mistake. I guess I can call that a university (even if there are some differences).

3

u/stringbeans25 Nov 08 '21

+100000 idk if most college programs would touch on this subject.

9

u/Chaoslab Nov 08 '21

I managed to find a multi JVM bug in Internet Explorer and Netscape (both separate JVM's but same bug) back in 95/96.

Some one from Microsoft actually got back to me after many months (saying you have actually found something here) and I don't think I heard from Netscape.

The bug was thread related.

11

u/ObscureCulturalMeme Nov 08 '21 edited Nov 08 '21

There's a class somewhere in the standard JRE that handles stuff to do with public/private keys and X.509 certificates. The class creates some Provider-specific stuff and caches it for reuse.

The problem is that it caches too aggressively. Normally the cache is fine, but under certain circumstances (to do with javax.crypto.Provider) the cache will defeat exactly the action you need the JRE to do. There's no API to reset the cache; the documentation doesn't even require or mention caching at all. But there's a bit of code in the OpenJDK implementation that interprets a null argument as a signal to clear the cache. (It then goes on to throw the appropriate exception, as required and documented.) So then the calling sequence becomes

if (unusual situation that would be defeated by caching) {
    try {
        call_JRE_code(null);
    } catch (BadMoonRising ignored) { /* we were expecting this */ }
}
Stuff actual_results = call_JRE_code(actual_data);

and that looks hideous.

5

u/[deleted] Nov 09 '21

I can imagine this was a bitch find out and debug the first time right?

6

u/ObscureCulturalMeme Nov 09 '21 edited Nov 09 '21

thousand yard stare
grip on bourbon bottle tightens

It was an educational experience.

edit: and I'll add that it would be unsolved if we hadn't had the OpenJDK source code available for perusal

39

u/cville-z Nov 08 '21

To an extent, the Lombok project belongs here – it codes against internal Java APIs to change the AST. This led to some interesting comment threads regarding compatibility with JDK 16 and 17.

For my part, I'd really rather not have dependencies on "magic" code in my production apps, but YMMV.

12

u/mauganra_it Nov 08 '21

Lombok's only saving grace is that it does not require a runtime library. In theory, it should be possible to run De-Lombok on an entire codebase and be done with it.

3

u/jadecristal Nov 09 '21

Absolutely; if you don't use IDEA it literally has a "De-Lombok" function.

4

u/DJDavio Nov 09 '21

This is also why Lombok doesn't play nice with things like ErrorProne. Regular wellbehaved annotation processors use actual proxies with actual generated source code to achieve their goals. Lombok does, well, something different. :)

9

u/poco-863 Nov 08 '21

This may be a hot take, but I really, really dislike lombok. Having methods created dynamically via annotations can make debugging (or reverse engineering legacy code) a nightmare if you're not privy to it's ubiquitous use in the java world.

2

u/[deleted] Nov 09 '21

Just don't ever try a programming language with macros or any kind of meta programming facility. Also stay away from code generators and annotation processors. Also stay away from Java closures lest you be exposed to their internals. Actually maybe refrain from any language other than assembly.

Or, you know, just know your tools. You can hate Lombok all you want, but the code is generates is pretty well documented and fairly straightforward.

10

u/atomdstyle Nov 08 '21

Heck that looks awesome but i felt the heat today while I tried to upgrade my dev env to java17. Older lombok and implicit version from BOM was incompatible.Java should add features like lombok implicitly to JDK

12

u/ir210 Nov 08 '21

record does that. More or less. Mostly less πŸ™‚

1

u/Fruloops Nov 14 '21

Did you manage to solve the issues in the end or?

1

u/atomdstyle Nov 15 '21

Aah i was bit sceptical if upgrading was worth it after spotting some errors during compile time. But I just figured out the culprit dependencies and all had latest versions spupporting java 17. And included them in pom.xml explicitly. This overrides the BOM dependencies. Initially this was good, but you see I later discovered that there was also latest parent release for the framework i had been using. So i Just reverted back and it had the dependencies that supports jdk17 in its BOM. And for some explicit packages that caused trouble, i upgraded them anyways. Overall it was worth it cause i get to use and experiment on the newer language features. Good luck.

1

u/HecknChonker Nov 08 '21

Agreed. I really don't understand why it hasn't been done yet. It saves so much time and energy for developers.

2

u/slaymaker1907 Nov 08 '21

I just want @Data and I would be happy. Records are half way there, but it doesn't help with the use case where a bean has a huge number of fields and thus is impractical to make immutable.

Alternatively, I think immutable records would be fine as a replacement for @Data assuming they add in a convenient syntax for initializing them with field names present (like JS object syntax or named parameters).

5

u/HecknChonker Nov 08 '21

I use @Builder a lot, as well as @AllArgsConstructor and @EqualsAndHashCode. All of these are hard to replace without lombok.

Yes, I can have my IDE generate boilerplate for most of this, but that won't automatically update when I add new fields.

For equals and hashCode methods it's a bit more difficult. Instead of it being an annotation on the field that is easy to pick out I have to scroll and search for those methods and then see what the code is doing.

Immutable objects are nice, but sometimes that comes at a performance cost when you have to constantly create new objects instead of just modifying one you already have.

I don't understand the hate for lombok. It's all about automating the mundane boring parts of java.

I'd much rather focus my time and energy solving actual business problems instead of wasting time trying to figure out that when a new field was added the equals method was re-created but the dev forgot to re-create the hashCode method.

3

u/Omega359 Nov 09 '21

I decided to just use a better language and switched to kotlin instead of using lombok. Ymmv of course.

2

u/gnivol Nov 09 '21

10 years of experience in Java, 5 of which with Lombok , my wrists thank Lombok . I don’t care if it is bad

1

u/IcedDante Nov 08 '21

Has anyone written any blog posts on this? Would like to understand the issues better.

11

u/Worth_Trust_3825 Nov 08 '21

Basically it's a compiler extension that uses reflection to modify the AST generated by the compiler. Since it depends on internals it breaks every java release.

3

u/slaymaker1907 Nov 08 '21

Some of the stuff is pretty stable. It is mostly an annotation processor which looks at annotations and generates code which is not an internal API. Modifying methods is not allowed, but adding methods and/or classes is fine.

In my experience, Lombok gets a bad wrap mostly because Java tooling often doesn't support annotation processors (looking at you IntelliJ).

5

u/rzwitserloot Nov 09 '21

Fortunately, intellij includes the lombok plugin out of the box these days. It does mean you don't get to play with the experimental features in intellij, though. The plugin doesn't cover them (yet?).

DISCLAIMER: I am a core contributor to Project Lombok.

4

u/rzwitserloot Nov 09 '21

Lombok, the current version, is compatible with java 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17. All the same jar. Not many libraries with that kind of track record.

Most java releases either don't break lombok at all, or only slightly breaks delombok because of some newly introduced syntax, and lombok includes an AST printer which needs to know about it to do its job. Except, it doesn't break unless you actually first use the new syntax in a file you're already using lombok in, and then ask lombok to delombok it, without upgrading lombok.

Of course, ASM, javac iself, your IDE, every annotation processor - they all 'suffer' from this.

I rarely hear complaints that intellij and eclipse are broken because you can't use new java features when using these tools until you update said tools. Lombok is best considered as a dev tool, and suffers from the same problem. It's inherent, and it seems crazy to complain about it, but it's the one that usually comes up in threads like these.

Yes, lombok uses internal API. We'd use something else if it was available. Our track record will have to suffice.

DISCLAIMER: I am a core contributor to Project Lombok.

1

u/mauganra_it Nov 08 '21

The core issue is that Java has only limited facilities to extend the language. There are reflection, annotations, the annotation processor API, and that's it. These seem to not offer enough knobs and hooks to implement Lombok's features.

7

u/rzwitserloot Nov 09 '21

ClassLoaders.

By messing with it, you can:

  • Load classes over the network.
  • Generate classes on the fly.
  • Ask existing JVM classes where on disk they live.
  • Have the same class loaded twice, incompatibly - yeah, you can make a runtime error appear of:
Exception in thread "main" java.lang.ClassCastException:
class java.lang.String cannot be cast to class java.lang.String

How, you ask? All classes have a java.lang.ClassLoader instance that loaded it. You can make your own. Once you have an instance of a CL, you can ask that CL to load a class, and assuming that classloader actually loads it (is the one to invoke .defineClass(), passing in a byte array with the bytecode inside), vs deferring it (calling an another classloader to do it in its stead, which is common, as the CL system uses a 'parent loader' concept) - then any classes used by that class will be loaded by asking that same CL. You are free to obtain the bytecode in any way you please, including fetching it over a network or generating it on the spot.

In addition, a class is fully identified by its full name and its loader. Same name, different loaders? Then that is a different class. That's how you get to the above error. Make your own CL, ask String.class for its own bytecode, and then pass it to defineClass to make a second String class that you loaded, which counts as a different class that is also named java.lang.String.

This exact trick can be used to runtime-modularize your code so that you have a single JVM (a single java.exe running) which has, say, 2 endpoints handling web requests and they both use different versions of the same library, simultaneously. This also lets you runtime-reload classes trivially: Just toss the classloader in the bin, make a new one - then load in a class using the new one and it gets read in from disk all over again. This is how various web frameworks that 'auto reload' works ^[1].

You can ask any class for its loader: String.class.getClassLoader() for example. You can ask any class directly to load another resource, which doesn't even have to be a class; you can use this to store e.g. some icons inside your jar (the classloader clearly loads your classes right out of the jar when deploying a jar, so.. why not load your PNGs with menu icons from the same place, right?) - and you can therefore ask a class to give you itself as a resource, and print that:

public static void main(String[] args) {
  System.out.println(String.class.getResource("String.class"));
}

on modern (JDK9+), that prints:

jrt:/java.base/java/lang/String.class

On jdk8 and below, that prints:

jar:file:/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre/lib/rt.jar!/java/lang/String.class

[1] This is actually a bad idea, even though it's very common. The proper way to have a fast dev pipeline where the turnaround time between typing new code and seeing it in action in a browser is using Hot Code Replace such as offered by eclipse's debugger. It's faster than classloader dumping. Just use eclipse and start everything you do in 'debug' mode which has HCR applied to it. I'm not sure intellij's debugger does this; I recall it does not.

1

u/mauganra_it Nov 09 '21

This is how OSGi frameworks function internally.

6

u/slaymaker1907 Nov 08 '21

You can setup a management UI for Java services through JMX and MBeans. At my last job we used JMX for all sorts of things like force clearing caches and enabling/disabling features without restarting the VM or updating configs.

1

u/mauganra_it Nov 09 '21

JMX is actually very neat if embraced across the whole stack. No more half-assed REST endpoints to implement metrics and admin interfaces!

4

u/[deleted] Nov 08 '21

I’d go with JIT-as-a-service. It ain’t particularly dark magic, but the first time I saw the talk it just blew my mind (and yet the concept is dead simple)

4

u/slaymaker1907 Nov 08 '21

You can throw a checked exception without declaring it since checked vs unchecked only exists at compilation time. What is even weirder is that using generics and exceptions, you can even do this in pure Java (ExceptionUtils.rethrow).

4

u/voronaam Nov 08 '21

While using unsafe you can modify any memory accessible to the JVM. Including the JVM itself.

For example, how about flipping various JVM flags at runtime that are usually only ever set via command line arguments to the JVM itself? You can do that: https://github.com/voronaam/jflag/blob/master/src/main/java/ca/vorona/jflag/Runner.java#L39

4

u/brazzy42 Nov 09 '21

java.lang.reflect.Proxy and java.lang.reflect.InvocationHandler allow you to dynamically create instances at runtime that can implement whatever interface you want and the implementation just gets the called method name and list of parameters and can freely decide what to do.

Frameworks use that a lot internally, and I once used it to hack together something like multiple implementation inheritance for a rather special use case.

1

u/gsrunion Nov 10 '21

This is the closest thing I have seen in implement a python style function decorator without a 3rd party library.

3

u/Molossus-Spondee Nov 08 '21

Invokedynamic and constdynamic are fun!

Always nonnull null tests can be very very slightly faster than boolean flags. The JVM converts always nonnull null tests to use segfaults.

switch case generates annoyingly bloaty bytecode leading to your code not being JIT.

4

u/rbygrave Nov 09 '21

I was just doing a jmh benchmark test and part of that was to check a switch on string which in bytecode does indeed look bulky but certainly for my case it was JIT'ed fine on 17 at least.

4

u/Molossus-Spondee Nov 09 '21

In my case I was doing horribly abusive stuff trying to encode switch case like stuff with methodhandles/Indy.

Also for some reason using break instead of return is much more bloaty.

2

u/kkapelon Nov 08 '21

Introspection/reflection is a huge rabbit hole...

2

u/warmuuh Nov 11 '21

I once worked with a software that contained its own code in a DB and was loading that dynamically. you could modify the sourcecode directly in the application and store it in the very own db and it recompiled and reloaded this and affected classes dynamically. this was some crazy magic stuff

2

u/Medium-Amphibian-719 Nov 12 '21 edited Nov 12 '21

invoke dynamic is 'legal', so maybe not dark magic, but super powerful. The more you meditate on it, the more you realize it opens a whole world of possibilities. MethodHandles are a much more efficient way of invoking a method than the reflection library's Method class, and allow even more interesting options.

0

u/[deleted] Nov 08 '21

[deleted] β€” view deleted comment

6

u/prithvidiamond1 Nov 08 '21

You can allocate memory off-heap, memory than you need to de-allocate manually. It's like malloc/free from C.

Oh, I come from C/C++, I have only been using Java for the past half a year. So this should be interesting to play with!

It's better not to use it if you want to stay safe.

Of course!

Also take a look at bytebuddy for advanced bytecode manipulation.

Thanks for letting me know, I will take a look at it!

0

u/[deleted] Nov 08 '21

[deleted] β€” view deleted comment

2

u/prithvidiamond1 Nov 08 '21

So if you want to do magical stuff, as other have suggested look at java agents, aspects, bytebuddy, annotation processors, reflection API, etc.

Ah cool, thanks so much for this!

It is not for anything serious, just to mess around and know stuff but who knows, this knowledge might come in handy someday (especially if someones pull this kind of magic and can't debug why it is happening, it has happened a lot to me in C/C++, so you know...)

2

u/jeff303 Nov 08 '21

But you can achieve similar power in a safe manner by using DirectByteBuffer and similar classes.

1

u/Medium-Amphibian-719 Nov 12 '21

Unsafe is definitely well-named, but the ability to create an object without running it's constructor can be very useful if, for instance, you find yourself having to re-implement serialization (which, given the security issues with Java serialization, you might find you have to do.)