Debugging (Failed) Instrumentation through sun.misc.Unsafe.defineAnonymousClass

defineAnonymousClass is a super handy method of the notorious sun.misc.Unsafe “escape hatch” in the JVM. defineAnonymousClass is used as a fast-path to define specialized anonymous classes which are not rigorously verified, are not loaded into a class loader (e.g. they exist only so long as you hold a reference to one, perhaps through an instance of an object of that type), and can have their constant pools lazily patched at load-time. For classes that do not need to be accessible from anywhere other than the class that defines them, defineAnonymousClass is a much faster way to load in a class. defineAnonymousClass is used under-the-hood to implement Lambdas, and we aggressively used it to implement¬†CROCHET.

One fun side-effect of defineAnonymousClass’ performance tweaks is that when you break verification of a class loaded through defineAnonymosuClass (for instance, if you are instrumenting classes with a JavaAgent, perhaps with ASM), you get a totally useless error message:

This is not helpful. If you use a fastDebug build of OpenJDK, then you can use the flags “-XX:+UnlockDiagnosticVMOptions -XX:+VerboseVerification”, which will produce a lot of output, ending up with something like:

Which can be enough to point you to the method that’s failing verification to help you figure out what the cause of the error is (in this case, it’s that my instrumentation is adding a reference to a class not loaded in a class loader that is within the scope of the anonymous class being defined).

Note also that due to a bug in OpenJDK versions 7 and 8, it is not possible to retransform lambdas using a java agent (thanks for the pointer, Katie!).