The dark side of Jack and Jill

Last year Google released new toolchain - Jack (Java Android Compiler Kit) and Jill(Jack Intermediate Library Linker) which is intended to replace existing javac + dxpipeline.

In this article I will try to gather my thoughts and concerns regarding this new toolchain.

But before I start digging deeper into Jack&Jill I want to take a little detour and give you a high level overview of the existing toolchain and the process of compiling your beloved Android app.

Android code compilation 101

To be completely honest, I will not go through entire build process - I will only concentrate on part which is the most relevant to our topic - transforming Java source code into DEX file.

Ever since the first dinosaur stepped on this planet, compilation process was going as follows:

We start with a plain Java source code. And the goal - is to compile this source code into executable instructions which JVM on your device can understand.

For plain Java (not Android) application, we just need a Java compiler (javac). This beast can compile Java source code into Java bytecode (*.class files). Java bytecode can be executed by a regular JVM which (most likely) is running on your machine.

The thing is, that on Android we use a non-standard JVM. We use its modified version which is highly optimized for mobile environment. Such JVM is called Dalvik (or ARTon L+ devices which is even more performant).

So since JVM is modified, Java bytecode needs to be modified as well, so Dalvik can understand it. That's the responsibility of dx tool - it takes Java bytecode (.class files) and transforms it into Android-friendly bytecode (*.dx file)

Its interesting to mention that when you include third party library in your project - it comes as a jar (or aar for Android libs) - which in turn is nothing more than just a zipped collection of *.class files1. So third party lib goes straight to dx tool because we don't need to compile it.

So far its pretty simple, right?

Bytecode manipulation

As time passed and Android developers became more experienced, people started developing cool tools and plugins which can enhance your code at the Java bytecode level (a.k.a bytecode manipulation).

Most popular tools you might have probably heard of:

• Proguard
• Jacoco coverage
• Retrolambda
• ...and many more

It gave us a cool ability to post-process our code w/o making changes to our original sources. F.i. Proguard can analyze your Java bytecode and remove parts which are not used (also known as minimization). Or Retrolambda replaces Java8 lambdas with anonymous inner classes, so your "lambdas" work on Android VM which does not support Java8 features2.

Here is what it looks like:

Each class (its bytecode) is processed by bytecode manipulation plugin and the result is fed to dx tool to produce final result.

Transform API

As number of such tools started to increase it became obvious that Android Gradle build system was not really designed for bytecode manipulators - the only way to "catch" the moment when Java bytecode is ready, but not yet processed by dx was to add Gradle task dependency to existing task created by Android Gradle plugin. The name of that task was an implementation detail, it was generated dynamically based on project configuration and Google kept changing it as Android Gradle Plugin evolved. This led to the problem that all those plugins kept breaking with every new Android Plugin release.

So Google needed to act. And they did - they introduce Transition API - a simple API which allows you to add a Transformer - class which will be called at the appropriate time of the build process. The input is a Java bytecode. This allows plugin developers to use a much more reliable way of manipulating bytecode and stop using undocumented private APIs.

Jack & Jill

At the same time, somewhere in parallel in a dungeon, group of Googlers were super busy creating something new, something which will blow everybody's mind! Self driving cars! Jack and Jill!

Jack - is a compiler. Similar to javac, but it does a slightly different thing: 

As you can see, Jack compiles Java source code straight into Dex file! We don't have intermediate *.class files anymore, so dx tool is not needed!

But wait! What if I include a third-party library in my project (which comes as a collection of .class files)?

And that's when Jill comes into play:

Jill can process class files and transform them into special Jayce format which can be used as an input for Jack compiler.

So now let's step aside for a second and think... What is going to happen to all those cool plugins we got so addicted to? They all need .class files and Jack compiler doesn't have those anymore...

Luckily, Jack provides some of those important for us features out of the box:

• Retrolambda - will not be needed. Jack can handle lambdas properly
• Proguard - it is baked into Jack now, so you can still use obfuscation and minimization

However, list of downsides is a bit concerning:

• Transform API is not supported by Jack - there is no intermediate Java bytecode you can modify, so some plugins I didn't mention here will stop working
• Annotation processing is not currently supported by Jack, so if you heavily depend on libraries like Dagger, AutoValue, etc., you should think twice before switching to Jack. EDIT: As pointed out by Jake Wharton, Jack in N Preview has annotation processing support, but it is not exposed yet through Gradle.
• Lint detectors which operate on a Java bytecode level are not supported
• Jack is currently slower than javac + dx
• Jacoco is not supported - well, I personally find Jacoco questionable (it doesnt really show what you want to see), so can totally live without it
• Dexguard - enterprise version of Proguard is not currently supported

I realize that things I just mentioned are temporary and Google is actively working on addressing them, but unfortunately all that excitement around Android supporting Java8 features will fade pretty soon when people start to realize the real cost behind switching to the new toolchain.

Jack is a really cool move and will give Google much more control and flexibility with the build pipeline, but it is in its very early stage and it will take a while before it will start gaining its popularity.

Always yours,
Pavel Dudka

1. AAR is actually a bit more than JAR - it also includes Android-related data like assets, resources and other data which regular JAR doesn't support ↩
2. Android VM running on latest N supports some of Java8 instructions ↩
3. Title Jack'n'Jill image is proudly "borrowed" from here ↩