Support for incremental compilation

[pbzdyl]

Currently Clojure compile and test compile tasks don't support incremental compilation.

We run builds in separated steps (e.g. gradle -x check build followed by gradle check in the same directory) and Clojure source files are recompiled unnecessarily.

It would be great if they at least supported a simple mode: no recompilation when no source files have been changed otherwise perform full compilation.

I tried to create a following method in ClojureCompiler class (and remove @TaskAction from the existing one):

@TaskAction
fun compile(inputs: IncrementalTaskInputs) {
  compile()
}

When there are no input files changed, Gradle won't call compile(IncrementalTaskInputs) at all (and will print UP-TO-DATE after the task name in the console; I have verified it with a simple task written in Groovy). Unfortunately, the way Clojure compile and test compile tasks are constructed and configured makes Gradle to treat files in build/classes/main as input files and as they are always newer than the last build Gradle invokes compile(IncrementalTasksInputs) with those files reported as outdated. (Source files were correctly detected as unmodified.) I was unable to find a way to fix this issue.

[pbzdyl]

It looks that the build/classes issue described above was caused by my existing and complex project setup and works with another sample project. I have submitted PR #12 to implement basic incremental task support.

[pbzdyl]

I have tested it again and it looks that the issue was not in my complex project but in the plugin itself.

Clojure compile task is adding mainSourceSet.output.classesDir to compileClasspath. The issue is that this is the same directory as the Clojure compile output path so running it will automatically update its input files (it will add compiled Clojure classes to the compile classpath) and thus automatically invalidate the files that were produced and Gradle's up to date check will fail.

On the other hand mainSourceSet.output.classesDir is required if someone needs to compile Clojure code using project's Java classes - they need to be available on classpath during Clojure compilation.

I have a solution in the submitted PR to add the output directory only to JavaExec classpath used to run Clojure compiler so compiled Java classes are available on Clojure compile process classpath but not as input files.

I guess another solution would be to have a separate output directory for Java and Clojure compiled classes. However, I am not sure how it would impact other parts of Gradle build like jar packaging, running tests, being part of a convention etc.

I tried to find a solution in Gradle plugins for other JVM languages (Groovy, Kotlin) but it seems that they use compilers that deal with Java and the other language mixed sources on their own and don't depend on Java classes produced by JavaCompileTask

[cursive-ghost]

Interesting, thanks for the investigation - I'll take a look when I get a chance. The Kotlin plugin actually does something similar, it compiles its classes to a separate directory and then copies them later, so in my plugin build I have to do this: compileClojure.classpath += files(compileKotlin.destinationDir). I can't remember the name of the task that copies the classes, it was something like copyKotlinClasses. I'll check how that task is configured, perhaps something like that would be a good idea.

[pbzdyl]

Yes, I think it would be actually better to have a separate output directory for Clojure compiled classes. Otherwise changes to Java source won't trigger Clojure recompilation.

This way one could setup mixed language projects (e.g. Java, Clojure, Kotlin) by specifying compilation order explicitly (adding output of one compile task as classpath item of another).

I will try to change the PR to use this approach.

[cursive-ghost]

Exactly. I think this is the most flexible approach, at the cost of requiring a little more configuration in users' build files. But I think that's an acceptable tradeoff, as long as we document how to do it (and it's not too hard to do).

[pbzdyl]

It looks that the solution would be related to #11 and could be implemented just by configuring separate source set in projects using separate source sets for Java and Clojure code.

We could have the plugin to iterate over all source sets in the project and configure Clojure compile task for them.

There should be instructions provided how to configure separate source sets for mixed projects and how they need to be linked depending on the desired compilation order (Java -> Clojure or Clojure -> Java).

Would such approach be acceptable for you?

[pbzdyl]

I have changed PR #12 to include POC of the above implementation. If it looks good I will add documentation describing how to use it in Clojure only as well as Clojure/Java mixed projects.

[cursive-ghost]

Thanks Piotrek, I'll take a look at this early next week.