Regression of Gradle incremental compilation

[ben-manes]

I am observing file overwrites which should have been resolved in AutoValue 1.9 with #1075. It was flagged by a security tool as a possible indicator of a supply chain compromise.

Error: StepSecurity Harden Runner: Source code overwritten file: /home/runner/work/caffeine/caffeine/simulator/build/generated/sources/annotationProcessor/java/main/com/github/benmanes/caffeine/cache/simulator/report/csv/AutoValue_PlotCsv_ChartStyle.java syscall: openat by exe: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.3-7/x64/bin/java

The build scan confirms this as a stableSources input change.

I reproduced this locally using,

./gradlew --console plain --no-build-cache simulator:clean simulator:compileJava \
  && ./gradlew --console plain --no-build-cache simulator:compileJava

$ ./gradlew --console plain --no-build-cache simulator:clean simulator:compileJava && ./gradlew --console plain --no-build-cache simulator:compileJava
Configuration on demand is an incubating feature.
> Task :caffeine:downloadCaffeineLocal UP-TO-DATE
> Task :caffeine:compileJavaPoetJava UP-TO-DATE
> Task :caffeine:processJavaPoetResources UP-TO-DATE
> Task :caffeine:javaPoetClasses UP-TO-DATE
> Task :simulator:clean
> Task :caffeine:generateNodes UP-TO-DATE
> Task :caffeine:generateLocalCaches UP-TO-DATE
> Task :caffeine:compileJava UP-TO-DATE
> Task :caffeine:processResources NO-SOURCE
> Task :caffeine:classes UP-TO-DATE
> Task :caffeine:compileCodeGenJava UP-TO-DATE
> Task :caffeine:processCodeGenResources NO-SOURCE
> Task :caffeine:codeGenClasses UP-TO-DATE
> Task :caffeine:jar UP-TO-DATE
> Task :simulator:downloadCaffeineLocal
> Task :simulator:compileJava

BUILD SUCCESSFUL in 40s
11 actionable tasks: 3 executed, 8 up-to-date
Configuration on demand is an incubating feature.
> Task :caffeine:downloadCaffeineLocal UP-TO-DATE
> Task :simulator:downloadCaffeineLocal UP-TO-DATE
> Task :caffeine:compileJavaPoetJava UP-TO-DATE
> Task :caffeine:processJavaPoetResources UP-TO-DATE
> Task :caffeine:javaPoetClasses UP-TO-DATE
> Task :caffeine:generateNodes UP-TO-DATE
> Task :caffeine:generateLocalCaches UP-TO-DATE
> Task :caffeine:compileJava UP-TO-DATE
> Task :caffeine:processResources NO-SOURCE
> Task :caffeine:classes UP-TO-DATE
> Task :caffeine:compileCodeGenJava UP-TO-DATE
> Task :caffeine:processCodeGenResources NO-SOURCE
> Task :caffeine:codeGenClasses UP-TO-DATE
> Task :caffeine:jar UP-TO-DATE
> Task :simulator:compileJava

BUILD SUCCESSFUL in 5s
10 actionable tasks: 1 executed, 9 up-to-dat

To avoid flukes, I used a resolution strategy to force the dependency to the right version if an old version was transitively referenced. I also tried to remove a convenience extension, auto-builder, in case it broke the incremental build. The above command still showed a rebuild locally.

This is a minor annoyance but it would be nice to have a clean bill of health from security audits.

[eamonnmcmanus]

Is this an issue with AutoValue or with Gradle? We don't have much Gradle expertise on this project, unfortunately.

[ben-manes]

I don't know, the inputs changed which would make sense as newly generated classes added to the classpath. However you would think Gradle would understand why they came about as implicit inputs and not invalidate the compilation unit. Unfortunately the compilation toolchain and that low level interaction is outside of my gradle experience. Probably if we tried something similar like MapStruct would help indicate if this is a bug in the annotation processor or the toolchain.

[ben-manes]

I was able to narrow the problem down to my build's configuration, sorry for the noise.

I copied an example (jhipster/generator-jhipster#9134) which had the following. The eclipse plugin doesn't have sourceSets so that referred to the top-level and then changed the java plugin's compile input, and caused the invalidation. My mistake of googling for a snippet instead of thinking through it myself.

eclipse {
  sourceSets.main {
    java.srcDir "${buildDir}/generated/sources/annotationProcessor/java/main"
  }
}

The proper fix is,

import org.gradle.plugins.ide.eclipse.model.SourceFolder

eclipse.classpath.file.beforeMerged {
  entries.add(new SourceFolder(
    "${buildDir.name}/generated/sources/annotationProcessor/java/main", 'bin/main'))
}

[tbroyer]

@ben-manes you might want to use https://plugins.gradle.org/plugin/com.diffplug.eclipse.apt for proper Eclipse support.

[ben-manes]

Thanks! Switching to that did not seem to find the sources, assuming it replaced my snippet. I'll play with it and see if there is some docs that I missed.