Status of Reproducible comparing tests on linux

[sophia-guo]

PPC64le and aarch64 was enabled on Sep 25th. adoptium/ci-jenkins-pipelines#1115. There is no data yet as no betaTrigger builds triggered with test enabled.

Recent linux update was merged in Sep 6th. .

Most recent ea pipeline jobs are triggered by test disabled (not sure why, for example https://ci.adoptium.net/job/build-scripts/job/openjdk21-pipeline/333/), which means most reproducible comparing tests are quite old and may not be worth looking at. FYI @smlambert @andrew-m-leonard

Only one test build with jdk24 is with recent update. https://trss.adoptium.net/output/test?id=66f72b85d24e1b006ee0dd01 @andrew-m-leonard triggered by https://ci.adoptium.net/job/build-scripts/job/openjdk24-pipeline/25/, test jenkins job is https://ci.adoptium.net/job/Test_openjdk24_hs_special.system_x86-64_linux/11/. Both job and rerun job have same results. 6 files differs.

17:51:56  Number of files: 105604
17:51:56  Comparing jdk-24+17 with tarJDK/jdk-24+17 ... output to file: reprotest.diff
17:52:00  Files jdk-24+17/jmods/expanded_java.base.jmod/lib/classlist and tarJDK/jdk-24+17/jmods/expanded_java.base.jmod/lib/classlist differ
17:52:00  Files jdk-24+17/jmods/expanded_java.base.jmod/lib/server/libjvm.so and tarJDK/jdk-24+17/jmods/expanded_java.base.jmod/lib/server/libjvm.so differ
17:52:00  Files jdk-24+17/lib/classlist and tarJDK/jdk-24+17/lib/classlist differ
17:52:00  Files jdk-24+17/lib/server/classes.jsa and tarJDK/jdk-24+17/lib/server/classes.jsa differ
17:52:00  Files jdk-24+17/lib/server/classes_nocoops.jsa and tarJDK/jdk-24+17/lib/server/classes_nocoops.jsa differ
17:52:00  Files jdk-24+17/lib/server/libjvm.so and tarJDK/jdk-24+17/lib/server/libjvm.so differ
17:52:00  Number of differences: 6
17:52:00  ReproduciblePercent = 99.99 %
17:52:00  Differences found..., logged in: reprotest.diff

@andrew-m-leonard I remember you mentioned those CDS files differences ( 4 files) are a known issue due to the classlist ipv6 difference. Compared with the job I tested before https://ci.adoptium.net/job/Test_openjdk23_hs_special.system_x86-64_linux_testList_0/29/console, number of files increased and two more file differs. jmods/expanded_java.base.jmod/lib/server/libjvm.so and /lib/server/libjvm.so and tarJDK/jdk-24+17/lib/server/libjvm.so. Are those two differences expected?

09:59:33  Number of files: 104722
09:59:33  Comparing jdk-23+36 with tarJDK/jdk-23+36 ... output to file: reprotest.diff
09:59:37  Files jdk-23+36/jmods/expanded_java.base.jmod/lib/classlist and tarJDK/jdk-23+36/jmods/expanded_java.base.jmod/lib/classlist differ
09:59:37  Files jdk-23+36/lib/classlist and tarJDK/jdk-23+36/lib/classlist differ
09:59:37  Files jdk-23+36/lib/server/classes.jsa and tarJDK/jdk-23+36/lib/server/classes.jsa differ
09:59:37  Files jdk-23+36/lib/server/classes_nocoops.jsa and tarJDK/jdk-23+36/lib/server/classes_nocoops.jsa differ
09:59:37  Number of differences: 4
09:59:37  ReproduciblePercent = 99.99 %
09:59:37  Differences found..., logged in: reprotest.diff``

[sophia-guo]

I see , most of September openjdk pipeline jobs disabled tests job as it's in Release period, which causes only one job with jdk24 available. adoptium/temurin#54 (comment)

[andrew-m-leonard]

@sophia-guo jdk-21.0.5+9 EA is building now.... so you should have a new test to look at soon...

The above differences need looking at, for Linux they should be 100% identical, I suspect an "environmental" difference that we need to investigate in this issue. I suspect one of:

• Differing TimeZone?
• Differing ipv6 enablement?

@sophia-guo What would be really useful output in the test, is if the "classlist" file appears in the diff, then do a verbose content diff just for that file, as that will reveal any of the differences suggested above ^

[sophia-guo]

I think you mentioned @andrew-m-leonard

I suspect you've added the CDS files for Linux due to the classlist ipv6 difference, that is a known issue, and we need to ensure we have the same ipv6 setting on the reproduce VM.

[sophia-guo]

The latest results -
linux-x64 https://ci.adoptium.net/job/Test_openjdk21_hs_special.system_x86-64_linux/27/artifact/aqa-tests/TKG/output_17279540359679/Test_openjdk21_hs_special.system_x86-64_linux.tap/*view*/

        Cleaning any Temurin build-scripts release file BUILD_INFO from tarJDK/jdk-21.0.5+9
        Number of files: 108609
        Comparing jdk-21.0.5+9 with tarJDK/jdk-21.0.5+9 ... output to file: reprotest.diff
        Files jdk-21.0.5+9/jmods/expanded_java.base.jmod/lib/classlist and tarJDK/jdk-21.0.5+9/jmods/expanded_java.base.jmod/lib/classlist differ
        Files jdk-21.0.5+9/lib/classlist and tarJDK/jdk-21.0.5+9/lib/classlist differ
        Files jdk-21.0.5+9/lib/server/classes.jsa and tarJDK/jdk-21.0.5+9/lib/server/classes.jsa differ
        Files jdk-21.0.5+9/lib/server/classes_nocoops.jsa and tarJDK/jdk-21.0.5+9/lib/server/classes_nocoops.jsa differ
        Number of differences: 4
        ReproduciblePercent = 99.99 %
        Differences found..., logged in: reprotest.diff

linux-aarch64 and ppc64le have the same result
https://ci.adoptium.net/job/Test_openjdk21_hs_special.system_ppc64le_linux/1/artifact/aqa-tests/TKG/output_17279518397907/Test_openjdk21_hs_special.system_ppc64le_linux.tap/*view*/

https://ci.adoptium.net/job/Test_openjdk21_hs_special.system_ppc64le_linux/1/artifact/aqa-tests/TKG/output_17279518397907/Test_openjdk21_hs_special.system_ppc64le_linux.tap/*view*/)

        Comparing jdk-21.0.5+9 with tarJDK/jdk-21.0.5+9 ... output to file: reprotest.diff
        Only in jdk-21.0.5+9: jdk-21.0.5+9
        Only in jdk-21.0.5+9: jdk-21.0.5+9-static-libs
        Files jdk-21.0.5+9/jmods/expanded_java.base.jmod/lib/classlist and tarJDK/jdk-21.0.5+9/jmods/expanded_java.base.jmod/lib/classlist differ
        Files jdk-21.0.5+9/lib/classlist and tarJDK/jdk-21.0.5+9/lib/classlist differ
        Files jdk-21.0.5+9/lib/server/classes.jsa and tarJDK/jdk-21.0.5+9/lib/server/classes.jsa differ
        Files jdk-21.0.5+9/lib/server/classes_nocoops.jsa and tarJDK/jdk-21.0.5+9/lib/server/classes_nocoops.jsa differ
        Number of differences: 6
        ReproduciblePercent = 99.99 %
        Differences found..., logged in: reprotest.diff