CRIU supports Java debugger via the restore option file #19754

JasonFengJ9 · 2024-06-24T22:53:32Z

CRIU supports Java debugger via the restore option file

Debugger support related code are guarded with isDebugOnRestoreEnabled();
For -Xint mode, do not disable JVMTI capabilities required for JDWP debugger before checkpoint, add a capability can_access_local_variables;
For JIT mode, add JVMTI capabilities required for JDWP debugger before checkpoint;
Hooked J9HOOK_VM_PREPARING_FOR_RESTORE event to determine if a JDWP agent is specified in the restore option file, also hooked J9HOOK_VM_CRIU_RESTORE to load the agent libraries;
Refactored the agent library creation from J9VMInitArgs;
Added some trace points.

Signed-off-by: Jason Feng fengj@ca.ibm.com

Closes #16959

JasonFengJ9 · 2024-06-25T13:12:03Z

FYI @tajila

JasonFengJ9 · 2024-08-22T18:47:09Z

The CRIU JDWP debugger via the restore option file requires both extension repo PR ibmruntimes/openj9-openjdk-jdk#591 and its backports (in progress) and this PR.
However, this PR alone won't break any existing builds such as Java 8 that won't be backported.

JasonFengJ9 · 2024-08-22T18:55:16Z

FYI @dsouzai
This PR has the following messages post-restore when no JDWP agent is specified in the restore option file.

JVMJITM044W Some or all compiled code in the code cache invalidated post restore.
JVMJITM043W AOT load and compilation disabled post restore.

which is expected to be resolved via a JIT change like the patch you provided.

Note: this JDWP debugger support via the option file is only available with -XX:+DebugOnRestore.

dsouzai · 2024-08-22T21:09:19Z

I opened #20047, which is essentially the patch I had given you.

dsouzai · 2024-08-23T15:31:34Z

I was reading over #18866, specifically the VM Coordination section, and wanted to check if Points 1 and 2 are addressed, namely:

Decompile immediately on restore if a user enables debug post-restore
If debug is not enabled post-restore, but normal HCR is possible, then the VM needs to ensure that stacks with FSD bodies get decompiled.

Regarding 1, although currently (i.e. with this PR and #20047) we will invalidate all compiled code if debug is enabled post-restore, it is still possible for JIT'd code to execute, namely:

JNI Thunks
Failed proactive compilation

For a failed proactive compilation, we will fall back to using the FSD body which means that a thread will continue to execute that body. Therefore, the VM needs to decompile immediately on restore if debug is enabled. Once we have the ability to reuse the FSD code we generate pre-checkpoint this won't be necessary, but for now I believe we need this to make debug fully functional.

In the case of JNI Thunks, I'm working on a PR for that.

I think once we address all of this, debug on restore should be completely functional (if not optimal). FYI @TobiAjila @vijaysun-omr

dsouzai · 2024-08-23T15:48:29Z

For a failed proactive compilation, we will fall back to using the FSD body which means that a thread will continue to execute that body. Therefore, the VM needs to decompile immediately on restore if debug is enabled. Once we have the ability to reuse the FSD code we generate pre-checkpoint this won't be necessary, but for now I believe we need this to make debug fully functional.

Hm actually... I guess it's not actually a problem if it's running FSD code because that is already set up to deal with debug events. So I guess I just need to address the JNI Thunk issue; the rest is covered by the method body invalidation that already occurs.

I'll think about this some more, but now I don't believe we need to decompile immediately anymore heh.

dsouzai · 2024-09-03T20:04:24Z

In the case of JNI Thunks, I'm working on a PR for that.

Opened #20108.

runtime/jvmti/jvmtiCapability.c

JasonFengJ9 · 2024-09-04T19:34:55Z

Rebased to resovle the merge conflict.

runtime/jvmti/jvmtiHook.c

tajila · 2024-09-11T15:15:57Z

@JasonFengJ9 How exatly did you enable debug on restore? whe I tried -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=y in the options file it failed with:

JVMJITM043W AOT load and compilation disabled post restore.
post restore hook
JVMJ9VM007E Command-line option unrecognised: -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=y
Exception in thread "main" org.eclipse.openj9.criu.JVMRestoreException: The JVM could not enable all the restore options specified
	at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVM(CRIUSupport.java:534)
	at Debug.checkPointJVM(Debug.java:32)
	at Debug.main(Debug.java:16)
Caused by: openj9.internal.criu.JVMRestoreException: The JVM could not enable all the restore options specified
	at java.base/openj9.internal.criu.InternalCRIUSupport.checkpointJVMImpl(Native Method)
	at java.base/openj9.internal.criu.InternalCRIUSupport.checkpointJVM(InternalCRIUSupport.java:997)
	at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVM(CRIUSupport.java:530)
	... 2 more

JasonFengJ9 · 2024-09-11T15:37:15Z

@tajila I was using -agentlib:jdwp=transport=dt_socket,server=y,suspend=y, will look into -Xrunjdwp option.

tajila · 2024-09-11T15:45:28Z

same issue

JVMJITM043W AOT load and compilation disabled post restore.
post restore hook
JVMJ9VM007E Command-line option unrecognised: -agentlib:jdwp=transport=dt_socket,server=y,suspend=y
Exception in thread "main" org.eclipse.openj9.criu.JVMRestoreException: The JVM could not enable all the restore options specified
	at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVM(CRIUSupport.java:534)
	at Debug.checkPointJVM(Debug.java:32)
	at Debug.main(Debug.java:16)
Caused by: openj9.internal.criu.JVMRestoreException: The JVM could not enable all the restore options specified
	at java.base/openj9.internal.criu.InternalCRIUSupport.checkpointJVMImpl(Native Method)
	at java.base/openj9.internal.criu.InternalCRIUSupport.checkpointJVM(InternalCRIUSupport.java:997)
	at openj9.criu/org.eclipse.openj9.criu.CRIUSupport.checkpointJVM(CRIUSupport.java:530)
	... 2 more

tajila · 2024-09-11T21:19:53Z

@JasonFengJ9 We will need to add a docs issue for this as well

JasonFengJ9 · 2024-09-13T18:16:03Z

-Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=y

Added support of -Xrunjdwp.

@tajila this is ready for another look.

tajila · 2024-09-13T18:29:54Z

jenkins test sanity xlinux jdk17

tajila · 2024-09-13T18:30:04Z

jenkins test sanity alinux64 jdk21

tajila · 2024-09-13T18:30:12Z

jenkins test sanity win jdk8

JasonFengJ9 · 2024-09-16T11:48:11Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_sanity.functional_x86-64_linux_Personal_testList_0/540/consoleFull

16:40:07  ---SUMMARY OF FAILED TESTS---
16:40:07  cma001
16:40:07  -----------------------------
16:40:07  
16:40:07  -----------------------------------
16:40:07  cmdLineTester_jvmtitests_1_FAILED

Looking at this PR test failure.

Debugger support related code are guarded with isDebugOnRestoreEnabled(); For -Xint mode, do not disable JVMTI capabilities required for JDWP debugger before checkpoint, add a capability can_access_local_variables; For JIT mode, add JVMTI capabilities required for JDWP debugger before checkpoint; Hooked J9HOOK_VM_PREPARING_FOR_RESTORE event to determine if a JDWP agent is specified in the restore option file, also hooked J9HOOK_VM_CRIU_RESTORE to load the agent libraries; Support -agentlib/-agentpath and -Xrunjdwp; Refactored the agent library creation from J9VMInitArgs; Added some trace points; Disable criu_jitPostRestore.xml failure conditions until the JIT PR is merged. Signed-off-by: Jason Feng <fengj@ca.ibm.com>

JasonFengJ9 · 2024-09-17T22:23:33Z

Fixed the PR test failure due to an earlier break for undecorated JDWP agent processing while multiple agents were present.

In addition, disabled following failure conditions within criu_jitPostRestore.xml:

		<!-- output type="failure" caseSensitive="yes" regex="no">Some or all compiled code in the code cache invalidated post restore.</output>
		<output type="failure" caseSensitive="yes" regex="no">JIT compilation disabled post restore.</output -->

Verified manually that these failure conditions can re-enabled with

Ensure JIT/AOT code is not invalidated post-restore under -XX:+DebugOnRestore #20047

Also moved DisposeEnvironment(jvmti_env) into jvmtiHookVMRestoreStartAgent() which is the last part of the cleanup if there was no JDWP agent specified. It releases VM access and hence can't be invoked within criuDisableHooks() from J9HOOK_VM_PREPARING_FOR_RESTORE.

@tajila this is ready for another look.

tajila · 2024-09-18T11:20:11Z

jenkins test sanity win jdk8

tajila · 2024-09-18T11:20:19Z

jenkins test sanity alinux64 jdk21

tajila · 2024-09-18T11:20:29Z

jenkins test sanity xlinux64 jdk17

tajila · 2024-09-18T12:15:03Z

jenkins test sanity xlinux jdk17

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

JasonFengJ9 added comp:vm criu Used to track CRIU snapshot related work labels Jun 24, 2024

JasonFengJ9 force-pushed the jdwpoptionfile branch from e05b61d to 3e888f0 Compare June 27, 2024 14:26

JasonFengJ9 mentioned this pull request Aug 13, 2024

Add support for a suspendOnRestore option in JDWP ibmruntimes/openj9-openjdk-jdk#591

Merged

JasonFengJ9 force-pushed the jdwpoptionfile branch from 3e888f0 to f6b5a0a Compare August 22, 2024 18:42

JasonFengJ9 marked this pull request as ready for review August 22, 2024 18:47

JasonFengJ9 requested a review from tajila August 22, 2024 18:47

dsouzai mentioned this pull request Aug 22, 2024

Ensure JIT/AOT code is not invalidated post-restore under -XX:+DebugOnRestore #20047

Merged

dsouzai mentioned this pull request Aug 23, 2024

Compiler Support for Debug On Restore #18866

Open

21 tasks

tajila reviewed Sep 4, 2024

View reviewed changes

runtime/jvmti/jvmtiCapability.c Outdated Show resolved Hide resolved

JasonFengJ9 force-pushed the jdwpoptionfile branch from f6b5a0a to 638cf50 Compare September 4, 2024 19:32

JasonFengJ9 force-pushed the jdwpoptionfile branch from 638cf50 to eebf643 Compare September 4, 2024 20:29

tajila reviewed Sep 5, 2024

View reviewed changes

runtime/jvmti/jvmtiHook.c Outdated Show resolved Hide resolved

JasonFengJ9 force-pushed the jdwpoptionfile branch from eebf643 to 20fc1bc Compare September 10, 2024 15:49

tajila added the doc:externals label Sep 11, 2024

JasonFengJ9 added comp:doc and removed comp:doc labels Sep 11, 2024

JasonFengJ9 mentioned this pull request Sep 13, 2024

CRIU enable JDWP support post-restore eclipse-openj9/openj9-docs#1407

Open

JasonFengJ9 force-pushed the jdwpoptionfile branch from 20fc1bc to a852204 Compare September 13, 2024 18:13

tajila mentioned this pull request Sep 17, 2024

CRIU: Remove CRIU debug interpreter #19835

Closed

ThanHenderson mentioned this pull request Sep 17, 2024

Remove CRIU Interpreter #20177

Merged

JasonFengJ9 force-pushed the jdwpoptionfile branch 2 times, most recently from 4fa7585 to 55712a0 Compare September 17, 2024 22:19

JasonFengJ9 force-pushed the jdwpoptionfile branch from 55712a0 to f812cad Compare September 17, 2024 22:21

tajila merged commit ebe8718 into eclipse-openj9:master Sep 18, 2024
9 checks passed

JasonFengJ9 deleted the jdwpoptionfile branch September 18, 2024 22:09

This was referenced Sep 18, 2024

(0.48) CRIU supports Java debugger via the restore option file #20191

Merged

80 FVT_Serviceability.diagnostics.xdump_thrstart.Mode101.1 Initialization error for library j9jvmti29(-3) #20194

Closed

dsouzai added a commit to dsouzai/openj9 that referenced this pull request Sep 19, 2024

Renable failure conditions disabled in eclipse-openj9#19754

ff8812d

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

JasonFengJ9 mentioned this pull request Sep 20, 2024

-agentpath: libraries require no path/name decoration #20202

Merged

dsouzai added a commit to dsouzai/openj9 that referenced this pull request Sep 20, 2024

Renable failure conditions disabled in eclipse-openj9#19754

d3a7474

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

dsouzai mentioned this pull request Sep 24, 2024

(0.48) Ensure JIT/AOT code is not invalidated post-restore under -XX:+DebugOnRestore #20219

Merged

luke-li-2003 pushed a commit to luke-li-2003/openj9 that referenced this pull request Sep 25, 2024

Renable failure conditions disabled in eclipse-openj9#19754

a05a599

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

pshipton mentioned this pull request Sep 25, 2024

cmdLineTester_criu_jitPostRestore Testing: Test -Xjit:exclude={*} AOT load and compilation disabled post restore #20233

Closed

JasonFengJ9 mentioned this pull request Oct 2, 2024

(0.48) -agentpath: libraries require no path/name decoration & Refactor a few CRIU APIs to take J9JavaVM* instead of J9VMThread* #20287

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CRIU supports Java debugger via the restore option file #19754

CRIU supports Java debugger via the restore option file #19754

JasonFengJ9 commented Jun 24, 2024 •

edited

Loading

JasonFengJ9 commented Jun 25, 2024

JasonFengJ9 commented Aug 22, 2024

JasonFengJ9 commented Aug 22, 2024

dsouzai commented Aug 22, 2024

dsouzai commented Aug 23, 2024 •

edited

Loading

dsouzai commented Aug 23, 2024

dsouzai commented Sep 3, 2024

JasonFengJ9 commented Sep 4, 2024

tajila commented Sep 11, 2024

JasonFengJ9 commented Sep 11, 2024

tajila commented Sep 11, 2024

tajila commented Sep 11, 2024

JasonFengJ9 commented Sep 13, 2024

tajila commented Sep 13, 2024

tajila commented Sep 13, 2024

tajila commented Sep 13, 2024

JasonFengJ9 commented Sep 16, 2024

JasonFengJ9 commented Sep 17, 2024

tajila commented Sep 18, 2024

tajila commented Sep 18, 2024

tajila commented Sep 18, 2024

tajila commented Sep 18, 2024

CRIU supports Java debugger via the restore option file #19754

CRIU supports Java debugger via the restore option file #19754

Conversation

JasonFengJ9 commented Jun 24, 2024 • edited Loading

JasonFengJ9 commented Jun 25, 2024

JasonFengJ9 commented Aug 22, 2024

JasonFengJ9 commented Aug 22, 2024

dsouzai commented Aug 22, 2024

dsouzai commented Aug 23, 2024 • edited Loading

dsouzai commented Aug 23, 2024

dsouzai commented Sep 3, 2024

JasonFengJ9 commented Sep 4, 2024

tajila commented Sep 11, 2024

JasonFengJ9 commented Sep 11, 2024

tajila commented Sep 11, 2024

tajila commented Sep 11, 2024

JasonFengJ9 commented Sep 13, 2024

tajila commented Sep 13, 2024

tajila commented Sep 13, 2024

tajila commented Sep 13, 2024

JasonFengJ9 commented Sep 16, 2024

JasonFengJ9 commented Sep 17, 2024

tajila commented Sep 18, 2024

tajila commented Sep 18, 2024

tajila commented Sep 18, 2024

tajila commented Sep 18, 2024

JasonFengJ9 commented Jun 24, 2024 •

edited

Loading

dsouzai commented Aug 23, 2024 •

edited

Loading