Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

superpmi: parallel mode doesn't handle crash of superpmi #56506

Open
BruceForstall opened this issue Jul 29, 2021 · 0 comments
Open

superpmi: parallel mode doesn't handle crash of superpmi #56506

BruceForstall opened this issue Jul 29, 2021 · 0 comments
Assignees
@BruceForstall
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Milestone
Future

Comments

@BruceForstall
Copy link
Member

BruceForstall commented Jul 29, 2021 •
edited
Loading

When investigating #56156, I found that if superpmi crashes in one of multiple superpmi parallel replay processes, that crash isn't caught and the superpmi replay "succeeds".

This was on Linux/arm32. Perhaps we don't catch SIGBUS "Bus error" exceptions, compared to others? We do have try/catch in place.

The crash is:

Thread 1 "superpmi" received signal SIGBUS, Bus error.
Compiler::impImportStaticReadOnlyField (this=0xd17844, fldAddr=0xd665d9, lclTyp=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/importer.cpp:7569
7569                lval = *((__int64*)fldAddr);
(gdb) bt
#0  Compiler::impImportStaticReadOnlyField (this=0xd17844, fldAddr=0xd665d9, lclTyp=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/importer.cpp:7569
#1  0xf5f6e1ae in Compiler::impImportBlockCode (this=<optimized out>, block=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/importer.cpp:14882
#2  0xf5f7266e in Compiler::impImportBlock(BasicBlock*)::$_0::operator()(Compiler::impImportBlock(BasicBlock*)::FilterVerificationExceptionsParam*) const (this=<optimized out>, pParam=0xfffed640)
    at /home/bruce/gh/runtime2/src/coreclr/jit/importer.cpp:17695
#3  Compiler::impImportBlock (this=0xd17844, block=0xd25178) at /home/bruce/gh/runtime2/src/coreclr/jit/importer.cpp:17705
#4  0xf5f746fc in Compiler::impImport (this=0xd17844) at /home/bruce/gh/runtime2/src/coreclr/jit/importer.cpp:18778
#5  0xf5f131be in Compiler::fgImport (this=0xd17844) at /home/bruce/gh/runtime2/src/coreclr/jit/flowgraph.cpp:625
#6  0xf6018f36 in Phase::Run (this=0xfffed7b4) at /home/bruce/gh/runtime2/src/coreclr/jit/phase.cpp:61
#7  0xf5ed3b20 in DoPhase (_compiler=0xd17844, _phase=PHASE_IMPORTATION, _action=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/phase.h:136
#8  Compiler::compCompile (this=0xd17844, methodCodePtr=0xfffede80, methodCodeSize=0x0, compileFlags=0xfffeddf0) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:4511
#9  0xf5ed6aa2 in Compiler::compCompileHelper (this=0xd17844, classPtr=<optimized out>, compHnd=<optimized out>, methodInfo=<optimized out>, methodCodePtr=0xfffede80, methodCodeSize=0x0, compileFlags=0xfffeddf0)
    at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:6407
#10 0xf5ed58d4 in Compiler::compCompile(CORINFO_MODULE_STRUCT_*, void**, unsigned int*, JitFlags*)::$_11::operator()(Compiler::compCompile(CORINFO_MODULE_STRUCT_*, void**, unsigned int*, JitFlags*)::__JITParam*) const (
    this=<optimized out>, __JITpParam=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:5684
#11 Compiler::compCompile (this=0xd17844, classPtr=0xefe9f000, methodCodePtr=0xfffede80, methodCodeSize=0x0, compileFlags=0xfffeddf0) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:5703
#12 0xf5ed745e in jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const::{lambda(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const::__JITParam*)#1}::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const::__JITParam*) const (this=<optimized out>, __JITpParam=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:7053
#13 jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const (this=<optimized out>, __JITpParam=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:7078
#14 jitNativeCode (methodHnd=0xe7767664, classPtr=0xefe9f000, compHnd=0x5a9c58, methodInfo=0xd239f0, methodCodePtr=0xfffede80, methodCodeSize=0x0, compileFlags=0xfffeddf0, inlineInfoPtr=0xfffede80)
    at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:7080
#15 0xf5f030cc in Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::$_0::operator()(Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::Param*) const (this=<optimized out>, pParam=0xfffede6c)
    at /home/bruce/gh/runtime2/src/coreclr/jit/fginline.cpp:973
#16 Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::$_0::__invoke(Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::Param*) (pParam=0xfffede6c)
    at /home/bruce/gh/runtime2/src/coreclr/jit/fginline.cpp:921
#17 0x004197c0 in RunWithErrorTrap(void (*)(void*), void*)::$_0::operator()(RunWithErrorTrap(void (*)(void*), void*)::TrapParam*) const (this=<optimized out>, pTrapParam=<optimized out>)
    at /home/bruce/gh/runtime2/src/coreclr/ToolBox/superpmi/superpmi-shared/errorhandling.cpp:138
#18 RunWithErrorTrap (function=<optimized out>, param=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/ToolBox/superpmi/superpmi-shared/errorhandling.cpp:140
#19 0xf5f0174a in Compiler::eeRunWithErrorTrap<Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::Param>(void (*)(Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::Param*), Compiler::fgInvokeInlineeCompiler(GenTreeCall*, InlineResult*)::Param*) (this=0xd147ac, function=0xd665d9, param=0xfffede6c) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.h:7957
#20 Compiler::fgInvokeInlineeCompiler (this=0xd147ac, call=0xd23860, inlineResult=0xfffee4f8) at /home/bruce/gh/runtime2/src/coreclr/jit/fginline.cpp:920
#21 0xf5fee9f0 in Compiler::fgMorphCallInlineHelper (this=0xd147ac, call=0xd23860, result=0xfffee4f8) at /home/bruce/gh/runtime2/src/coreclr/jit/morph.cpp:6721
#22 0xf5fee816 in Compiler::fgMorphCallInline (this=0xd147ac, call=0xd23860, inlineResult=0xfffee4f8) at /home/bruce/gh/runtime2/src/coreclr/jit/morph.cpp:6603
#23 0xf5f0020a in Compiler::fgInline (this=0xd147ac) at /home/bruce/gh/runtime2/src/coreclr/jit/fginline.cpp:153
#24 0xf6018f36 in Phase::Run (this=0xfffee63c) at /home/bruce/gh/runtime2/src/coreclr/jit/phase.cpp:61
#25 0xf5ed3cb6 in DoPhase (_compiler=0xd147ac, _phase=PHASE_MORPH_INLINE, _action=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/phase.h:136
#26 Compiler::compCompile (this=0xd147ac, methodCodePtr=0xfffeec8c, methodCodeSize=0xfffeed8c, compileFlags=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:4684
#27 0xf5ed6aa2 in Compiler::compCompileHelper (this=0xd147ac, classPtr=<optimized out>, compHnd=<optimized out>, methodInfo=<optimized out>, methodCodePtr=0xfffeec8c, methodCodeSize=0xfffeed8c, compileFlags=0xfffeeca0)
    at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:6407
#28 0xf5ed58d4 in Compiler::compCompile(CORINFO_MODULE_STRUCT_*, void**, unsigned int*, JitFlags*)::$_11::operator()(Compiler::compCompile(CORINFO_MODULE_STRUCT_*, void**, unsigned int*, JitFlags*)::__JITParam*) const (
    this=<optimized out>, __JITpParam=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:5684
#29 Compiler::compCompile (this=0xd147ac, classPtr=0xe77f360c, methodCodePtr=0xfffeec8c, methodCodeSize=0xfffeed8c, compileFlags=0xfffeeca0) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:5703
#30 0xf5ed745e in jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const::{lambda(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const::__JITParam*)#1}::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const::__JITParam*) const (this=<optimized out>, __JITpParam=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:7053
#31 jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::$_13::operator()(jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*)::__JITParam*) const (this=<optimized out>, __JITpParam=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:7078
#32 jitNativeCode (methodHnd=0xe77f9c54, classPtr=0xe77f360c, compHnd=0x5a9c58, methodInfo=0xfffeeda4, methodCodePtr=0xfffeec8c, methodCodeSize=0xfffeed8c, compileFlags=0xfffeeca0, inlineInfoPtr=0x0)
    at /home/bruce/gh/runtime2/src/coreclr/jit/compiler.cpp:7080
#33 0xf5edf25e in CILJit::compileMethod (this=<optimized out>, compHnd=0x5a9c58, methodInfo=0xfffeeda4, flags=<optimized out>, entryAddress=0xfffeed90, nativeSizeOfCode=0xfffeed8c)
    at /home/bruce/gh/runtime2/src/coreclr/jit/ee_il_dll.cpp:276
#34 0x0040d474 in JitInstance::CompileMethod(MethodContext*, int, bool)::$_0::operator()(JitInstance::CompileMethod(MethodContext*, int, bool)::Param*) const (this=<optimized out>, pParam=0xfffeed94)
    at /home/bruce/gh/runtime2/src/coreclr/ToolBox/superpmi/superpmi/jitinstance.cpp:314
#35 JitInstance::CompileMethod (this=0x59d608, MethodToCompile=<optimized out>, mcIndex=0x2a03d, collectThroughput=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/ToolBox/superpmi/superpmi/jitinstance.cpp:373
#36 0x00410d90 in main (argc=<optimized out>, argv=<optimized out>) at /home/bruce/gh/runtime2/src/coreclr/ToolBox/superpmi/superpmi/superpmi.cpp:361
(gdb) ? fldAddr
Undefined command: "".  Try "help".
(gdb) print fldAddr
$1 = (void *) 0xd665d9

The "successful" run looks like:

$ /home/brucefo/build/runtime/artifacts/tests/coreclr/Linux.arm.Checked/Tests/Core_Root/superpmi -p /home/brucefo/build/runtime/artifacts/tests/coreclr/Linux.arm.Checked/Tests/Core_Root/libclrjit.so /home/brucefo/bugs/spmicollect/small.mch
Using child (/home/brucefo/build/runtime/artifacts/tests/coreclr/Linux.arm.Checked/Tests/Core_Root/superpmi) with args ( /home/brucefo/build/runtime/artifacts/tests/coreclr/Linux.arm.Checked/Tests/Core_Root/libclrjit.so /home/brucefo/bugs/spmicollect/small.mch)
 workerCount=4, skipCleanup=0.
Loaded 3  Jitted 3  FailedCompile 0 Excluded 0 Missing 0

But there are 5 method contexts in this example: the first process that was supposed to handle 2 of them crashed.

It does exit with an error code of 1. Maybe it should be more verbose about this (if -v e is used). Maybe superpmi.py doesn't pay attention to the return code?

category:eng-sys
theme:super-pmi
skill-level:intermediate
cost:medium
impact:small
@BruceForstall BruceForstall added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 29, 2021
@BruceForstall BruceForstall added this to the 6.0.0 milestone Jul 29, 2021
@BruceForstall BruceForstall self-assigned this Jul 29, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Jul 29, 2021
@BruceForstall BruceForstall modified the milestones: 6.0.0, 7.0.0 Jul 29, 2021
@BruceForstall BruceForstall removed the untriaged New issue has not been triaged by the area owner label Jul 29, 2021
@BruceForstall BruceForstall modified the milestones: 7.0.0, 8.0.0 May 26, 2022
@jakobbotsch jakobbotsch mentioned this issue Oct 24, 2022
Merged
@BruceForstall BruceForstall modified the milestones: 8.0.0, Future Jun 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BruceForstall BruceForstall

Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Projects
None yet
Milestone
Future
Development

No branches or pull requests
1 participant
@BruceForstall