Test failure Regressions\\coreclr\\GitHub_22348\\Test22348\\Test22348.cmd #81120

JulieLeeMSFT · 2023-01-24T20:27:15Z

Failed in run: runtime-coreclr outerloop 20230119.3

Failed tests:

R2R-CG2 windows arm64 Checked @ Windows.11.Arm64.Open
  - Regressions\\coreclr\\GitHub_22348\\Test22348\\Test22348.cmd

Error message:

Unhandled exception. System.ArgumentNullException: Invalid handle.
at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
at System.Exception.get_StackTrace()
at System.Exception.ToString()
at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
at Internal.JitInterface.CorInfoImpl.resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 449
at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr& exception, IntPtr jit, IntPtr thisHandle, IntPtr callbacks, CORINFO_METHOD_INFO& info, UInt32 flags, IntPtr& nativeEntry, UInt32& codeSize)
at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841 at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775 at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileMethodList|2(IEnumerable1 methodList) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 735
at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(List1 obj) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 659 at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer2.ComputeDependencies(List1 deferredStaticDependencies) in /_/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 177 at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer2.ComputeMarkedNodes() in //src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 316
at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 343
at ILCompiler.Program.RunSingleCompilation(Dictionary2 inFilePaths, InstructionSetSupport instructionSetSupport, String compositeRootPath, Dictionary2 unrootedInputFilePaths, HashSet`1 versionBubbleModulesHash, ReadyToRunCompilerContext typeSystemContext) in //src/coreclr/tools/aot/crossgen2/Program.cs:line 616
at ILCompiler.Program.Run() in //src/coreclr/tools/aot/crossgen2/Program.cs:line 289
at ILCompiler.Crossgen2RootCommand.<>c__DisplayClass187_0.<.ctor>b__0(InvocationContext context) in /_/src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs:line 293
at System.CommandLine.Invocation.AnonymousCommandHandler.Invoke(InvocationContext context)
at Syste

Stack trace:

   at Regressions_coreclr._GitHub_22348_Test22348_Test22348_._GitHub_22348_Test22348_Test22348_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

The text was updated successfully, but these errors were encountered:

JulieLeeMSFT · 2023-01-24T20:28:04Z

Hi @mangod9, it is a one time failure in outerloop. PTAL. Please feel free to remove blocking label.

JulieLeeMSFT · 2023-01-24T21:52:21Z

This failed in #81121. Closed the duplicate there.

R2R-CG2 windows arm64 Checked @ Windows.11.Arm64.Open
  - Interop\\PInvoke\\SetLastError\\SetLastErrorTest\\SetLastErrorTest.cmd

AaronRobinsonMSFT · 2023-01-24T22:05:12Z

I did a brief investigation at #11360 (comment). The result was that the arrayTypeHandle was null. I'm not sure how we got that far however.

runtime/src/coreclr/vm/comutilnative.cpp

Lines 949 to 977 in 3d61658

    
           /*===============================AllocateNewArray=============================== 
        
           **Action: Allocates a new array object. Allows passing extra flags 
        
           **Returns: The allocated array. 
        
           **Arguments: elementTypeHandle -> type of the element, 
        
           **           length -> number of elements, 
        
           **           zeroingOptional -> whether caller prefers to skip clearing the content of the array, if possible. 
        
           **Exceptions: IDS_EE_ARRAY_DIMENSIONS_EXCEEDED when size is too large. OOM if can't allocate. 
        
           ==============================================================================*/ 
        
           FCIMPL3(Object*, GCInterface::AllocateNewArray, void* arrayTypeHandle, INT32 length, INT32 flags) 
        
           { 
        
               CONTRACTL { 
        
                   FCALL_CHECK; 
        
               } CONTRACTL_END; 
        
               OBJECTREF pRet = NULL; 
        
               TypeHandle arrayType = TypeHandle::FromPtr(arrayTypeHandle); 
        
               HELPER_METHOD_FRAME_BEGIN_RET_0(); 
        
               //Only the following flags are used by GC.cs, so we'll just assert it here. 
        
               _ASSERTE((flags & ~(GC_ALLOC_ZEROING_OPTIONAL | GC_ALLOC_PINNED_OBJECT_HEAP)) == 0); 
        
               pRet = AllocateSzArray(arrayType, length, (GC_ALLOC_FLAGS)flags); 
        
               HELPER_METHOD_FRAME_END(); 
        
               return OBJECTREFToObject(pRet); 
        
           } 
        
           FCIMPLEND

JulieLeeMSFT · 2023-01-24T22:48:05Z

This comment is related to #81109.

I did a brief investigation at #11360 (comment). The result was that the arrayTypeHandle was null. I'm not sure how we got that far however.

runtime/src/coreclr/vm/comutilnative.cpp

Lines 949 to 977 in 3d61658

/*===============================AllocateNewArray===============================

**Action: Allocates a new array object. Allows passing extra flags

**Returns: The allocated array.

**Arguments: elementTypeHandle -> type of the element,

** length -> number of elements,

** zeroingOptional -> whether caller prefers to skip clearing the content of the array, if possible.

**Exceptions: IDS_EE_ARRAY_DIMENSIONS_EXCEEDED when size is too large. OOM if can't allocate.

==============================================================================*/

FCIMPL3(Object*, GCInterface::AllocateNewArray, void* arrayTypeHandle, INT32 length, INT32 flags)

{

CONTRACTL {

FCALL_CHECK;

} CONTRACTL_END;

OBJECTREF pRet = NULL;

TypeHandle arrayType = TypeHandle::FromPtr(arrayTypeHandle);

HELPER_METHOD_FRAME_BEGIN_RET_0();

//Only the following flags are used by GC.cs, so we'll just assert it here.

_ASSERTE((flags & ~(GC_ALLOC_ZEROING_OPTIONAL | GC_ALLOC_PINNED_OBJECT_HEAP)) == 0);

pRet = AllocateSzArray(arrayType, length, (GC_ALLOC_FLAGS)flags);

HELPER_METHOD_FRAME_END();

return OBJECTREFToObject(pRet);

}

FCIMPLEND

v-wenyuxu · 2023-02-03T06:50:29Z

Failed again in: runtime-coreclr crossgen2 20230201.1

Failed test"

R2R-CG2 windows arm64 Checked @ Windows.11.Arm64.Open

- profiler\\eventpipe\\eventpipe_readevents\\eventpipe_readevents.cmd
- Interop\\MarshalAPI\\FunctionPointer\\FunctionPtrTest\\FunctionPtrTest.cmd

Error message:

Unhandled exception. System.ArgumentNullException: Invalid handle.
at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
at System.Exception.get_StackTrace()
at System.Exception.ToString()
at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
at Internal.JitInterface.CorInfoImpl.resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 449
at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr& exception, IntPtr jit, IntPtr thisHandle, IntPtr callbacks, CORINFO_METHOD_INFO& info, UInt32 flags, IntPtr& nativeEntry, UInt32& codeSize)
at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompilationThread|3(Object objThreadId) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 753

Return code:      1
Raw output file:      C:\h\w\B65E0A02\w\9C4808CD\uploads\Reports\Interop.MarshalAPI\FunctionPointer\FunctionPtrTest\FunctionPtrTest.output.txt
Raw output:
BEGIN EXECUTION
FunctionPointerNative.dll
FunctionPtrTest.dll
TestLibrary.dll
3 file(s) copied.
8:19:30.58
Response file: C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\FunctionPointerNative.dll.rsp
C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\IL-CG2\FunctionPointerNative.dll
-o:C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\FunctionPointerNative.dll
--targetarch:arm64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:\h\w\B65E0A02\p\System..dll
-r:C:\h\w\B65E0A02\p\Microsoft..dll
-r:C:\h\w\B65E0A02\p\mscorlib.dll
-r:C:\h\w\B65E0A02\p\netstandard.dll
-O
" "dotnet" "C:\h\w\B65E0A02\p\crossgen2\crossgen2.dll" @"C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\FunctionPointerNative.dll.rsp"   -r:C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\IL-CG2*.dll"
No input files are loadable
8:19:30.85
8:19:30.85
Response file: C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\FunctionPtrTest.dll.rsp
C:\h\w\B65E0A02\w\9C4808CD\e\Interop\MarshalAPI\FunctionPointer\FunctionPtrTest\IL-CG2\FunctionPtrTest.dll
-o:C:\h\w\B65E0A02


Stack trace
   at Interop_MarshalAPI._FunctionPointer_FunctionPtrTest_FunctionPtrTest_._FunctionPointer_FunctionPtrTest_FunctionPtrTest_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

v-wenyuxu · 2023-02-13T01:43:05Z

Failed again in: runtime-coreclr r2r 20230211.1

Failed tests:

R2R-CG2 windows arm64 Checked @ Windows.11.Arm64.Open

- JIT/Methodical/xxobj/sizeof/sizeof_Target_64Bit_and_arm_il_d/sizeof_Target_64Bit_and_arm.cmd

Error message:

Unhandled exception. System.ArgumentNullException: Invalid handle.
  at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
  at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
  at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
  at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
  at System.Exception.get_StackTrace()
  at System.Exception.ToString()
  at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
  at Internal.JitInterface.CorInfoImpl._resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 449
  at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr& exception, IntPtr jit, IntPtr thisHandle, IntPtr callbacks, CORINFO_METHOD_INFO& info, UInt32 flags, IntPtr& nativeEntry, UInt32& codeSize)
  at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
  at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompilationThread|3(Object objThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 753

Return code:      1
Raw output file:      C:hwB5CB0A35wABC408FCuploads\xxobjsizeofsizeof_Target_64Bit_and_arm_il_doutput.txt
Raw output:
BEGIN EXECUTION
sizeof_Target_64Bit_and_arm.dll
TestLibrary.dll
       2 file(s) copied.
7:16:28.53
Response file: C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dsizeof_Target_64Bit_and_arm.dll.rsp
C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dIL-CG2sizeof_Target_64Bit_and_arm.dll
-o:C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dsizeof_Target_64Bit_and_arm.dll
--targetarch:arm64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:hwB5CB0A35pSystem.*.dll
-r:C:hwB5CB0A35pMicrosoft.*.dll
-r:C:hwB5CB0A35pmscorlib.dll
-r:C:hwB5CB0A35p
etstandard.dll
-O
" "dotnet" "C:hwB5CB0A35pcrossgen2crossgen2.dll" @"C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dsizeof_Target_64Bit_and_arm.dll.rsp"   -r:C:hwB5CB0A35wABC408FCeJITMethodicalMethodical_d2IL-CG2*.dll  -r:C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dIL-CG2*.dll"
Emitting R2R PE file: C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dsizeof_Target_64Bit_and_arm.dll
" "dotnet" "C:hwB5CB0A35p
2rdump
2rdump.dll" --header --sc --in C:hwB5CB0A35wABC408FCeJITMethodical\xxobjsizeofsizeof_Target_64Bit_and_arm_il_dsizeof_Target_64Bit_and_arm.dll --out C:hwB5CB0A35wABC408FCeJI

Stack trace:

  at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
  at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String basePath, String assemblyPath)
  at Program.<<Main>$>g__TestExecutor7|0_6(<>c__DisplayClass0_0&)

BruceForstall · 2023-02-13T23:17:03Z

@dotnet/crossgen-contrib Another case of this failure happened yesterday:

Unhandled exception. System.ArgumentNullException: Invalid handle.
at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)

Unfortunately, the failures seem to occur when compiling various tests, so we can't just disable one or two specific tests to get the CI clean.

It looks like most reports have been win-arm64, but some are win-arm.

Can someone investigate?

@trylek @markples It looks like you have investigated the same issue over in #81109?

trylek · 2023-02-13T23:30:13Z

There's obviously something weird going on, at the first glance I would tend to rather speculate it might be related to

#75244

/cc @janvorli who was involved in investigation of a few similar issues; I'll try to find a way to repro this locally.

Thanks

Tomas

janvorli · 2023-02-13T23:38:07Z

@trylek I don't see any relation to the #75244. This seems to be happening in managed code of the crossgen while the other is a native runtime problem.

trylek · 2023-02-13T23:43:50Z

Thanks @janvorli for your feedback. I have reassigned the issue to myself and I'll investigate what's going on.

BruceForstall · 2023-02-14T18:18:56Z

Another case:

https://dev.azure.com/dnceng-public/public/_build/results?buildId=171029&view=ms.vss-test-web.build-test-results-tab&runId=3425228&resultId=117258&paneView=debug

JIT/Methodical/xxobj/sizeof/sizeof64_Target_64Bit_and_arm_il_d/sizeof64_Target_64Bit_and_arm.cmd

R2R-CG2 windows arm64 Checked @ Windows.11.Arm64.Open

11:36:02.939 Running test: JIT/Methodical/xxobj/sizeof/sizeof64_Target_64Bit_and_arm_il_d/sizeof64_Target_64Bit_and_arm.cmd
Unhandled exception. System.ArgumentNullException: Invalid handle.
   at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
   at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
   at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
   at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
   at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
   at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
   at System.Exception.get_StackTrace()
   at System.Exception.ToString()
   at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
   at Internal.JitInterface.CorInfoImpl._resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 449
   at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr& exception, IntPtr jit, IntPtr thisHandle, IntPtr callbacks, CORINFO_METHOD_INFO& info, UInt32 flags, IntPtr& nativeEntry, UInt32& codeSize)
   at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
   at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
   at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841
   at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775
   at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileMethodList|2(IEnumerable`1 methodList) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 735
   at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(List`1 obj) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 659
   at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeDependencies(List`1 deferredStaticDependencies) in /_/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 177
   at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes() in /_/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 316
   at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 343
   at ILCompiler.Program.RunSingleCompilation(Dictionary`2 inFilePaths, InstructionSetSupport instructionSetSupport, String compositeRootPath, Dictionary`2 unrootedInputFilePaths, HashSet`1 versionBubbleModulesHash, ReadyToRunCompilerContext typeSystemContext) in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 616
   at ILCompiler.Program.Run() in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 289
   at ILCompiler.Crossgen2RootCommand.<>c__DisplayClass187_0.<.ctor>b__0(InvocationContext context) in /_/src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs:line 293
   at System.CommandLine.Invocation.AnonymousCommandHandler.Invoke(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseVersionOption>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Invocation.InvocationPipeline.<Invoke>g__FullInvocationChain|3_0(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.Invoke(IConsole console)
   at System.CommandLine.Parsing.ParseResultExtensions.Invoke(ParseResult parseResult, IConsole console)
   at System.CommandLine.Parsing.ParserExtensions.Invoke(Parser parser, String[] args, IConsole console)
   at ILCompiler.Program.Main(String[] args) in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 892

Return code:      1
Raw output file:      C:\h\w\AD4C0959\w\B72F0977\uploads\xxobj\sizeof\sizeof64_Target_64Bit_and_arm_il_d\output.txt

trylek · 2023-02-16T01:38:04Z

Encouraging news - I've finally managed to locally repro this on my SurfacePro X after about 500 iterations of the DelegateTest. I'm looking into improving internal diagnostics and I'll be hopefully able to shed more details on the issue soon.

BruceForstall · 2023-02-16T22:26:39Z

Another case: Interop\StringMarshalling\VBByRefStr\VBByRefStrTest\VBByRefStrTest.cmd

https://dev.azure.com/dnceng-public/public/_build/results?buildId=174073&view=ms.vss-test-web.build-test-results-tab&runId=3460125&resultId=115189&paneView=debug

trylek · 2023-02-18T17:17:06Z

I understand this a bit more now. The exception thrown from _decodeToken is real and is caused by a bug in the CLRTest.Crossgen.targets script - we only provide System.☆.dll and Microsoft.☆.dll as refererences to Crossgen2 compilation so it's unable to resolve references to xunit.assert that are used within the tests. The exception falls through JIT to Crossgen2 which just marks the method as requiring runtime JIT so it's benign from the execution viewpoint however apparently there's a separate arm64-specific bug in the exception construction logic that occasionally crashes when filling in the exception stack trace information. In my above mention of local repro I probably misread the number on the SurfacePro screen, I think it was actually 5000 iterations, in my second repro attempt it failed after about 9000 iterations so the repro rate is actually pretty low. I'll ultimately fix the CLRTest.Crossgen.targets bug but I don't want to do that before the arm64-specific crash is understood and fixed because fixing the script would just hide the problem.

BruceForstall · 2023-02-24T19:09:26Z

Another case:

ilasm\PortablePdb\IlasmPortablePdbTests\IlasmPortablePdbTests.cmd

R2R-CG2 windows arm64 Checked jitstress1_tiered @ Windows.11.Arm64.Open

https://dev.azure.com/dnceng-public/public/_build/results?buildId=183267&view=ms.vss-test-web.build-test-results-tab&runId=3562043&resultId=122011&paneView=debug

trylek · 2023-02-26T20:43:29Z

Hmm, I tried to repro this locally but while I'm usually able to repro this after 5000~10000 iterations when running crossgen2 through dotnet, I was unable to repro this after about 30000 iterations when running crossgen2 through corerun. I tried it with checked build of corerun so I guess I'll try once more with release build of the runtime and, if that fails, I'll try to figure out how to catch the dotnet crash in the debugger.

v-wenyuxu · 2023-02-28T01:07:58Z

Failed again in: runtime-coreclr r2r-extra 20230226.1

Failed tests:

R2R-CG2 windows arm64 Checked jitstress2 @ Windows.11.Arm64.Open
    - JIT\\Regression\\VS-ia64-JIT\\V1.2-M02\\b27077\\b27077\\b27077.cmd

Error message:

Unhandled exception. System.ArgumentNullException: Invalid handle.
  at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
  at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
  at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
  at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
  at System.Exception.get_StackTrace()
  at System.Exception.ToString()
  at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
  at Internal.JitInterface.CorInfoImpl._resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 449
  at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr& exception, IntPtr jit, IntPtr thisHandle, IntPtr callbacks, CORINFO_METHOD_INFO& info, UInt32 flags, IntPtr& nativeEntry, UInt32& codeSize)
  at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
  at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileMethodList|2(IEnumerable`1 methodList) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 735
  at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(List`1 obj) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 659
  at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes() in /_/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 316
  at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 343
  at ILCompiler.Program.RunSingleCompilation(Dictionary`2 inFilePaths, InstructionSetSupport instructionSetSupport, String compositeRootPath, Dictionary`2 unrootedInputFilePaths, HashSet`1 versionBubbleModulesHash, ReadyToRunCompilerContext typeSystemContext) in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 616
  at ILCompiler.Program.Run() in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 289
  at ILCompiler.Crossgen2RootCommand.<>c__DisplayClass187_0.<.ctor>b__0(InvocationContext context) in /_/src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs:line 293
  at System.CommandLine.Invocation.AnonymousCommandHandler.Invoke(InvocationContext context)
  at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
  at System.CommandLine.CommandLineBuilderExtensions.<>c__Dis

Stack trace:

  at JIT_Regression._VS_ia64_JIT_V1_2_M02_b27077_b27077_b27077_._VS_ia64_JIT_V1_2_M02_b27077_b27077_b27077_cmd()
  at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
  at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

markples · 2023-03-06T21:49:33Z

@trylek Have you had any more luck with the repro attempts?

We have both this issue and #81109 open. It appears that both have the same initial symptom (resolveToken). 81109 then fails in

Fatal error. Internal CLR error. (0x80131506)
at System.GC.AllocateNewArray(IntPtr, Int32, GC_ALLOC_FLAGS)

whereas this one has

Unhandled exception. System.ArgumentNullException: Invalid handle.
at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)

Until previously, 81109 was arm and this one was arm64, but 81109 now also has an arm64 repro. Neither call stack makes a lot of sense (agree?) so do you think it is reasonable to guess that both are coming from some sort of corruption (possibly something arm{,64}-specific like memory model)?

trylek · 2023-03-06T22:13:06Z

I am able to reproduce this locally on my arm64 Surface Pro X but the repro rate is low - it typically takes 5000 to 20000 iterations to trigger the failure - and so far I have been mostly unsuccessful trying to repro this using instrumented coreclr that would let me investigate the issue in more detail in the debugger. Yesterday was the only time I managed to hit a repro with the instrumented runtime so it opened the Just-in-Time debugging dialog but when I confirmed I want to JIT debug the failure in the installed VS 2022 preview, it hung so it was no use. I keep trying but I don't have any great ideas how to debug the failure without using an instrumented local build with symbols etc.

jkotas · 2023-03-07T17:08:14Z

Failed again in: runtime-coreclr r2r-extra 20230226.1

This is crashing in the LKG runtime that comes with SDK (8.0.0-alpha.1.23058.2 built on Jan 8 2023).

The LKG runtime that comes with the SDK was updated last week: https://github.com/dotnet/runtime/pull/82599/files#diff-8df3cd354bc584349d04ad5675b33c042d8b99b741b8b95af394c55e0f5001bfL3

Have we seen any instances of these crashed since the LKG runtime was updated?

trylek · 2023-03-07T17:42:04Z

This is an interesting observation as I've spent several days trying to repro this with the newest CoreCLR runtime without any success while I'm able to make it crash relatively easily with the older SDK. It would be great to hear that it's fixed now.

markples · 2023-03-07T19:19:29Z

a40d411#diff-8df3cd354bc584349d04ad5675b33c042d8b99b741b8b95af394c55e0f5001bf updated it on Jan 12 from

  "sdk": {
    "version": "7.0.100",

to

  "sdk": {
    "version": "8.0.100-alpha.1.23061.8",

Earliest known failure was in runtime-coreclr outerloop 20230119.3, which seems to support this theory.

trylek · 2023-03-07T19:20:58Z

I'm keeping my fingers crossed...

v-wenyuxu · 2023-03-08T01:13:33Z

Failed again in: runtime-coreclr r2r 20230306.1

Failed test:

R2R-CG2 windows arm Checked no_tiered_compilation @ Windows.11.Arm64.Open

- Loader\\classloader\\generics\\Instantiation\\Negative\\param05\\param05.cmd

Error message:

Unhandled exception. System.ArgumentNullException: Invalid handle.
at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
at System.Exception.get_StackTrace()
at System.Exception.ToString()
at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
at Internal.JitInterface.CorInfoImpl.getClassSize(IntPtr thisHandle, IntPtr* ppException, CORINFO_CLASS_STRUCT* cls) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 775
at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in //src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841 at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775 at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileMethodList|2(IEnumerable1 methodList) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 735
at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(List1 obj) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 659 at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer2.ComputeMarkedNodes() in //src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 316
at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile) in //src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 343
at ILCompiler.Program.RunSingleCompilation(Dictionary2 inFilePaths, InstructionSetSupport instructionSetSupport, String compositeRootPath, Dictionary2 unrootedInputFilePaths, HashSet`1 versionBubbleModulesHash, ReadyToRunCompilerContext typeSystemContext) in //src/coreclr/tools/aot/crossgen2/Program.cs:line 616
at ILCompiler.Program.Run() in //src/coreclr/tools/aot/crossgen2/Program.cs:line 289
at ILCompiler.Crossgen2RootCommand.<>c__DisplayClass187_0.<.ctor>b__0(InvocationContext context) in //src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs:line 293
at System.CommandLine.Invocation.AnonymousCommandHandler.Invoke(InvocationContext context)
at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<<UseHelp>b__0>d.MoveNext()

Stack trace
   at Loader_classloader._generics_Instantiation_Negative_param05_param05_._generics_Instantiation_Negative_param05_param05_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

v-wenyuxu · 2023-03-16T01:04:17Z

Failed in Run: runtime-coreclr outerloop 20230315.3

Failed tests:

R2R-CG2 linux arm64 Checked @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - ilverify/ILVerificationTests.sh

Error message:

/root/helix/work/workitem/e/ilverify/ILVerificationTests.sh: line 305: -r:/root/helix/work/workitem/e/ilverify/IL-CG2/*.dll: No such file or directory
Unhandled exception. System.ArgumentNullException: Invalid handle.
  at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
  at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
  at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
  at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
  at System.Exception.get_StackTrace()
  at System.Exception.ToString()
  at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
  at Internal.JitInterface.CorInfoImpl._resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 449
/root/helix/work/workitem/e/ilverify/ILVerificationTests.sh: line 246:    47 Aborted                 (core dumped) $__Command

Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/Reports/ilverify/ILVerificationTests.output.txt
Raw output:
BEGIN EXECUTION
in takeLock
/root/helix/work/workitem/e/ilverify/IL-CG2/ILVerificationTests.dll
19:42:24
Response file: /root/helix/work/workitem/e/ilverify/ILVerificationTests.dll.rsp
/root/helix/work/workitem/e/ilverify/IL-CG2/ILVerificationTests.dll
-o:/root/helix/work/workitem/e/ilverify/ILVerificationTests.dll
-r:/root/helix/work/correlation/System.*.dll
-r:/root/helix/work/correlation/Microsoft.*.dll
-r:/root/helix/work/correlation/xunit.*.dll
-r:/root/helix/work/correlation/mscorlib.dll
--verify-type-and-field-layout
--method-layout:random
--targetarch:arm64
--targetos:linux
-O
Running CrossGen2:  dotnet /root/helix/work/correlation/crossgen2/crossgen2.dll @/root/helix/work/workitem/e/ilverify/ILVerificationTests.dll.rsp  
[createdump] Gathering state for process 47 dotnet
[createdump] Crashing thread 00000038 signal 00000006
[createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.47.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.47.dmp
[createdump] Written 136974336 bytes (33441 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 3833ms
Running R2RDump:  dotnet /root/helix/work/correlation/R2RDump/R2RDump.dll --header --sc --in /root/helix/work/workitem/e/ilverify/ILVerificationTests.dll --out /root/helix/work/workitem/e/ilverify/ILVerificationTests.dll.r2rdump --val
Error: System.BadImageFormatException: The file is not a ReadyToRun image
  at ILCompiler.Reflection.ReadyToRun.ReadyToRunReader.Initialize(IAssemblyMetadata metadata) in /_/src/coreclr/tools/aot/ILCompiler.Reflection.ReadyToRun/ReadyToRunReader.cs:line 470
  at ILCompiler.Reflection.ReadyToRun.ReadyToRunReader..ctor(IAssemblyResolver assemblyResolver, String filename) in /_/src/coreclr/tools/aot/ILCompiler.Reflection.ReadyToRun/ReadyToRunReader.cs:line 406
  at R2RDump.Program.Run() in /_/src/coreclr/tools/r2rdump/Program.cs:line 447
19:42:33
Crossgen2 failed with exitcode: 134
in ReleaseLock
Test failed. Trying to see if dump file was created in /home/helixbot/dotnetbuild/dumps since 3/15/2023 7:42:24 PM
Processing /home/helixbot/dotnetbuild/dumps/coredump.47.dmp.crashreport.json
Printing stacktrace from '/home/helixbot/dotnetbuild/dumps/coredump.47.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pr

Stack trace:

  at ilverify._ILVerificationTests_._ILVerificationTests_sh()
  at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
  at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2023-03-15T19:12:44.911Z	runtime-coreclr outerloop	Checked	ubuntu.1804.armarch.open	arm64	ilverify/ILVerificationTests.sh	Failed	{"configuration":"Checked","Reason":"Schedule","System.StageName":"Build","architecture":"arm64","AzurePipelinesTestRunId":"3832248","System.PhaseName":"run_test_p1_R2R_CG2_linux_arm64_checked","BuildNumber":"20230315.3","BuildId":"205850","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","System.JobAttempt":"1","DefinitionName":"runtime-coreclr outerloop","operatingSystem":"(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8","System.StageAttempt":"1","System.JobId":"2e21645a-5873-5561-8d4d-44187d33621f","System.JobName":"__default","DefinitionId":"108","Project":"public"}	Schedule
2023-01-15T14:40:22.35Z	runtime-coreclr gcstress0x3-gcstress0xc	Checked-gcstress0x3	ubuntu.1804.armarch.open	arm	ilverify/ILVerificationTests.sh	Failed	{"System.StageName":"Build","System.JobName":"__default","DefinitionId":"112","Reason":"Schedule","System.JobAttempt":"1","System.StageAttempt":"1","Project":"public","BuildNumber":"20230115.1","System.PhaseName":"run_test_p1__linux_arm_checked","DefinitionName":"runtime-coreclr gcstress0x3-gcstress0xc","operatingSystem":"(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7","configuration":"Checked-gcstress0x3","architecture":"arm","System.JobId":"860c965f-b28e-50d5-65d2-e40bd59f3a69","BuildId":"138186","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"3004928","CollectionUri":"https://dev.azure.com/dnceng-public/"}	Schedule
2023-01-14T14:54:53.012Z	runtime-coreclr gcstress0x3-gcstress0xc	Checked-gcstress0x3	ubuntu.1804.armarch.open	arm	ilverify/ILVerificationTests.sh	Failed	{"configuration":"Checked-gcstress0x3","System.PhaseName":"run_test_p1__linux_arm_checked","DefinitionId":"112","AzurePipelinesTestRunId":"2997718","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobAttempt":"1","System.StageAttempt":"1","System.JobId":"860c965f-b28e-50d5-65d2-e40bd59f3a69","Reason":"Schedule","architecture":"arm","System.JobName":"__default","BuildId":"137723","BuildNumber":"20230114.1","System.StageName":"Build","System.PhaseAttempt":"1","DefinitionName":"runtime-coreclr gcstress0x3-gcstress0xc","Project":"public","operatingSystem":"(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7"}	Schedule
2022-11-19T21:17:27.562Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	osx.1200.amd64.open	x64	ilverify/ILVerificationTests.sh	Failed	{"DefinitionName":"runtime-coreclr ilasm","System.PhaseAttempt":"1","architecture":"x64","System.PhaseName":"run_test_p1__OSX_x64_checked","System.JobName":"__default","configuration":"Checked-ilasmroundtrip","System.StageAttempt":"1","System.JobAttempt":"1","BuildId":"88711","DefinitionId":"140","AzurePipelinesTestRunId":"1867570","System.StageName":"Build","operatingSystem":"OSX.1200.Amd64.Open","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","CollectionUri":"https://dev.azure.com/dnceng-public/","Project":"public","Reason":"Schedule","BuildNumber":"20221119.1"}	Schedule
2022-11-19T21:16:39.834Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	osx.1200.arm64.open	arm64	ilverify/ILVerificationTests.sh	Failed	{"AzurePipelinesTestRunId":"1867566","BuildNumber":"20221119.1","System.StageName":"Build","CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","System.JobName":"__default","DefinitionName":"runtime-coreclr ilasm","Project":"public","architecture":"arm64","configuration":"Checked-ilasmroundtrip","operatingSystem":"OSX.1200.ARM64.Open","DefinitionId":"140","System.JobAttempt":"1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.PhaseName":"run_test_p1__OSX_arm64_checked","System.PhaseAttempt":"1","BuildId":"88711"}	Schedule
2022-11-19T20:51:51.061Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.amd64.open.rt	x64	ilverify/ILVerificationTests.sh	Failed	{"System.StageName":"Build","DefinitionId":"140","BuildId":"88711","Project":"public","System.JobAttempt":"1","configuration":"Checked-ilasmroundtrip","System.StageAttempt":"1","AzurePipelinesTestRunId":"1867338","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionName":"runtime-coreclr ilasm","BuildNumber":"20221119.1","System.PhaseName":"run_test_p1__Linux_x64_checked","operatingSystem":"Ubuntu.1804.Amd64.Open","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"x64","System.JobId":"6dfec640-5b96-5d95-2a5b-7c236ea2d5c5","Reason":"Schedule"}	Schedule
2022-11-19T20:51:17.451Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.armarch.open	arm64	ilverify/ILVerificationTests.sh	Failed	{"architecture":"arm64","System.JobId":"34b2f53b-add1-5e74-7a39-5a11483297fc","BuildId":"88711","Project":"public","System.PhaseName":"run_test_p1__Linux_arm64_checked","DefinitionId":"140","System.JobName":"__default","CollectionUri":"https://dev.azure.com/dnceng-public/","AzurePipelinesTestRunId":"1867336","Reason":"Schedule","operatingSystem":"(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8","System.StageAttempt":"1","System.PhaseAttempt":"1","BuildNumber":"20221119.1","System.StageName":"Build","System.JobAttempt":"1","configuration":"Checked-ilasmroundtrip","DefinitionName":"runtime-coreclr ilasm"}	Schedule
2022-11-19T20:48:25.822Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.armarch.open	arm	ilverify/ILVerificationTests.sh	Failed	{"BuildNumber":"20221119.1","operatingSystem":"(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7","AzurePipelinesTestRunId":"1867310","DefinitionId":"140","System.StageName":"Build","System.JobId":"860c965f-b28e-50d5-65d2-e40bd59f3a69","architecture":"arm","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobAttempt":"1","DefinitionName":"runtime-coreclr ilasm","System.PhaseName":"run_test_p1__Linux_arm_checked","BuildId":"88711","System.JobName":"__default","Reason":"Schedule","System.StageAttempt":"1","Project":"public","System.PhaseAttempt":"1","configuration":"Checked-ilasmroundtrip"}	Schedule

v-wenyuxu · 2023-03-27T01:06:59Z

Failed in Run: runtime-coreclr r2r-extra 20230325.1

Failed tests:

R2R-CG2 windows arm64 Checked jitstressregs0x2000 @ Windows.11.Arm64.Open

- Interop\\PInvoke\\CustomMarshalers\\CustomMarshalersTest\\CustomMarshalersTest.cmd

Error message:

Unhandled exception. System.ArgumentNullException: Invalid handle.
  at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, RuntimeMethodHandleInternal methodHandle)
  at System.RuntimeType.GetMethodBase(RuntimeType reflectedType, IRuntimeMethodInfo methodHandle)
  at System.Diagnostics.StackFrameHelper.GetMethodBase(Int32 i)
  at System.Diagnostics.StackFrame..ctor(StackFrameHelper stackFrameHelper, Int32 skipFrames, Boolean needFileInfo)
  at System.Diagnostics.StackTrace.CaptureStackTrace(Int32 skipFrames, Boolean fNeedFileInfo, Exception e)
  at System.Exception.get_StackTrace()
  at System.Exception.ToString()
  at Internal.JitInterface.CorInfoImpl.AllocException(Exception ex) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 164
  at Internal.JitInterface.CorInfoImpl._pInvokeMarshalingRequired(IntPtr thisHandle, IntPtr* ppException, CORINFO_METHOD_STRUCT_* method, CORINFO_SIG_INFO* callSiteSig) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 332
  at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr& exception, IntPtr jit, IntPtr thisHandle, IntPtr callbacks, CORINFO_METHOD_INFO& info, UInt32 flags, IntPtr& nativeEntry, UInt32& codeSize)
  at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 355
  at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 659
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 841
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 775
  at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass46_0.<ComputeDependencyNodeDependencies>g__CompileMethodList|2(IEnumerable`1 methodList) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 735
  at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(List`1 obj) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 659
  at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes() in /_/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 316
  at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 343
  at ILCompiler.Program.RunSingleCompilation(Dictionary`2 inFilePaths, InstructionSetSupport instructionSetSupport, String compositeRootPath, Dictionary`2 unrootedInputFilePaths, HashSet`1 versionBubbleModulesHash, ReadyToRunCompilerContext typeSystemContext) in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 616
  at ILCompiler.Program.Run() in /_/src/coreclr/tools/aot/crossgen2/Program.cs:line 289
  at ILCompiler.Crossgen2RootCommand.<>c__DisplayClass187_0.<.ctor>b__0(InvocationContext context) in /_/src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs:line 293
  at System.CommandLine.Invocation.AnonymousCommandHandler.Invoke(InvocationContext context)
  at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
  at System.CommandLine.C

Stack trace:

  at Interop_PInvoke._CustomMarshalers_CustomMarshalersTest_CustomMarshalersTest_._CustomMarshalers_CustomMarshalersTest_CustomMarshalersTest_cmd()
  at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
  at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

markples · 2023-04-17T17:50:25Z

Closing alongside #81109 as SDK has been updated to preview.3

JulieLeeMSFT added area-VM-coreclr blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs labels Jan 24, 2023

JulieLeeMSFT assigned mangod9 Jan 24, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Jan 24, 2023

runfoapp bot mentioned this issue Jan 24, 2023

Infrastructure - Status/Health #702

Closed

JulieLeeMSFT mentioned this issue Jan 24, 2023

Test failure Interop\\PInvoke\\SetLastError\\SetLastErrorTest\\SetLastErrorTest.cmd #81121

Closed

This was referenced Jan 24, 2023

Test failure JIT/Methodical/ELEMENT_TYPE_IU/i_array_merge_Target_32BIT_il_d/i_array_merge_Target_32BIT.cmd #81116

Closed

Test failure JIT/Methodical/xxobj/sizeof/sizeof64_Target_64Bit_and_arm_il_r/sizeof64_Target_64Bit_and_arm.cmd #81109

Closed

trylek assigned trylek and unassigned mangod9 Feb 13, 2023

trylek mentioned this issue Mar 14, 2023

Add references to xunit.*.dll to CLRTest.Crossgen.targets #83413

Merged

markples closed this as completed Apr 17, 2023

ghost locked as resolved and limited conversation to collaborators May 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failure Regressions\\coreclr\\GitHub_22348\\Test22348\\Test22348.cmd #81120

Test failure Regressions\\coreclr\\GitHub_22348\\Test22348\\Test22348.cmd #81120

JulieLeeMSFT commented Jan 24, 2023

JulieLeeMSFT commented Jan 24, 2023

JulieLeeMSFT commented Jan 24, 2023

AaronRobinsonMSFT commented Jan 24, 2023

JulieLeeMSFT commented Jan 24, 2023

v-wenyuxu commented Feb 3, 2023

v-wenyuxu commented Feb 13, 2023

BruceForstall commented Feb 13, 2023

trylek commented Feb 13, 2023

janvorli commented Feb 13, 2023

trylek commented Feb 13, 2023

BruceForstall commented Feb 14, 2023

trylek commented Feb 16, 2023

BruceForstall commented Feb 16, 2023

trylek commented Feb 18, 2023 •

edited

Loading

BruceForstall commented Feb 24, 2023

trylek commented Feb 26, 2023

v-wenyuxu commented Feb 28, 2023

markples commented Mar 6, 2023

trylek commented Mar 6, 2023

jkotas commented Mar 7, 2023

trylek commented Mar 7, 2023

markples commented Mar 7, 2023

trylek commented Mar 7, 2023

v-wenyuxu commented Mar 8, 2023

v-wenyuxu commented Mar 16, 2023

v-wenyuxu commented Mar 27, 2023

markples commented Apr 17, 2023

Test failure Regressions\\coreclr\\GitHub_22348\\Test22348\\Test22348.cmd #81120

Test failure Regressions\\coreclr\\GitHub_22348\\Test22348\\Test22348.cmd #81120

Comments

JulieLeeMSFT commented Jan 24, 2023

JulieLeeMSFT commented Jan 24, 2023

JulieLeeMSFT commented Jan 24, 2023

AaronRobinsonMSFT commented Jan 24, 2023

JulieLeeMSFT commented Jan 24, 2023

v-wenyuxu commented Feb 3, 2023

v-wenyuxu commented Feb 13, 2023

BruceForstall commented Feb 13, 2023

trylek commented Feb 13, 2023

janvorli commented Feb 13, 2023

trylek commented Feb 13, 2023

BruceForstall commented Feb 14, 2023

trylek commented Feb 16, 2023

BruceForstall commented Feb 16, 2023

trylek commented Feb 18, 2023 • edited Loading

BruceForstall commented Feb 24, 2023

trylek commented Feb 26, 2023

v-wenyuxu commented Feb 28, 2023

markples commented Mar 6, 2023

trylek commented Mar 6, 2023

jkotas commented Mar 7, 2023

trylek commented Mar 7, 2023

markples commented Mar 7, 2023

trylek commented Mar 7, 2023

v-wenyuxu commented Mar 8, 2023

v-wenyuxu commented Mar 16, 2023

v-wenyuxu commented Mar 27, 2023

markples commented Apr 17, 2023

trylek commented Feb 18, 2023 •

edited

Loading