Repository navigation
Test failure: NullReferenceException in Internal.JitInterface.CorInfoImpl._beginInlining #105441
Description
Failed in: runtime-coreclr outerloop 20240724.3
Failed tests:
R2R-CG2 linux arm64 Checked @ (Alpine.317.Arm64.Open)Ubuntu.2004.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.17-helix-arm64v8
- Regressions/coreclr/GitHub_87879/test87879/test87879.cmd
Error message:
/root/helix/work/workitem/e/Regressions/Regressions/../coreclr/GitHub_87879/test87879/test87879.sh: line 315: -r:/root/helix/work/workitem/e/Regressions/coreclr/GitHub_87879/test87879/IL-CG2/*.dll: No such file or directory
Unhandled exception. ILCompiler.CodeGenerationFailedException: Code generation failed for method '[test87879]__GeneratedMainWrapper..ctor()'
---> System.NullReferenceException: Object reference not set to an instance of an object.
at Internal.JitInterface.CorInfoImpl._beginInlining(IntPtr thisHandle, IntPtr* ppException, CORINFO_METHOD_STRUCT_* inlinerHnd, CORINFO_METHOD_STRUCT_* inlineeHnd) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 154
--- End of inner exception stack trace ---
at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 381
at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 810
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 898
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 833
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompilationThread|3(Object objThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 811
Unhandled exception. System.ArgumentNullException: Value cannot be null. (Parameter 'array')
at System.Array.Clear(Array array, Int32 index, Int32 length)
at Internal.JitInterface.CorInfoImpl.CompileMethodCleanup() in /_/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 700
at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 826
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 898
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 833
at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompilationThread|3(Object objThreadId) in /_/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 811
/root/helix/work/workitem/e/Regressions/Regressions/../coreclr/GitHub_87879/test87879/test87879.sh: line 254: 4711 Aborted (core dumped) $__Command
Return code: 1
Raw output file: /root/helix/work/workitem/uploads/coreclr/GitHub_87879/test87879/output.txt
Raw output:
BEGIN EXECUTION
in takeLock
/root/helix/work/workitem/e/Regressions/coreclr/GitHub_87879/test87879/IL-CG2/test87879.dll
19:50:09
Response file: /root/helix/work/workitem/e/Regressions/coreclr/GitHub_87879/test87879/test87879.dll.rsp
/root/helix/work/workitem/e/Regressions/coreclr/GitHub_87879/test87879/IL-CG2/test87879.dll
-o:/root/helix/work/workitem/e/Regressions/coreclr/GitHub_87879/test87879/test87879.dll
-r:/root/helix/work/correlation/System.*.dll
-r:/root/helix/work/correlation/Microsoft.*.
Stack trace:
at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
at Program.<<Main>$>g__TestExecutor74|0_75(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
Known Issue Error Message
Fill the error message using step by step known issues guidance.
{
"ErrorMessage": ["NullReferenceException","_beginInlining"],
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": false
}Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=753860
Error message validated: [NullReferenceException _beginInlining]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/27/2024 1:53:56 AM UTC
Report
| Build | Definition | Step Name | Console log | Pull Request |
|---|---|---|---|---|
| 2582592 | dotnet-runtime | Build managed CoreCLR and host components, all libraries, and packs | Log | |
| 2580533 | dotnet-runtime | Build managed CoreCLR and host components, all libraries, and packs | Log | |
| 2578580 | dotnet-runtime | Build managed CoreCLR and host components, all libraries, and packs | Log | |
| 2575423 | dotnet-runtime | Build managed CoreCLR and host components, all libraries, and packs | Log |
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 0 | 0 | 4 |
Metadata
Metadata
Assignees
Labels
Use this to report build issues in the .NET Helix tabCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIBlocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runsThere is an active PR which will close this issue when it is mergedLinux OS (any supported distro)
Activity
[-]Test failure: Regressions/coreclr/GitHub_87879/test87879/test87879.cmd[/-][+]Test failure: NullReferenceException in Internal.JitInterface.CorInfoImpl._beginInlining[/+]jkotas commentedon Jul 25, 2024
@steveisok We have picked up intermittent crossgen crash on arm64 in
_beginInliningmethod. It is causing a lot of noise (see duplicate issues). Could you please take a look?jkotas commentedon Jul 25, 2024
I think it was likely introduced by the #104696 toolset update, so it is not easy to roll-back.
steveisok commentedon Jul 25, 2024
Yep, I can have someone take a closer look.
v-wenyuxu commentedon Jul 26, 2024
Failed in: runtime-coreclr outerloop 20240725.5
Failed tests:
Error message:
Stack trace:
jakobbotsch commentedon Jul 26, 2024
This might be the same issue as #102370 and #104123.
JIT: Fix placement of `GT_START_NOGC` for tailcalls in face of bulk c…
JIT: Fix placement of `GT_START_NOGC` for tailcalls in face of bulk c…
85 remaining items
jkotas commentedon Sep 27, 2024
SetClassInitedon dynamic statics does is done using InterlockedCompareChange that should be full store release barrier:runtime/src/coreclr/vm/methodtable.h
Line 598 in c20bdf6
I think we are missing barrier on the reader side as @EgorBo pointed in #105441 (comment)
AndyAyersMS commentedon Sep 27, 2024
Ah, ok. What's (a little) surprising is that even in Tier0 the JIT is baking the static address into codegen.
So the helper is just called for effect and the static field load afterwards is not data dependent on it.
Seems like for Tier0 code would be more compact if we didn't do this.
davidwrighton commentedon Sep 27, 2024
Ah, that's the problem. This really needs to either use the result of the helper, or have a barrier inserted. We could insert a barrier by adding a barrier to the code of the static base helper, or into the generated code stream any time after the call to GET_GCSTATIC_BASE. Any one of these 3 solutions would work. We could probably get the JIT to do the right thing with a simple change to the jit interface, where we simply refuse to provide a pre-computed address for a static without requiring that the class be initialized. I'll put together a proposed fix with that approach, so we can see what it looks like.
davidwrighton commentedon Sep 27, 2024
@EgorBo I don't have a setup for actually running any arm64 code available tonight, but I think either of the two PR's I just linked to this issue should solve this problem. Could you try them out? The code to have the jit interface force the use of helpers is probably not a final change (we wouldn't need that logic on any platform with a TSO based memory model), but it should be enough to verify that it works.
AndyAyersMS commentedon Sep 27, 2024
I can do some testing.
EgorBo commentedon Sep 27, 2024
#108309 didn't fix the issue for me, but #108311 did
AndyAyersMS commentedon Sep 27, 2024
#108311 works for me too, didn't try the other one.
davidwrighton commentedon Sep 27, 2024
Well, I'm going to want to investigate why #108309 didn't work later, since I'm kinda surprised it didn't fix that test case, so I'm not understanding some part of this system, which isn't good, but the barriers are strictly a more complete solution, so let's go with that.
AndyAyersMS commentedon Sep 27, 2024
I think it also needs fixes in
getFieldInfoand maybe other places.Seems like giving the JIT static addresses before the class is initialized is asking for trouble. Also (as above) it is creating bigger code. So maybe we want both of these?
jkotas commentedon Sep 27, 2024
It can expose bugs elsewhere in the system as we have seen here, but I do not think there are fundamental problems with it.
It is creating bigger code only when the JIT is not inlining the cctor check (e.g. optimizations are off).
jakobbotsch commentedon Oct 24, 2024
@jkotas Do you think this is likely to still reproduce in CI even if the underlying issue turns out to not have been fixed? IIUC the testing strategy for crossgen2 has changed significantly recently.
jkotas commentedon Oct 24, 2024
In main branch, both crossgen2 and ilc run on runtime specified in https://github.com/dotnet/runtime/blob/main/global.json#L8 . This version is at 9.0.100-rc.2.24474.11 currently. 9.0.100-rc.2.24474.11 does not contain fix #108347. This failure should be gone once we update global.json to rtm.
EgorBo commentedon Nov 27, 2024
🎉