Repository navigation
Stack overflow caused by SqlConfigurableRetryLogicLoader.Defaul_resolving #2214
Copy link
Copy link
Open
Description
Describe the bug
I have an Blazor web app on .NET 7, using EF and Microsoft.Data.SqlClient 5.0.2. The web app was running fine for several months. It crashed this morning, with the following exception:
Stack overflow.
at System.RuntimeMethodHandle._GetCurrentMethod(System.Threading.StackCrawlMark ByRef)
at System.Reflection.MethodBase.GetCurrentMethod()
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.Default_Resolving(System.Runtime.Loader.AssemblyLoadContext, System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr, System.Reflection.AssemblyName)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.MakeFullPath(System.String, System.String, System.String)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.Default_Resolving(System.Runtime.Loader.AssemblyLoadContext, System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr, System.Reflection.AssemblyName)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.MakeFullPath(System.String, System.String, System.String)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.Default_Resolving(System.Runtime.Loader.AssemblyLoadContext, System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr, System.Reflection.AssemblyName)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.MakeFullPath(System.String, System.String, System.String)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.Default_Resolving(System.Runtime.Loader.AssemblyLoadContext, System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr, System.Reflection.AssemblyName)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.MakeFullPath(System.String, System.String, System.String)
at Microsoft.Data.SqlClient.SqlConfigurableRetryLogicLoader.Default_Resolving(System.Runtime.Loader.AssemblyLoadContext, System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(System.Reflection.AssemblyName)
at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr, System.Reflection.AssemblyName)
...
To reproduce
Have restarted the app, but was not (yet) successful to repro.
Didn't configure any special options, environment variables, etc. AFAIK. This is the connection string (password etc. changed):
Data Source=tcp:super-secret.database.windows.net,1433;Initial Catalog=mywebsite.com;Integrated Security=False;Persist Security Info=False;User ID=super-secret;Password=123456;Encrypt=True
Further technical details
Microsoft.Data.SqlClient version: 5.0.2
.NET target: 7.0
SQL Server version: Azure SQL
Operating system: Ubuntu 20.04.3 LTS
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
- StatusShow more project fieldsBacklog
- StatusShow more project fieldsUnder Investigation
Activity
ErikEJ commentedon Nov 12, 2023
Have you tried 5.1.2 ?
sfiruch commentedon Nov 13, 2023
Yes, this is the version that's running now.
I didn't notice any changes in the relevant areas (but didn't look very hard). It'll take months to determine whether the problem is fixed. I had hoped that the person working in this area could make sense of the stack overflow immediately.
arellegue commentedon Nov 25, 2023
@deiruch, before I can begin investigation, may I ask for a repro code, please? I would like to know how SqlConfigurableRetryLogic is configured in your code.
sfiruch commentedon Nov 26, 2023
I only have a full website that showed the behavior after running for several months. The source code is private. Should I share the code with you?
The code is not special at all: It's using EF Core 7.0.13.0 with default options for database access. I didn't configure
SqlConfigurableRetryLogicmanually.arellegue commentedon Dec 5, 2023
@deiruch, could you at least provide information such as:
sfiruch commentedon Dec 6, 2023
Sure, I'm happy to! But I fear there's nothing of interest in there. It's really just using defaults.
App.config
Does not exist
appsettings.json
{ "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning" } }, "EmailSender": { "Host": "smtp-pulse.com", "Port": 2525, "EnableSSL": false, "UserName": "123", "Password": "123", "From": "123@secret.com" }, "AllowedHosts": "*", "LettuceEncrypt": { "AcceptTermsOfService": true, "DomainNames": [ "secret.com" ], "EmailAddress": "123@secret.com", "AllowedChallengeTypes": "Http01", "UseStagingServer": false }, "https_port": 443, "Kestrel": { "Endpoints": { "Http": { "Url": "http://*:80" }, "Https": { "Url": "https://*:443" } } } }OnConfiguring
Fix | Scope configurable retry logic assembly resolution to opt-in ca…
Fix | Scope configurable retry logic assembly resolution to opt-in ca…
Bump the nuget group with 6 updates (#61)
priyankatiwari08 commentedon Sep 17, 2026
/triage
github-actions commentedon Sep 17, 2026
🔍 Triage Summary (on-demand re-triage)
Area\EngineeringAnalysis
The reported
StackOverflowExceptionoriginates inSqlConfigurableRetryLogicLoader.Default_Resolving, where theAssemblyLoadContext.Default.Resolvingevent handler callsMakeFullPath/LoadFromAssemblyPath, which under certain conditions (e.g. failing to find the assembly, or a load failure that re-triggers resolution) appears to recursively re-enter the sameResolvingevent, eventually overflowing the stack. This is severe (P1) since aStackOverflowExceptionis fatal and unrecoverable, crashing the entire process without warning, and it's non-deterministic (the reporter could not repro on demand after a restart, suggesting a race condition or state built up over time, consistent with the AssemblyLoadContext resolving handler being globally shared/reentrant). No repro code is available, and the reporter's usage is default EF CoreUseSqlServerconfiguration without explicit configurable-retry-logic settings, meaning the resolving handler was likely subscribed unintentionally or left subscribed longer than necessary. This aligns with the current in-repoAssemblyResolutionSubscriptionpattern (subscribe/unsubscribe around resolution, restricted to#if NET), which appears to have been introduced later to scope the handler's lifetime — but it's unclear if it fully addresses recursive self-triggering during resolution failure.Next Steps
Default_ResolvinginSqlConfigurableRetryLogicLoader.cscan recursively re-enter itself whenMakeFullPath/LoadFromAssemblyPathtriggers additionalResolvingevents (e.g., when loading a dependent assembly that itself can't be resolved, or Windows/Linux path/probing differences causing repeated re-invocation), and confirm the existingAssemblyResolutionSubscriptionscoping (subscribe only while a custom retry-logic type is configured, unsubscribe immediately after) fully prevents this recursion path, adding a guard/reentrancy check if not.LoadFromAssemblyPathto avoid nested resolution triggering the same handler) as a general hardening measure even without an exact repro.Bump the nuget-minor-and-patch group with 4 updates (#65)
Bump Microsoft.Data.SqlClient from 6.1.6 to 7.1.0 (#66)
Bump the nuget-ecosystem group with 18 updates (#48)