OSX failure on StringTests.StartsWithNoMatch_StringComparison

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=940013
Build error leg or test failing: System.Tests.StringTests.StartsWithNoMatch_StringComparison
Pull request: #111877

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Log

[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc] Failed tests:
[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc] 1) 	[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison   Test name: System.Tests.StringTests.StartsWithNoMatch_StringComparison
[17:27:34] dbug: Assembly:  [System.Runtime.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc]    Exception messages: Assert.Equal() Failure: Values differ
[17:27:34] dbug: Expected: True
[17:27:34] dbug: Actual:   False   Exception stack traces:    at System.Tests.StringTests.StartsWithNoMatch_StringComparison()

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=940013
Error message validated: [[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 2/5/2025 4:19:32 PM UTC

Report

Build	Definition	Test	Pull Request
1316114	dotnet/runtime	System.Runtime.Tests.WorkItemExecution	#125056

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	1

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

@matouskozak

@matouskozak please triage. Thanks!

@kotlarmilos

Looks like duplicate of #108832 but with slightly different error message. fyi: @kotlarmilos

@vitek-karas

Tagging subscribers to 'os-tvos': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

@vitek-karas

Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Note

This analysis was generated by GitHub Copilot.

Analysis: Root cause identifiable from code inspection

A dump or repro isn't strictly necessary here — this is a wrong-return-value bug, not a crash, so a dump wouldn't add much. The root cause is identifiable from the code.

What the test does

The test (StartsWithNoMatch_StringComparison) creates strings from chars \u0001–\u0096 (control characters + basic Latin), introduces a single-char mismatch at each position, then asserts that culture-aware StartsWith returns consistent results between calls:

Assert.Equal(
    firstSpan.ToString().StartsWith(secondSpan.ToString(), StringComparison.CurrentCulture),
    firstSpan.StartsWith(secondSpan, StringComparison.CurrentCulture));

The failure (Expected: True, Actual: False) means the same underlying comparison returns different results on different invocations for the same input.

Where it goes wrong

On Apple mobile in hybrid globalization mode, culture-aware StartsWith routes through GlobalizationNative_StartsWithNative in pal_collation.m:

NSString *prefixStrComposed = RemoveWeightlessCharacters(
    prefixString.precomposedStringWithCanonicalMapping);
NSString *sourceStrComposed = RemoveWeightlessCharacters(
    sourceString.precomposedStringWithCanonicalMapping);

int32_t result = (int32_t)[sourceStrComposed compare:prefixStrComposed
                                    options:options
                                    range:sourceRange
                                    locale:currentLocale];

The intermittent nature suggests Apple's precomposedStringWithCanonicalMapping (NFC normalization) or [NSString compare:options:range:locale:] is non-deterministic for strings containing low Unicode code points — particularly the C1 control character range (\u0080–\u009F).

Suggested next step

Rather than trying to obtain a dump, enhance the test with diagnostic output to capture the exact failing parameters:

bool expected = firstSpan.ToString().StartsWith(secondSpan.ToString(), comp);
bool actual = firstSpan.StartsWith(secondSpan, comp);
Assert.True(expected == actual,
    $"length={length}, mismatchIndex={mismatchIndex}, comp={comp}, " +
    $"chars=({(int)first[mismatchIndex]},{(int)second[mismatchIndex]})");

Re-enable on Apple mobile and wait for CI to capture the specific (length, mismatchIndex, StringComparison) combination. That would clarify whether the issue is in NFC normalization of specific control chars or non-deterministic NSString comparison, and whether a workaround in pal_collation.m is feasible vs. filing an Apple bug.

Note

This analysis was generated by GitHub Copilot.

Not specific to a particular Apple OS version

The Helix queue configs show this has reproduced across multiple macOS versions:

Timeframe	Helix Queue	macOS Version
Oct 2024 (first failure)	`OSX.1200.Amd64.Open`	macOS 12 Monterey
Mar 2026 (latest hit, build 1316114)	`OSX.15.Amd64.Open`	macOS 15 Sequoia

This spans 17+ months across macOS 12 through 15, so it's not a regression in a specific Apple release — it's a long-standing behavior in Apple's [NSString compare:options:range:locale:] or precomposedStringWithCanonicalMapping for low Unicode code points (control characters).

@kotlarmilos

@kotlarmilos is this proposed tracing worth adding to the test to find out more?

Yes this is sound, let's give it a try

Fix intermittent OSX CI failures in StartsWithNoMatch_StringCompariso…

Navigation Menu

Repository navigation

OSX failure on StringTests.StartsWithNoMatch_StringComparison #112195

Build Information

Error Message

Known issue validation

Report

Summary

2 remaining items

Analysis: Root cause identifiable from code inspection

What the test does

Where it goes wrong

Suggested next step

Not specific to a particular Apple OS version

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

OSX failure on StringTests.StartsWithNoMatch_StringComparison #112195

Description

Build Information

Error Message

Known issue validation

Report

Summary

Activity

dotnet-policy-service commented on Feb 5, 2025

dotnet-policy-service commented on Feb 5, 2025

tarekgh commented on Feb 5, 2025

matouskozak commented on Feb 6, 2025

2 remaining items

dotnet-policy-service commented on Feb 6, 2025

dotnet-policy-service commented on Feb 6, 2025

danmoseley commented on Mar 22, 2026

Analysis: Root cause identifiable from code inspection

What the test does

Where it goes wrong

Suggested next step

danmoseley commented on Mar 22, 2026

Not specific to a particular Apple OS version

danmoseley commented on Mar 22, 2026

kotlarmilos commented on Mar 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions