Skip to content

Copy the stored broker metadata before building the usage report - #5972

Open
johnsimons wants to merge 1 commit into
masterfrom
john/report_broker_data_copy
Open

johnsimons wants to merge 1 commit into
masterfrom
john/report_broker_data_copy

Conversation

@johnsimons

@johnsimons johnsimons commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Should_count_the_choices_made_in_servicepulse_without_revealing_them fails intermittently on Linux-SqlServer (runs 37422880064, 37610293714 and 37859193494). Each failure reported values that belong to the other test in When_reporting_the_environment. Two runs had the heartbeat keys wrong (Heartbeats.TrackInstancesDefault Enabled, Heartbeats.TrackInstancesOverrides 0, Heartbeats.MonitoredInstances 0). The third had Recoverability.Redirects and Licensing.ReportMasks at 0 instead.

Cause

ThroughputCollector.GenerateThroughputReport uses brokerMetaData.Data from the licensing store as the report's EnvironmentData and writes every report key into it. When no broker metadata is stored, as with the Learning transport, the RavenDB and EF Core stores return a shared static default, so every report in the process writes into the same dictionary. The acceptance tests run in parallel in one process. When both licensing tests build a report at the same time, one report can be signed with values the other wrote, and two concurrent writes can corrupt the dictionary.

HeartbeatEndpointSettingsSyncHostedService was the first suspect because it upserts endpoint settings. It never ran during the failing tests: they finish in 2 to 3 seconds and the service waits 20 seconds before its first sync. It also cannot change redirects or report masks.

Change

The report copies brokerMetaData.Data into a new dictionary before adding its own keys. New test Should_leave_the_stored_broker_metadata_unchanged.

#5965 carries the same fix under "Also fixed". This PR lets it reach master on its own.

Testing

  • Reproduced on SQL Server by temporarily adding a 5 second pause before the report is signed and a 4 second pause before the other test downloads its report. The test failed with all five values from the other test. With the fix and the same pauses, it passed.
  • The new unit test fails without the fix.
  • Particular.LicensingComponent.UnitTests: 90/90.
  • When_reporting_the_environment: 10 runs on SQL Server and 10 on PostgreSQL, all green.
  • ServiceControl.AcceptanceTests filtered to Licensing: 6/6 on RavenDB, SQL Server and PostgreSQL.

…ctionary

EnvironmentData referenced the same dictionary instance returned by the
broker metadata store, so later mutations of the report data could
leak back into persisted state. Assign a new dictionary when building
the report and add a test asserting stored broker metadata stays
unchanged after generating a report.
@johnsimons johnsimons self-assigned this Oct 9, 2026
@johnsimons johnsimons changed the title Copy broker metadata into the report instead of sharing the stored dictionary Copy the stored broker metadata before building the usage report Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants