RunAllTestsWithMixedTFMsWillProvideAdditionalInformationToTheDebugger is often failing

[nohwnd]

https://dev.azure.com/dnceng-public/public/_build/results?buildId=1086121&view=ms.vss-test-web.build-test-results-tab&runId=29562688&resultId=101138&paneView=debug

Expected collection to be empty, but found at least one item {"The active Tests Run was aborted. Microsoft.TestPlatform.VsTestConsole.TranslationLayer.TransationLayerException: Failed to receive message from vstest.console process
at Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleRequestSender.d__44.MoveNext() in //src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/VsTestConsoleRequestSender.cs:line 1413
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleRequestSender.SendMessageAndListenAndReportTestResults(String messageType, Object payload, ITestRunEventsHandler eventHandler, ITelemetryEventsHandler telemetryEventsHandler, ITestHostLauncher customHostLauncher) in //src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/VsTestConsoleRequestSender.cs:line 1148

But cannot repro locally

[nohwnd]

It is always row 19, and sometimes row 18, never saw the same locally, even when building and running as Release:

RunAllTestsWithMixedTFMsWillProvideAdditionalInformationToTheDebugger (Row: 19, Every runner, Runner = net8.0, TargetFramework = net8.0, InIsolation, vstest.console = 17.6.2 [MostDownloaded], Testhost = 17.15.0-ci [Latest], MSTest = 3.9.3 [LatestStable])

[nohwnd]

[TestMethod]
[TestCategory("Windows-Review")]
[TestCategory("Feature")]
// "Just row" used here because mstest does not cooperate with older versions of vstest.console correctly, so we test with just the latest version available..
[RunnerCompatibilityDataSource(AfterFeature = Features.MULTI_TFM, JustRow = 0)]
public void RunAllTestsWithMixedTFMsWillProvideAdditionalInformationToTheDebugger(RunnerInfo runnerInfo)

Suppressed to have a look later, without it we cannot merge.

[nohwnd]

Triage Analysis 🤖

Looking at the current state of this test:

1. Features.MULTI_TFM is now commented out in test/Microsoft.TestPlatform.TestUtilities/Features.cs — the feature gate no longer exists.

2. The test has been refactored — RunAllTestsWithMixedTFMsWillProvideAdditionalInformationToTheDebugger now uses [WrapperCompatibilityDataSource] instead of [RunnerCompatibilityDataSource(AfterFeature = Features.MULTI_TFM, JustRow = 0)]. The WrapperCompatibilityDataSource tests compatibility of the wrapper with different versions of vstest.console — it generates a different set of test rows than the original RunnerCompatibilityDataSource matrix.

3. No JustRow suppression exists anywhere in the test codebase (only in utility class definitions), meaning the test is running with the full data set from WrapperCompatibilityDataSource.

The original flakiness (row 19 of RunnerCompatibilityDataSource failing with TransationLayerException: Failed to receive message) appears to have been addressed as part of the test infrastructure refactoring. The test has been re-enabled without the JustRow = 0 workaround.

Could you confirm this issue can be closed as resolved?

🔍 Triaged by Issue Repro Triage & Auto-Fix 🔍

🔍 Triaged by Issue Repro Triage & Auto-Fix 🔍