Skip to content

Add optional WinApp UI Automation integration for MSTest and MTP #11588

Description

Summary

Add an optional MSTest/Microsoft.Testing.Platform integration layer over the published Microsoft.Windows.SDK.BuildTools.WinApp.UIAutomation package.

The integration should let Windows desktop UI tests combine MSTest-managed application/window lifecycle with semantic selectors, inspection and actions, input, waits/assertions, and screenshot-on-failure artifacts without moving those capabilities into MSTest core.

Background and Motivation

MSTest.Windows.UIAutomation now provides the narrow lifecycle seam that MSTest should own: start a full-trust desktop application, wait for a window, expose it as a UIA2 AutomationElement, and stop the process. Its package documentation explicitly leaves locators, automatic waits, screenshots, and a multi-window model to libraries layered on top (package scope, layering guidance).

WinAppCLI now publishes such a library as a separate package rather than requiring consumers to invoke the CLI:

  • the package is explicitly described as the reusable engine behind winapp ui (project/package boundary);
  • IUiAutomation exposes element search/inspection and UIA actions such as invoke, value, focus, scrolling, and capture (interface, actions);
  • target resolution supports process/title/PID/HWND scenarios (IUiTargetResolver);
  • UiTarget.FromWindowHandle is explicitly intended to bridge from a framework that already owns a window, including MSTest.Windows.UIAutomation (HWND bridge);
  • the services are registered behind public interfaces (DI registration).

This is a concrete opportunity to make Windows UI testing with MSTest/MTP more capable without duplicating WinAppCLI's selector/action/input implementation. It would also give MTP a good artifact story: MSTest result files registered through TestContext.AddResultFile are converted to MTP FileArtifactProperty attachments (adapter publication).

Proposed Feature

Provide a small, optional integration package and/or first-class sample that:

  1. References MSTest.Windows.UIAutomation and Microsoft.Windows.SDK.BuildTools.WinApp.UIAutomation; neither MSTest core nor the base MSTest.TestFramework package should take this dependency.
  2. Converts WindowTest.MainWindow.Current.NativeWindowHandle to a UiTarget through UiTarget.FromWindowHandle, preserving MSTest's lifecycle ownership while using WinApp's richer automation surface.
  3. Makes semantic selectors, inspection/properties, invoke/value/focus/scroll operations, keyboard/mouse/pointer input, and target resolution straightforward from an MSTest test class.
  4. Adds test-focused waiting/assertion helpers around the underlying APIs, with cancellation and useful timeout diagnostics.
  5. On test failure, optionally captures the target window, writes the image into the test's result/temp area, and calls TestContext.AddResultFile so both MTP and VSTest publish it as a test attachment.
  6. Documents safe parallelization: UI tests that inject input or change foreground focus must not run concurrently on the same interactive desktop unless they participate in a shared desktop coordinator.
  7. Includes an end-to-end sample and acceptance coverage under MTP, plus VSTest coverage if the integration is advertised as runner-neutral.

The exact public shape needs design, but it should be a thin adapter/composition layer. For example, it could expose a fixture/service that owns the WinApp UI Automation service provider and derives the UiTarget from the WindowTest HWND, rather than wrapping every upstream type.

Package/API Boundary

  • Keep the integration opt-in and Windows-only.
  • Prefer composing the published WinApp interfaces and models rather than copying implementation into TestFx.
  • Avoid broad new MSTest core APIs. If a helper package is added, its public surface should focus on MSTest lifecycle, diagnostics, and artifact publication.
  • Keep screenshot capture separate from continuous video recording. Native recording work is being evaluated independently and should not block this integration.

Compatibility and Tradeoffs

  • Target frameworks: the reviewed WinApp UI Automation package targets .NET 10 Windows TFMs, while MSTest.Windows.UIAutomation currently supports older Windows-targeted applications. The integration must be TFM-gated or wait for a compatible upstream target; it must not silently raise the minimum TFM of the base MSTest package.
  • Desktop ownership: the published WinApp package deliberately excludes CLI-level cross-process desktop coordination (package warning). Until a reusable coordinator exists, documentation should require serialization or isolated interactive desktops for focus/input-sensitive tests.
  • Capture behavior: background/occluded WinUI capture depends on the selected target framework and Windows Graphics Capture support (capture tradeoff). Tests should report unsupported capture clearly rather than producing an empty attachment.
  • Ownership: MSTest should continue to own process/test lifecycle and result publication; WinApp UI Automation should continue to own Windows UI automation primitives.

Non-goals

  • Replacing Microsoft.Testing.Extensions.PackagedApp or changing packaged-app activation.
  • Taking a dependency on or invoking the full winapp CLI.
  • Reusing WinAppCLI's Sandbox protocol, generic process runner, SDK/runtime installation, or dotnet run launch targets.
  • Moving selectors/input/capture implementations into MSTest core.
  • Implementing the separate native Windows recording POC.

Acceptance Criteria

  • A documented optional integration boundary is selected (package, supported sample, or equivalent) and does not add WinApp UI Automation to MSTest core dependencies.
  • An MSTest test can bridge its existing WindowTest HWND into a WinApp UiTarget and use semantic selectors plus representative inspection/action/input operations.
  • Waiting/assertion helpers honor TestContext.CancellationToken and produce actionable timeout diagnostics.
  • A failed test can produce a window screenshot that appears as a test attachment under MTP; unsupported capture fails explicitly.
  • MTP acceptance coverage validates lifecycle, action, failure capture, and artifact publication.
  • TFM support and desktop-parallelization constraints are documented and enforced where possible.
  • PackagedApp replacement, full CLI consumption, Sandbox/process/runtime provisioning, overlapping launch targets, and continuous recording remain out of scope.

Alternative Designs

  1. Document raw package composition only. This has the smallest maintenance cost, but every test suite must repeat DI setup, HWND conversion, waits, failure cleanup, and attachment publication.
  2. Expand MSTest.Windows.UIAutomation directly. This is convenient but would couple the base MSTest package to a larger, independently versioned automation engine and could raise its target-framework floor.
  3. Vendor the WinApp implementation. This avoids a package dependency but creates duplicated Windows interop, selector semantics, and fixes. Reusing the published package is the preferable seam.

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/mstestMSTest framework (not analyzers or assertions).area/mtpMicrosoft.Testing.Platform core library.area/winuiWinUI support.needs/triageNeeds triage by a maintainer.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions