You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Send keys to windows whose focus isn't a text field #13
Found in the Ace desktop tools study (githubnext/ace#427), scenario D2 on Canary 0.0.46+dd15840.
Observed:desktop_key Escape to a Finder window, with a file row selected, was refused twice: "The snapshot has no exact focused control. Click a control, then inspect the window again." The agent clicked a row and tried again, and it was refused again.
Cause:native/sources/client/actions.swift refuses key and insert when the snapshot has no focusedElement. Peekaboo's background input driver only resolves a focused text element as the receiver (focusedTextElement in BackgroundInputDriver.swift). So keys only work while a text field has focus. Escape, arrow keys, Return, Space (Quick Look) and Cmd shortcuts in lists, outlines, canvases and the Finder can't be sent. The hint suggests a click that can't help.
Options:
Admit a non-text focused receiver (a list, outline, table or web area) with the same exact process, window and focus checks, using background delivery when the app accepts it.
Or expose foreground key delivery for an exact window that has been activated, with the outcome left unknown when focus can't be read back. This is the --foreground gap from [Meta] Complete native computer use in Ace #11.
Short term, the refusal should say that keys need a focused text field, and point to desktop_menu when the command is in a menu.
Activity
fix: say why a key has no receiver and point to the menu (#14)
feat: send keys to windows whose focus isn't an observed control
feat: send keys to windows whose focus isn't an observed control
feat: send keys to windows whose focus isn't an observed control (#51)
feat: system-wide shortcuts with desktop_system_key (#54)