Run the live checks deferred while the screen was locked #42

One list of the live checks that must pass on an unlocked Mac before this issue closes. Run them with the dev app (npm run dev-app) or the installed release; docs/live-checks.md has the setup. Tick an item when it passes. When one fails, open an issue and link it next to the item.

Each item names the PR it came from. Everything the harness already runs is the first item; the rest needs a person, a second display or Space, or an app the harness can't set up.

1. Automated

• npm run live-check reports PASS for every check (--list shows them). It covers feat: wait until a window, dialog, element, text or app condition holds #70, feat: hold modifier keys during clicks, and press key sequences against one snapshot #77, feat: set date and time pickers from an ISO 8601 date #79, feat: save a screen or window capture as a full-resolution PNG, optionally one region #81, feat: type into web text fields with key events so page scripts see the input #82, feat: report focus, appearance, session and input state with the app inventory #83, feat: read a table, outline or grid as columns and rows with each row's element ID #86, feat: select text in web pages through WebKit text markers #88, feat: expect a condition after input whose outcome is unknown #89, feat: list keyboard input sources and select one by ID #90, feat: write one table cell by row and column, confirmed by reading the table again #92, feat: save the clipboard image's original bytes with desktop_clipboard_read save_path #96, feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99, feat: turn on Chromium and Electron web accessibility before inspect and read, once per app launch #107, feat: turn on web accessibility for desktop_wait's element and text polls too #110, feat: retry an incomplete Accessibility read once in inspect and read, and say retried when it fails again #111, feat: paste files into a Finder folder with desktop_insert, confirmed by their names appearing there (#3) #114 and feat: ask once per agent before it controls the Mac, and keep agent input off Desktop Tools' own UI (#24) #120.

2. Window outline (#43)

• Inspect, then click: the target window gets the outline and label.
• Switching Spaces: the outline disappears on the other Space and comes back on return.
• A full-screen target: the outline sits at the screen edges, above the window.
• Dragging the target window by hand: the outline follows smoothly.
• Turning Highlight agent's window off removes the outline; turning it on restores it.
• Clicks on the outline's margin and label reach the window underneath.
• Hover, press and drag outline their target windows too.

3. Pointer: hover, press, drag, modifier clicks

• feat: hover and press-and-hold #45 desktop_hover on a Safari toolbar button: completed, pointer_moved: true, pointer at the element's center, tooltip in an inspect about 1 s later.
• feat: hover and press-and-hold #45 Hover on a point another window covers: refused ("Another window covers this point"), the pointer doesn't move.
• feat: hover and press-and-hold #45 Hover with a snapshot taken before the window moved: refused.
• feat: hover and press-and-hold #45 Overlay: the agent cursor glides to the point and the real pointer follows; the agent cursor never shows in desktop_inspect or desktop_screen images.
• feat: hover and press-and-hold #45 desktop_press duration_ms: 1500 on Safari's Back button (with history): completed, release: "released", the history menu in the next inspect. The overlay cursor stays pressed for 1.5 s and lifts with a ripple, with no extra delay.
• feat: hover and press-and-hold #45 desktop_press button: "right" on a text view: context menu; a later left click works normally.
• feat: hover and press-and-hold #45 Close or move the window during a 5 s press: unknown, release: "windowChanged", no stuck button.
• feat: hover and press-and-hold #45 / Finish native pointer delivery and validation #2 SIGKILL the client during a 5 s press, and during a 5 s exact-window drag: the button is up within about 7 s, the overlay lifts, and a later click works.
• feat: drag between windows and apps #52 Drag a Finder file into a TextEdit document (to_snapshot): unknown, pointer_moved: true, the file's name or contents inserted.
• feat: drag between windows and apps #52 Drag selected TextEdit text into a Notes or Safari text field: the text is copied there.
• feat: drag between windows and apps #52 Drag a Finder file between two Finder windows: the file moves.
• feat: drag between windows and apps #52 Drag to an end point another window covers, and drag with a snapshot taken before the window moved: both refused, the pointer doesn't move.
• feat: drag between windows and apps #52 An exact-window drag within one window behaves as before; the overlay draws the trail and the highlight follows; a normal click works after every drag.
• feat: drags with modifier keys and multi-point paths #64 Finder: Option-drag copies a file, Command-drag moves it. In a drawing app, Shift-drag constrains.
• feat: hold modifier keys during clicks, and press key sequences against one snapshot #77 Option-click a disclosure triangle by element: it uses the pointer route and expands all.

4. Element actions and values (#47)

• AXShowMenu on a Finder file shows its context menu in the next inspect; AXCancel dismisses it.
• Finder list view disclosure triangle: desktop_set_value true expands it, false collapses it.
• TextEdit Format › Font panel stepper: AXIncrement, AXDecrement and desktop_set_value all change it.
• Displays brightness slider: set it and restore it.
• A pop-up button: choose Breeze, read it back, restore Boop.
• AXConfirm, AXRaise and AXScrollToVisible work on elements that advertise them.
• desktop_type into a TextEdit or System Settings search field still completes.

5. Keys

• feat: send every printable key and the remaining named keys #46 Command-comma in TextEdit opens Settings.
• feat: send every printable key and the remaining named keys #46 Typing ,./;'[] one key at a time into TextEdit gives exactly those characters.
• feat: send every printable key and the remaining named keys #46 / feat: system-wide shortcuts with desktop_system_key #54 / with Shift types ? on a US layout; ? alone is refused with the "/ with Shift" hint.
• feat: send keys to windows whose focus isn't an observed control #51 Finder list view, row selected: down reports receiver: window_focus, completed, and selects the next row.
• feat: send keys to windows whose focus isn't an observed control #51 Finder: space opens Quick Look and escape closes it; Command-down opens the selected folder and Command-up goes back; return starts a rename and escape cancels it.
• feat: send keys to windows whose focus isn't an observed control #51 Finder, file selected: Command-C reports receiver: window_focus, and desktop_clipboard_read returns the file reference.
• feat: send keys to windows whose focus isn't an observed control #51 A Safari page with no focused field: down and space scroll the page.
• feat: send keys to windows whose focus isn't an observed control #51 A text field still reports receiver: observed_control, and a lands in it.
• feat: send keys to windows whose focus isn't an observed control #51 A key with a snapshot from before another window took focus: refused, nothing sent.
• feat: system-wide shortcuts with desktop_system_key #54 desktop_system_key: Command-Space opens Spotlight (unknown, escape closes it); Command-Tab changes the frontmost app; Control-Up opens Mission Control; Command-Shift-5 and Command-Shift-4 open the screenshot tools (escape dismisses); Control-Space switches input source and back. Each reports globalEvents/foreground delivery.
• feat: press media, volume and brightness keys as system-defined events #80 volume_up/volume_down change the volume and show the HUD; mute toggles it.
• feat: press media, volume and brightness keys as system-defined events #80 With Music playing: play_pause pauses and resumes; next_track and previous_track change the track.
• feat: press media, volume and brightness keys as system-defined events #80 brightness_up/brightness_down change the built-in display (note whether an external display responds); keyboard_brightness_up/down change a backlit keyboard.
• feat: press media, volume and brightness keys as system-defined events #80 Every delivered media key reports unknown.
• feat: expect a condition after input whose outcome is unknown #89 Finder Command-click on a second file with expect (element_appears for the row, or text_changes on the status bar): completed, or unknown with last_observation.
• feat: list keyboard input sources and select one by ID #90 With Japanese Romaji enabled: select its Hiragana mode, then desktop_insert (type route) into TextEdit gives the expected characters.
• feat: select the input source through the host #101 desktop_input_source select: completes, reads back, and Recent Activity lists "Select input source". With Pause All Input on, it is refused, listed as "paused", and current is unchanged.

6. Menus, launch, open, dialogs

• feat: report which windows a menu command, launch or open created #56 TextEdit File › New via desktop_menu: one window in created_windows, exclusive, and it inspects.
• feat: report which windows a menu command, launch or open created #56 Two File › New calls about 0.3 s apart from two clients: the later one reports ambiguous when both windows land in its interval.
• feat: report which windows a menu command, launch or open created #56 desktop_open of a .txt with TextEdit running: the new document window, exclusive.
• feat: report which windows a menu command, launch or open created #56 desktop_launch of an app already running with no new window: none, or no field when nothing was sent.
• feat: choose a path in an exact Open or Save panel #57 Open and Save panels: desktop_dialog_answer navigates to a path.
• fix: admit keyboard input to sheet fields of the observed window #9 desktop_insert into TextEdit's Save sheet filename field: completed; the parent window with the sheet attached is refused.
• Ace: agents can't use Ace's own menus and dialogs when Ace embeds the host ace#531 Ace Open Folder: Command-O, desktop_dialog, desktop_dialog_answer path to an owned folder, button: "Open" adds the project; Cancel leaves the projects unchanged.

7. Reading and tables

• feat: read window text, inspect one subtree, and recognize text in captures #55 Safari, a long Wikipedia article: desktop_read returns the whole text in order, below the fold too; truncated is false or names text_limit.
• feat: read window text, inspect one subtree, and recognize text in captures #55 Safari: desktop_read scoped to the AXWebArea has no toolbar or tab text.
• feat: read window text, inspect one subtree, and recognize text in captures #55 Safari: desktop_inspect scoped to the web area lists deep links with attributes.depth; clicking one by ID navigates.
• feat: read window text, inspect one subtree, and recognize text in captures #55 Reusing a scope after the page scrolled: refused ("not found unchanged"), no input.
• feat: read window text, inspect one subtree, and recognize text in captures #55 Mail's message list and a Finder folder with hundreds of files: desktop_read lists the rows, and a scoped inspect shows rows past the first 200 elements.
• feat: read window text, inspect one subtree, and recognize text in captures #55 System Settings › Privacy & Security: desktop_read text is complete.
• feat: read window text, inspect one subtree, and recognize text in captures #55 A canvas or Electron/Qt window: mode: "pixels", ocr: true returns lines, and clicking a line's box center hits that text.
• feat: read window text, inspect one subtree, and recognize text in captures #55 desktop_screen ocr: true: line bounds match the windows on screen.
• feat: read window text, inspect one subtree, and recognize text in captures #55 A long TextEdit document: the AXTextArea value appears once.
• feat: read window text, inspect one subtree, and recognize text in captures #55 A huge page: the read stays under 25 s and reports ax_truncated_deadline instead of failing.
• feat: read a table, outline or grid as columns and rows with each row's element ID #86 Finder table read: rows have depth and expanded after expanding a subfolder, and selected on the selected file; clicking a row's element selects it.
• feat: read a table, outline or grid as columns and rows with each row's element ID #86 A Finder folder of 300+ files: read with whole-window scope (many row IDs null), then with scope on the outline (rows have IDs); page with row_offset.
• feat: read a table, outline or grid as columns and rows with each row's element ID #86 Chrome: a <table> with <th> gives columns and drops the header row.
• feat: read a table, outline or grid as columns and rows with each row's element ID #86 Mail message list: columns and the selected row.
• feat: select text in web pages through WebKit text markers #88 Chrome: desktop_select a phrase; Command-C copies exactly it.
• feat: select text in web pages through WebKit text markers #88 An element longer than the 2048-step / 300 ms cap: refused, nothing selected.
• feat: write one table cell by row and column, confirmed by reading the table again #92 A checkbox column (for example a System Settings list) via desktop_set_value.
• feat: type into web text fields with key events so page scripts see the input #82 desktop_type into Ace's ProseMirror composer: completed with readback.
• feat: type insertions where no text control can be observed #68 desktop_insert into a contenteditable with no observed focused control: typed via window_focus, unknown, the text is there.
• feat: turn on Chromium and Electron web accessibility before inspect and read, once per app launch #107 A Slack, VS Code or Discord window: the first inspect reports web_accessibility: "enabled" (via manual) once per launch and lists its web fields; the window still moves and resizes normally; All Activity lists Turn on web accessibility once; it still runs while input is paused.

8. Waits (#70)

• window_appears with title_contains, started before desktop_open of a TextEdit file, matches the new window; window_gone matches after desktop_close.
• element_appears for TextEdit's close button returns a snapshot that desktop_click accepts with that element ID.
• text_contains with an element matcher, and text_changes reporting before.
• dialog_appears after Command-S on an untitled document, dialog_gone after Cancel.
• app_quit after desktop_quit of TextEdit.
• A scope search on a long Safari page for an element deeper than 200 elements.
• While a wait polls, another agent's action interleaves; lsof shows no ~/.peekaboo lock held between polls.

9. Windows, Spaces, displays

• fix: refuse actions on a minimized window with a restore hint #59 Minimize a TextEdit window, then desktop_insert, desktop_click and desktop_key with the old snapshot: refused, WINDOW_MINIMIZED, with a cause. desktop_restore, inspect, insert: completed.
• fix: refuse actions on a minimized window with a restore hint #59 A refusal on a window that isn't minimized (an element ID not in the snapshot) keeps its own message.
• Explain native inspection failure when the desktop or target window becomes unavailable #1 A window on another Space: availability.reason: other_space with its space_id; desktop_switch_space to it, then inspect: available.
• Explain native inspection failure when the desktop or target window becomes unavailable #1 Minimized and hidden windows are classified; desktop_restore / desktop_unhide recover them.
• Explain native inspection failure when the desktop or target window becomes unavailable #1 A window moved off every display: off_screen; desktop_move back recovers it.
• feat: list Spaces and switch a display's Space #66 desktop_switch_space really switches; Control-arrow works without fn; it refuses with two displays; a window moved to another Space by hand in Mission Control is found there.
• feat: enter or exit native full screen for one window #78 desktop_fullscreen enters full screen (completed, space_kind: fullscreen), reports no change when already there, and exits back to a desktop Space.
• Finish native pointer delivery and validation #2 Two displays, one left of or above the main one: on the secondary display, desktop_click, desktop_scroll and an exact-window desktop_drag each land at the intended point. Repeat on a second Mac.
• Finish native pointer delivery and validation #2 Rearrange or unplug a display between inspect and click: refused, nothing sent.

10. Clipboard and insert

• fix: say that a failed keyboard preparation sent no keys #62 About 70 inserts while a second agent types: every insert lands, and the clipboard is never restored over newer contents.
• Add temporary clipboard paste and lifecycle validation #3 SIGKILL the MCP client during an insert into TextEdit: the clipboard holds either the original contents or the inserted text with the reservation held, never older contents restored over a newer copy.
• Add temporary clipboard paste and lifecycle validation #3 Quit Desktop Tools right after an uncertain insert: after relaunch, desktop_clipboard_write is still refused until the receiving app quits, and nothing earlier is restored.
• feat: save the clipboard image's original bytes with desktop_clipboard_read save_path #96 Copy a JPEG and save it with .jpg: the bytes equal the source.
• feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99 Two files into TextEdit rich text (format: "files"): two attachments, completed only if both read back as U+FFFC.
• feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99 Paste an image over a selected word: the word is replaced, completed.
• feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99 A plain-text TextEdit document: unknown, clipboard_ownership: "reserved", and desktop_clipboard_write refused until TextEdit quits.
• feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99 clipboard: "replace" into Slack or a browser field: the image arrives, unknown, clipboard_cleanup: "left_replaced", clipboard_ownership: "released", and the next desktop_clipboard_write is admitted.
• feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99 A 10 MiB TIFF pastes; an action input over 64 KiB is accepted.
• feat: paste an image or files into a native text area with desktop_insert, or leave the clipboard replaced on request #99 Clipboard read access set to Ask: the default refuses before input, and replace still pastes.

11. Notifications (#74)

• desktop_notifications with one banner: one entry with app, title, body, fields_from, and actions including open and Close.
• close: completed, notification_gone: true, and the banner is gone.
• open on a fresh banner: the posting app comes forward; completed (or unknown if the banner stays).
• A named app action (Reminders "Mark as Completed" or Calendar "Snooze"): the result matches the screen.
• Reusing a consumed ID: refused ("Notification gone; read again"), nothing sent.
• Notification Center open with a group stack: open expands it, and a re-read lists its members.
• UserNotificationCenter alerts appear only under system_alerts, and nothing is pressed.
• A non–Notification Center window with custom actions (Mail's message list) has no attributes.customActions.

12. Menu bar and Dock (#41)

• desktop_dock launches or activates TextEdit by its exact title.
• desktop_dock right-click with menu_item (Finder › New Finder Window) works; it moves the physical pointer.
• desktop_menubar_click on a third-party status item (1Password) opens its menu; note whether the menu window shows in desktop_windows.

13. The Desktop Tools menu

• feat: list recent agent activity in the app's menu #98 After a click, three keys, an inspect, a clipboard read and a refused action from an agent with DESKTOP_TOOLS_AGENT set: Recent Activity lists the last 5, newest first, with agent, pid, time, outcome icon, app and window title; the keys fold into "Key ×3"; All Activity (n) lists up to 20; typed text and clipboard contents appear nowhere; the list is empty after a restart.
• feat: pause all input from the app menu #100 Pause All Input: the icon becomes a pause sign and the status reads "(input paused)"; desktop_click is refused with the pause message and listed as "paused"; desktop_clipboard_write is refused and the clipboard is unchanged; desktop_windows, desktop_inspect and capture still work; after Resume the same click completes; a relaunch starts resumed.
• feat: ask once per agent before it controls the Mac, and keep agent input off Desktop Tools' own UI (#24) #120 Approved Agents: a new agent's first input raises the prompt; Allow lets it through; Revoke refuses it again; Allow and Forget work from the submenu; Recent Activity names Claude Code claude-code.

14. Other

• Ship a signed Desktop Tools app for agents outside Ace #27 Codex: with the [mcp_servers.desktop] entry from docs/mcp.md, desktop_apps, desktop_inspect on TextEdit and one desktop_type complete.
• fix: name the processes holding a timed-out desktop lane lock #58 On a quiet machine only (it blocks every other agent): hold ~/.peekaboo/global.lock from a throwaway process for 30 s; desktop_windows is refused after 15 s, naming that process.