Skip to content

Add native targeted scrolling - #70

Merged
iamnbutler merged 6 commits into
feat/github-cachefrom
codex/desktop-pointer
Oct 5, 2026
Merged

iamnbutler merged 6 commits into
feat/github-cachefrom
codex/desktop-pointer

Conversation

@iamnbutler

@iamnbutler iamnbutler commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Adds desktop_scroll for an observed element or normalized screenshot point, with up/down/left/right direction and 1–20 native scroll units. It reuses exact snapshot/window authority, background delivery, and pi's unsafe-tool outcome handling. The request retains the snapshot's original focus evidence so a successfully delivered scroll's signed receipt verifies correctly.

Part of githubnext/desktop-tools#2 and #8. This focused diff preserves the landed click, app-search, and window-management tools. Drag work remains on codex/desktop-drag for a separate PR.

Validation:

  • The signed d9b0c869 candidate passed five real AppKit scrolls: element down, point down revealing initially offscreen content, element up, point right, and element left.
  • Real pi/Anthropic then performed inspect → element up1 → inspect. The receiver moved from 2063 to 1834; horizontal position and unrelated pointer state were unchanged. Exact tool text/images and usage survived reopening the same pi store without repeated input.
  • Final extracted head f1682d028a9f810c93c103286949c1ad69c8ef13 passed type checks, formatting/lint, Swift parsing, diff checks, and independent source review. The scroll implementation is unchanged from the signed candidate; no new signed build was made for the extraction/base integration.

Hosted workspace execution, worker-crash replay, self-Ace scrolling, multi-display/Retina/second-machine behavior, and the later invalid/moved-window checks remain unverified. Pixels-only observations do not provide reusable pointer authority. githubnext/desktop-tools#2 stays open for those checks and the remaining drag scope.

Built in Ace

Git/GitHub operations, checks, signed candidate/fixture builds, and committed-range review ran in a dedicated Ace terminal. The extraction was also reviewed through Ace's changes/patch APIs. Native and real-provider checks used Ace's signed bridge, actual tool schema/host adapter, and real receiver state.

Codex's external file/command tools handled source edits, log/receipt inspection, temporary helper preparation, and review coordination because Ace lacks external harness integration (#9) and a file-editing surface (#22).

@iamnbutler
iamnbutler force-pushed the codex/desktop-pointer branch from cbd895e to 0029579 Compare October 5, 2026 12:08
@iamnbutler
iamnbutler changed the base branch from codex/desktop-keyboard to feat/github-cache October 5, 2026 12:08
@iamnbutler iamnbutler changed the title Add native pointer clicks, scrolling, and atomic drags Add native scrolling and atomic drags Oct 5, 2026
@iamnbutler iamnbutler changed the title Add native scrolling and atomic drags Add native targeted scrolling Oct 5, 2026
@iamnbutler
iamnbutler marked this pull request as ready for review October 5, 2026 12:51
@iamnbutler
iamnbutler merged commit 4431242 into feat/github-cache Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant