Repository navigation
Inspect native pixels and report target availability - #71
Merged
Merged
Conversation
iamnbutler
force-pushed
the
codex/desktop-availability
branch
from
October 5, 2026 12:07
3f47a50 to
a99b284
Compare
iamnbutler
changed the base branch from
codex/desktop-keyboard
to
feat/github-cache
October 5, 2026 12:07
iamnbutler
marked this pull request as ready for review
October 5, 2026 12:11
Contributor
Author
|
The real pi/Anthropic inspection smoke passed: pixels, Accessibility, and absent-window diagnostics. Exact tool text, images, and error flags survived reopening pi's store with unchanged model usage and no repeated native calls. Evidence: |
This was referenced Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Failed native window inspections retain their original error and add a bounded, later read of target availability. Explicit
desktop_inspectpixels mode returns a screenshot with exact target and image-digest checks, without a reusable action snapshot. Failed reinspection also preserves an already completed action's outcome.Relates to #64 and #8. These diagnostics do not establish or fix the cause of intermittent offscreen or incomplete Accessibility observations.
This focused PR is based on merged application-state and selection/shortcut changes (
9418c8e); it no longer depends on #67. Availability diagnostics retain the newis_active_knownflag.Validation: type checks, formatting/lint, the independent signed desktop build, deep/strict signature verification, diff checks, and independent source review pass. Fresh native calls against that signed candidate passed exact app/window generation, Accessibility and pixel inspection, absent-window original-error diagnostics, unknown-state flags, and invalid-mode rejection. Pixel results have a screenshot but no action snapshot, element IDs, or detection metadata. Ace's initial webview AX tree was sparse; this does not claim that separate issue is fixed.
A real AppKit Close Alpha button ran exactly once and closed its target while another fixture window remained open. The returned action retained its original completed outcome/native receipt and separately reported the failed follow-up inspection. Callback evidence confirmed the actual close; no input was replayed. The initial verifier incorrectly expected the closed window to disappear from inventory; macOS retained an offscreen pixels-only row. Rechecking the saved result against the correct contract passed without repeating the click. Existing pi result/history transport is unchanged; broader visual workflows remain tracked in #8.
A real pi/Anthropic run using this PR's desktop tool schema and host adapter passed exactly three read-only inspections: pixels, Accessibility, and an absent window. The stored tool results retained the native text, screenshots, and error flags exactly. Reopening the same pi SQLite store returned identical results and images with unchanged model usage and no repeated native calls. Evidence:
/tmp/ace-availability-model-odVTLq/result.json.Built in Ace
Ace's terminal ran installation, builds, checks, Git/GitHub operations, and prior native validation. Codex coordinated and read/edited source while Ace's external-harness support (#9) and file editor (#22) remain unfinished. Native checks ran through Ace’s terminal against the isolated signed app. External CUA was used only to quit the prior validation build and launch the disposable AppKit fixture while Ace’s application-lifecycle support remains incomplete under #8.