You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Cancelled Action run leaves the coordination lock held until TTL expiry, blocking re-runs #1835
A cancelled Action run leaves the per-repo coordination lock (coordination/<owner>/<repo>/locks/action.json) held until its 15-minute TTL runs out. Every attempt to re-run during that window skips, because the action sees "Another Fro Bot Action run is active for this repository", even though no run is active.
What happened
Observed on fro-bot/agent@83ed683 (v0.119.2), in a scheduled run on marcusrbrown/infra:
Attempt 1 acquired the lock as action:<runId>:1 and renewed the lease every 30 s. I cancelled it about two minutes later to switch models. The runner killed the main step before runCleanup's finally block reached releaseLock (src/harness/phases/cleanup.ts). The last lease renewal was the last write to the lock.
The post: step then ran its not-persisted retry path. It saved the cache and synced sessions/opencode.db to the object store. The post hook has no handle on the lock (src/harness/post.ts), so the lock stayed held.
Attempts 2 and 3 of the same run got a new holder ID each (action:<runId>:2, action:<runId>:3). Both read the lock with the etag from attempt 1's last renewal, logged lock-held-by-other-surface … heldBy: action:<runId>:1, reason: coordination lease held, and skipped without running the agent. Both attempts failed in the next step, which depends on the agent run.
The run became usable only after the lease expired. At that point reclaimableWithoutConfirmation (src/harness/phases/acquire-lock.ts) allows taking over the expired Action-surface lease.
Why the lock should be releasable
When attempt N of a run starts, every earlier attempt of that run has finished. GitHub starts a re-run only after the previous attempt completes, post steps included. The runner also kills the job's leftover processes when the job ends: attempt 1's log shows Terminate orphan process: pid (…) (opencode). A lock held by action:<runId>:<M> with M < N can no longer have a live writer.
Proposed fixes
Either fix alone would have prevented this:
Release in the post hook. The main step records its holder ID with core.saveState after acquiring the lock, and clears that record after a successful release. When the post hook finds the record still set, it deletes the lock with a conditional delete that only succeeds while holder_id still matches. This covers a cancellation or crash between acquiring and releasing.
Same-run takeover. In acquire-lock, a lease held by an earlier attempt of the same run (parseActionHolderRunId(holder) === runId and holder attempt < current attempt) can be taken over without waiting for expiry. The existing etag precondition keeps the takeover safe.
The second fix also covers runners that never get to run the post step, for example when the runner is lost.
Cancellation can leave the Action coordination lease behind, blocking immediate reruns for up to 15 minutes after the last successful renewal. The reported behavior is consistent with current main.
Evidence
I reproduced the lock behavior locally using the real runtime acquisition/renewal functions and an in-memory conditional-write store:
attempt 1: acquired and renewed; release omitted to simulate interrupted cleanup
attempt 2: active-holder, heldBy=action:1234:1
attempt 3: active-holder, heldBy=action:1234:1
after expiry: acquired
This confirms the lock-state failure; I did not perform a live Actions cancellation or inspect the original run logs. The existing runtime lock, Action acquisition, and post-hook suites also pass: 114 tests across 3 files, indicating missing regression coverage.
Root cause
cleanup.ts:445–477 releases in finally, using the latest renewal ETag. That handles ordinary exceptions, but cannot guarantee release when the process is forcibly terminated.
post.ts retries persistence but has no lock-release path or saved lock-ownership state.
lock.ts:186–208 rejects every unexpired holder before evaluating reclaimableWithoutConfirmation. The Action holder includes the attempt number, but no earlier-attempt exception exists.
Important implementation detail: broadening the existing reclaim predicate alone will not fix immediate reruns—it is only consulted after expiry.
Recommended next steps
Prioritize same-run, earlier-attempt takeover. Add an explicit pre-expiry recovery path restricted to GitHub-surface Action holders with the same run ID and a strictly lower, valid attempt number. Replace using the observed ETag; a concurrent renewal/replacement must yield a conflict. Preserve exclusion for other runs, equal/newer attempts, and malformed holders.
Add post-hook release as complementary recovery. Save ownership immediately after acquisition. After any allowed post persistence finishes, read the current Action lock, verify the saved holder still owns it, and conditionally delete using that read’s ETag. Clear saved ownership after successful main-step release. Release must also run when persistence is skipped or throws; it must never delete a successor’s lock. Saving only the initial ETag is insufficient because renewal changes it.
Cover both recovery paths: unexpired earlier-attempt takeover; rejection of unrelated/equal/newer holders; ETag conflicts; cancellation-state post release; changed-holder protection; and post persistence failure/skip branches. Then validate with one controlled cancel-and-rerun workflow.
Until fixed, wait for the lease to expire before rerunning. The lock protects the shared S3 session database, so weakening repository-wide exclusion is not an appropriate workaround.
Activity
fro-bot commentedon Oct 15, 2026
Triage: confirmed lock-recovery bug
Cancellation can leave the Action coordination lease behind, blocking immediate reruns for up to 15 minutes after the last successful renewal. The reported behavior is consistent with current
main.Evidence
I reproduced the lock behavior locally using the real runtime acquisition/renewal functions and an in-memory conditional-write store:
This confirms the lock-state failure; I did not perform a live Actions cancellation or inspect the original run logs. The existing runtime lock, Action acquisition, and post-hook suites also pass: 114 tests across 3 files, indicating missing regression coverage.
Root cause
cleanup.ts:445–477releases infinally, using the latest renewal ETag. That handles ordinary exceptions, but cannot guarantee release when the process is forcibly terminated.post.tsretries persistence but has no lock-release path or saved lock-ownership state.lock.ts:186–208rejects every unexpired holder before evaluatingreclaimableWithoutConfirmation. The Action holder includes the attempt number, but no earlier-attempt exception exists.Important implementation detail: broadening the existing reclaim predicate alone will not fix immediate reruns—it is only consulted after expiry.
Recommended next steps
Until fixed, wait for the lease to expire before rerunning. The lock protects the shared S3 session database, so weakening repository-wide exclusion is not an appropriate workaround.
Run Summary
fix(action): match post-hook release on a per-acquisition token and a…