AI Coach review fails with "unusable" when a proposed swap targets an exercise already in that routine (validator rule absent from the prompt)

[fama-lama]

Summary

POST /api/coach/review fails with errorClass: "unusable" (UI: "The Coach answered with something the app couldn't use.") whenever the model proposes a swap-exercise (or add-exercise) whose after.id is already present in the target routine.

The validator is right to reject it — but the review prompt never states that constraint, so the model has no way to know it. And because HTTP providers are pinned to temperature: 0 with exactly one repair round, the retry replays the same answer and the job is guaranteed to fail.

Model-independent: reproduced against a local OpenAI-compatible endpoint (llama.cpp / llama-swap behind an OpenAI-style API), but the same payload also fails attempt 1 on two other models of very different sizes.

Steps to reproduce

1. Configure Coach → OpenAI-compatible endpoint, pick any model.
2. Use a plan where one routine already contains both a flat barbell press and an incline dumbbell press (i.e. a pair a model would plausibly swap between, so a swap-exercise proposal duplicates the exercise that is already there).
3. Press review my routine.

Observed

The provider answers 200 with valid JSON:

{
  "coach_contract": 1,
  "summary": "...",
  "changes": [
    { "type": "swap-exercise",
      "target": { "routineId": "<push day routine id>", "exId": "<flat barbell press id>" },
      "after":  { "id": "<incline dumbbell press id>", "sets": 4, "reps": 8, "weight": 26 },
      "why": "barbell bench press has 4 consecutive stalls..." }
  ]
}

parse.js → fine. contractOK → fine. Then:

validateReview → changes[0].after.id "<id>" is already in routine "Push Day"

The repair round returns a byte-identical answer (md5 equal on both rounds), fails the same way, and the job is recorded outcome: failed, errorClass: unusable. The user sees a hard failure with no path forward, and every retry from the client fails identically.

Root cause

api/coach/prompts/review.md (→ generated core/prompts.js) lists every allowed change type and its target/after shape, but never states either of these rules that core/validate.js enforces:

• a swap-exercise / add-exercise after.id must not already exist in the target routine; and
• target.exId must belong to target.routineId.

The repair prompt mentions "a target naming a routine or exercise that is not in the plan" — that is existence in the plan, which is not the same rule.

The information needed to satisfy both rules is in the payload (plan.routines[].ex[].id and the library array), so this is purely a prompt gap, not a payload gap.

Suggested fix

Add to the review prompt's change-type section, and mirror it in the repair prompt's common causes:

A swap-exercise or add-exercise after.id must not already be present in the target routine — check plan.routines[].ex[].id before you propose it and pick a different library id. A change's target.exId must belong to target.routineId.

Verified: with exactly that sentence appended to the system message — nothing else changed, same payload, same model, same settings — the same request returns a usable proposal (attempt 1 then fails on a smaller pairing error that the single repair round fixes).

Secondary: the single repair round cannot recover from a deterministic rejection

HTTP providers send temperature: 0 and get one repair round. In this failure the retry produced a byte-identical answer, so the retry carries no new information. A non-zero temperature on the repair attempt (or an error-specific nudge) would let it actually re-decide instead of replaying.

Ruled out (so nobody re-tests these)

• Not the 1,000-character input cap (Make AI Coach message length configurable in Admin Advanced settings #267). Reproduced with no note at all.
• Not the payload. It carries each routine's full ex[] with ids.
• Not model size. On the same payload, three models all fail attempt 1: one proposes the duplicate id, a second makes the routine/exercise pairing error (plus ~9 changes where the prompt asks for about six), a third emits non-integer values for integer-valued change types. Only one of the three recovered in its repair round.
• Not client-fixable. Putting the constraint in the review note does not work: the text does reach payload.userNote, and the model still repeats the identical proposal. Per the common prompt, user free text is data rather than instruction, and there is no admin field for extra Coach instructions.

Environment

Reported from a self-hosted deployment; coach provider = OpenAI-compatible endpoint, response_format: json_schema, temperature: 0, max_tokens: 16000. Reproduced by invoking the app's own attemptOnce / runPipeline path in coach/core/pipeline.js against the live state, so it is the exact code path the review my routine button uses.

[DuarteSantos8]

Thanks @fama-lama, a very precise report. Fixed in v1.3.9:

• the review prompt now states the rule, so the model knows it can't swap in an exercise the routine already has;
• if it does anyway, that one change is left out and the rest of the proposal is kept, but only when no other change in that routine refers to the exercise it swapped out or brought in. Otherwise the answer goes back for repair instead of being applied with a different meaning.

Released in v1.3.9: https://github.com/DuarteSantos8/openGym/releases/tag/v1.3.9