Skip to content

Correct docs that disagreed with the code (docs accuracy pass) - #330

Merged
siebe41 merged 3 commits into
mainfrom
sprint/release/O03-docs-accuracy
Oct 23, 2026
Merged

siebe41 merged 3 commits into
mainfrom
sprint/release/O03-docs-accuracy

Conversation

@siebe41

@siebe41 siebe41 commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

What changed

A docs accuracy pass: I checked every command, flag, configuration key, endpoint, header, default, button label and cross-doc link in README.md and docs/*.md against the code. 17 statements disagreed with it (20 edits), across the README and nine docs. Each is fixed in the doc. No code changed.

Doc Was Now Evidence
docs/host.md (service table, Logs, Windows) log in the configuration directory in the local data directory (%LOCALAPPDATA%\semantic-router with the default directory) ServiceCommand.cs:387 passes LocalDataDirectory; ServicePlatforms.cs:434 logs there
docs/host.md (What init adds) size tiers ≤ 9B: 1 ≤ 9.5B: 1 ModelLineup.cs:383 <= 9.5 => 1
docs/providers.md (Model lineups, LM Studio row) ≤ 9B 1 ≤ 9.5B 1 same
docs/network.md (ONNX model download) copy models/{name} into the configuration directory into the local data directory RouterHost.cs:95 → LocalData.ModelsDirectory = LocalDataDirectory/models
docs/memory.md (Configuration) buttons Add Claude Code, Add folder, Add MCP graph Add Claude Code's memory, Add a notes folder, Add an MCP memory graph wwwroot/js/memory.js:135-137
README.md (Quick start) init asks before enabling each subscription CLI; --yes enables them asks about claude and codex; copilot is added disabled unless --enable-cli names it InitCommand.cs:294-300 (copilot Ask = false), :335-340; Program.cs:97 help text
README.md (Claude Code example) ANTHROPIC_AUTH_TOKEN: "the router never reads or forwards it" any value unless gateway keys are set; never forwarded GatewayGuard.cs:95-132 reads it as a gateway key
README.md (dashboard sample comment) loopback only unless a token is set unless RequireLocalClient is off and a token is set DashboardAccess.cs:217-224; RequireLocalClient defaults to true
README.md (Dashboard) Admin: the JSON editor and the Audit log Admin: Service, JSON editor, Audit log wwwroot/js/core.js:110
docs/routing-quality.md (API) route health open to the read-only role needs the admin role (it embeds every utterance) EvaluationApi.cs:171 DashboardWriteMetadata; DashboardAccess.cs admits write metadata for Admin only
docs/architecture.md (projects table) doctor (RouterDoctor) in Hosting RouterDoctor is in AspNetCore; Hosting has the command AspNetCore/Diagnostics/RouterDoctor.cs:32; Hosting/DoctorCommand.cs:41
docs/desktop.md (background service) the service manager restarts the router if it fails (all platforms) on Linux and macOS; the Windows sign-in entry does not ServicePlatforms.cs WindowsSignInService is a Run-key value; host.md's table says "not automatically"
docs/desktop.md (Where things live, uninstall, managed deployment) everything, logs and webview/ included, in the configuration directory on Windows with the default directory, logs, webview/ and the database are in %LOCALAPPDATA%\semantic-router Desktop/App.cs:549, RouterLog.cs:15, HostSettings.LocalDataDirectoryFor
docs/dashboard.md (Pages, Activity) the last 500 routed requests the last 200 wwwroot/js/shell.js:28,132 load activity?limit=200 on start and every refresh
docs/dashboard.md (Admin API roles) admin list missing POST api/reliability/reset, POST api/config/template-defaults/use/keep, GET api/routes/health, POST api/prices/lookup ("everything else is open to the read-only role") added all carry DashboardWriteMetadata (ReliabilityApi.cs:30, TemplateDefaultsApi.cs:46,57, EvaluationApi.cs:171, RouterDashboardEndpoints.cs:506); DashboardAccess.cs:353 refuses them to non-admins
docs/observability.md (Metrics) "The names are constants on SemanticRouterDiagnostics" except semantic_router.state.dropped_writes, from Storage.Sqlite SqliteRouterState.cs:60
docs/observability.md (Subscription value) "(see below)" for Look up prices "(see Looking prices up, above)" the paragraph is the one before

Review follow-up (0bb325d)

The review found the same stale claims surviving in other docs. Fixed:

Doc Was Now
docs/clients.md (Claude Code) ANTHROPIC_AUTH_TOKEN: "the router never reads or forwards it" any value from this machine; with gateway keys set, send yours (required from other machines); never forwarded
docs/architecture.md (Safety rules) never reads or forwards credentials of a client never forwards a client's; reads its key only to check it against the gateway keys; never reads or forwards a CLI's
docs/host.md (Run it in the background) restarts if it fails on Linux and macOS; the Windows sign-in entry does not
docs/architecture.md (Durable state) semantic-router.db next to the configuration in the local data directory (%LOCALAPPDATA%\semantic-router on Windows by default)
docs/desktop.md (Uninstall, Managed deployment) only the database and logs move models and the web view profile move too; uninstall names both directories
docs/TERMS.md (Using it at work), docs/host.md (init) asks before enabling each subscription CLI; --yes enables them asks about claude and codex; copilot is added disabled, even with --yes, unless --enable-cli names it
docs/routing-quality.md (API) reliability reset missing from the admin-only sentence added
README.md gateway-key comment as in clients.md; Service page "(the background service)" the clients.md wording; "(how the router runs, who can connect, the background service)"
docs/observability.md "records on the same meter" "under the same meter name (SemanticRouter)"

Why

These docs are what users configure the router from, and before 1.0 they should describe what the code does. Most of the errors were stale after a change elsewhere:

  • Windows' local data directory moved the logs, models, database and web view profile out of the roaming profile. host.md's "Where things live" already said so, but the service table, network.md and desktop.md did not.
  • New admin-only endpoints were not added to the role list.
  • The dashboard's Activity limit changed.
  • init's copilot consent differs from claude and codex.

How it was verified

  • Each finding was confirmed by reading the code before editing (the evidence column above). Findings I could not confirm were dropped.
  • Six parallel read-only passes covered the README and every docs file. For each, they listed the claim categories that were correct: CLI commands and flags, every options class's keys and defaults, the endpoint maps, headers, metric and span names, workflow and secret names, deploy/ scripts, and anchors.
  • A link and anchor check (GitHub heading slugs) over the ten changed files found 0 broken links. The checker flags a deliberately broken anchor, so the result is meaningful.
  • No .NET file changed, so dotnet build, dotnet test, dotnet format and dotnet pack are unaffected and were not run. The Pages workflow builds the site with VitePress's dead-link check on this PR.
  • Docs-only changes need no CHANGELOG line (CONTRIBUTING.md#changelog).
  • After the review follow-up: npm ci + npm run build in site/ (VitePress, with its dead-link check) completes; no test asserts the changed wording (searched tests/).

Not done

  • The same claims in code (the service-restart text in service install, the desktop app and the dashboard; the Connect clients gateway-key comment; init --help's consent line; the ServicePlatforms.cs:406 log comment) are filed as Service restart, gateway key and init consent claims the docs corrected are still in the tool, app and dashboard text #339, since this PR is docs-only.
  • Not checkable offline: external dates (the GitHub Models retirement, Gemini CLI) and the egress hosts' live reachability (tools/check-egress-hosts.py makes network calls).
  • docs/host.md's release-to-schema table could not be tied to tags (CHANGELOG has no 0.5.0 section). It is consistent with SchemaVersion = 4.

Sprint task O03 (overflow); no issue tracks it.

Twenty-one corrections across the README and nine docs, each checked
against the code: where Windows keeps logs, models and the web view
profile, which dashboard endpoints need the admin role, init's
consent for the copilot CLI, the size cut-off for tier 1, button
labels and the Activity page's limit.

Co-Authored-By: Claude <noreply@anthropic.com>
@siebe41 siebe41 added the usage-sprint Opened by the Oct 2026 usage sprint label Oct 21, 2026

@siebe41 siebe41 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs changes, small

All 20 edits in this PR match the code on c67c91b, and the PR changes no code (all 10 files are .md). The problem is that three of the corrected statements survive, still wrong, in other docs the pass was meant to cover. The worst is docs/clients.md, the page the README sends readers to for client setup.

Findings

Medium: docs/clients.md:23 still says the gateway key is never read.

  • The line is export ANTHROPIC_AUTH_TOKEN=semantic-router # any value: the router never reads or forwards it. This is the statement the PR fixed at README.md:299.
  • GatewayAuthentication.Admit (GatewayGuard.cs:95-132) reads that token as a gateway key.
  • docs/architecture.md:194 says "the router never reads or forwards credentials of a client".
  • The dashboard's Connect clients snippet says the same thing in code (wwwroot/js/config.js:447, "the router ignores it and never forwards it").
  • Fix: correct clients.md the same way as the README. Note the config.js text as a follow-up, since this PR is docs-only.

Medium: docs/host.md:55 still says the background service "restarts if it fails", for every platform.

  • The table at host.md:73 says Windows does "not automatically", and the PR fixed the same claim in desktop.md:125.
  • The tool and the app tell Windows users the same thing:
    • ServiceCommand.cs:97, printed by service install on every platform;
    • WelcomeWindow.cs:39 and SettingsWindow.cs:59;
    • the dashboard's wwwroot/js/service.js:229;
    • the doc comment at Desktop/RouterService.cs:7.
  • Fix: correct host.md:55 here, and file a follow-up for the code strings.

Low: docs/architecture.md:115 still says "semantic-router.db next to the configuration". On Windows with the default directory, the database is in %LOCALAPPDATA%\semantic-router (LocalData.StateFile, RouterHost.cs:97). This is the same stale claim the PR fixed in desktop.md and network.md.

Low: docs/desktop.md:74 and :174 say only "the database and logs" move to the local data directory.

  • LocalData.Folders also moves models/ and webview/, as host.md:125 lists.
  • In the uninstall paragraph (:74) this matters. On Windows, two folders stay behind after uninstall, but the sentence reads as if the configuration directory is the only one to delete.
  • Suggested wording: "…the configuration directory and, on Windows with the default directory, the local data directory %LOCALAPPDATA%\semantic-router (state database, models, logs, web view profile) stay; delete them yourself…".

Low: the init consent wording the PR fixed in the README survives in two places.

  • docs/TERMS.md:139 says "init asks before enabling each subscription CLI it finds".
  • docs/host.md:45 says "asking before it enables a subscription CLI … --yes enables them without asking".
  • Copilot is never asked about, and --yes does not enable it (InitCommand.cs:294-300, :335-340).
  • host.md:45 links to the section that explains this, so it is borderline. TERMS.md, though, is the page people read for work use, so it should be exact.
  • The tool's own help text has the same imprecision (Program.cs:70); its --yes line (:97) is already right.

Low: docs/routing-quality.md:258 leaves POST api/reliability/reset out of the admin-only sentence. The endpoint is in the table two lines above (:256) and carries DashboardWriteMetadata (ReliabilityApi.cs:30). The PR added it to dashboard.md's admin list, but not here.

nit: README.md:299 is still slightly imprecise. With gateway keys set, a client on this machine is still admitted with any value unless Middleware:RequireGatewayKeyForLocalClients is on (off by default; GatewayGuard.cs:98). The key is only required from other machines. The new wording is safe advice, just stricter than the code. A more exact version: "any value from this machine; with gateway keys, send one (required from other machines)".

nit: README.md:321 describes the Service page as "(the background service)". The page also covers how this router runs (Restart, Stop) and who can connect (dashboard.md:50). "Service (how the router runs, who can connect, the background service)", or just "Service", would be closer.

nit: docs/observability.md:124 says "records on the same meter". SqliteRouterState creates its own new Meter(SemanticRouterDiagnostics.MeterName) (SqliteRouterState.cs:58). That is the same meter name (SemanticRouter), not the same instance. AddMeter("SemanticRouter") picks it up either way, so "under the same meter name" would be exact.

Checked

Every edit was checked against the code on c67c91b, and all are right:

  • Windows service logs go to the local data directory (ServiceCommand.cs:387, RouterLog.FileOf).
  • Size tiers are ≤ 9.5B: 1 in host.md and providers.md (ModelLineup.cs:383), and init uses the same ModelLineup.Suggest.
  • ONNX models go to LocalData.ModelsDirectory in the tool (RouterHost.cs:95).
  • Memory page button labels match memory.js:135-137.
  • README init consent: claude and codex are asked about, copilot is added disabled unless --enable-cli names it, and --no-cli adds none (InitCommand.cs:288-345).
  • Dashboard remote access: RequireLocalClient defaults to true and refuses remote callers before any token is checked (DashboardAccess.cs:217-224).
  • Admin rail: Service, JSON, Audit log (core.js:110).
  • Route health is admin-only (EvaluationApi.cs:171).
  • Dashboard admin list: dashboard.md:204 now names every endpoint marked DashboardWriteMetadata or DashboardAdminReadMetadata.
  • RouterDoctor is in AspNetCore and DoctorCommand is in Hosting.
  • The Windows sign-in entry has no restart: the app only polls (DesktopRouter.Refresh).
  • desktop.md "Where things live": logs/ and webview/ are in the local data directory, and desktop.json stays with the configuration.
  • Activity shows the last 200 (shell.js:28,132). Live events can briefly push the in-page list up to 500 (live.js:74) until the next refresh, which is probably where the old number came from.
  • dropped_writes is the only metric not defined on SemanticRouterDiagnostics.
  • Links: all #anchor links in the changed docs resolve against GitHub-style heading slugs.
  • CHANGELOG: no line is needed, because CONTRIBUTING.md#changelog exempts documentation-only changes.

CI is green and the PR doesn't conflict with main. Nothing pushed.

…d, Windows does not restart the service, the database is in the local data directory, init never asks about copilot, reliability reset is admin-only

clients.md and architecture.md: a client's key is checked against the gateway keys (any value from this machine), never forwarded. host.md: restarts on Linux and macOS only. architecture.md: semantic-router.db in the local data directory. desktop.md: models and webview move too, and the uninstall paragraph names both directories. TERMS.md and host.md: claude and codex are asked about; copilot is added disabled unless --enable-cli names it. routing-quality.md: reliability reset needs the admin role. README: gateway key and Service page wording. observability.md: same meter name. Code strings filed as #339.
@siebe41

siebe41 commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

Review addressed. I merged origin/main (937a08d), and the fixes are in 0bb325d.

  1. Medium, docs/clients.md:23 gateway key: fixed in 0bb325d. The comment now says "any value from this machine; with gateway keys set, send yours (required from other machines). Never forwarded". docs/architecture.md's safety rules now say the router never forwards a client's credentials, reads its key only to check it against the gateway keys, and never reads or forwards a CLI's. The config.js:447 snippet is filed as Service restart, gateway key and init consent claims the docs corrected are still in the tool, app and dashboard text #339.
  2. Medium, docs/host.md:55 restart: fixed in 0bb325d. It now says "restarts if it fails (on Linux and macOS; the Windows sign-in entry does not)". The code strings (ServiceCommand.cs:97, WelcomeWindow.cs:39, SettingsWindow.cs:59, service.js:229 and :328, the RouterService.cs:7 comment) are filed as Service restart, gateway key and init consent claims the docs corrected are still in the tool, app and dashboard text #339 and linked from Pre-V1 audit: desktop app and service on real Windows and macOS #208.
  3. Low, docs/architecture.md:115 database location: fixed in 0bb325d. The database is in the local data directory, which is %LOCALAPPDATA%\semantic-router on Windows with the default directory.
  4. Low, docs/desktop.md uninstall and managed deployment: fixed in 0bb325d. The uninstall paragraph uses your suggested wording and names both directories. The managed-deployment line lists models and the web view profile too.
  5. Low, init consent in TERMS.md:139 and host.md:45: fixed in 0bb325d. Both say that claude and codex are asked about, and that copilot is added disabled, even with --yes, unless --enable-cli names it. The Program.cs:70 help text is filed as Service restart, gateway key and init consent claims the docs corrected are still in the tool, app and dashboard text #339.
  6. Low, docs/routing-quality.md:258: fixed in 0bb325d. Resetting a model's reliability count is now in the admin-only sentence, and reading the figures stays open to the read-only role.
  7. nit, README.md:299: fixed in 0bb325d with the same wording as clients.md.
  8. nit, README.md:321: fixed in 0bb325d. The text is now "Service (how the router runs, who can connect, the background service)".
  9. nit, docs/observability.md:124: fixed in 0bb325d. The text is now "under the same meter name (SemanticRouter)".

Also: the ServicePlatforms.cs:406 comment that the body listed as Not done is now part of #339.

Verified: npm ci + npm run build in site/ (VitePress, with its dead-link check) completes, and no test asserts the changed wording. CI: all 15 checks green on 0bb325d (deploy skipped, as on every PR).

@siebe41
siebe41 marked this pull request as ready for review October 23, 2026 16:22
@siebe41
siebe41 merged commit 4b7318e into main Oct 23, 2026
16 checks passed
@siebe41
siebe41 deleted the sprint/release/O03-docs-accuracy branch October 23, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

usage-sprint Opened by the Oct 2026 usage sprint

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant