Repository navigation
Check the vocabulary word order in the build (check 10) - #767
Merged
Merged
Conversation
41 of the 71 language files, every one but und.toml: by kind in CATEGORY's order, then by the same_as path from the root head, then by text. Whole entries move; nothing inside one changes, so the compiled module is byte-identical. The next commit adds the check that holds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/vocab/build.py refuses a language file whose words are out of order, naming the entry and the word it belongs after; it never rewrites a file, since a comment above an entry must move with it. und.toml is exempt: its vetting comments point along the order of the Python lists it came from. A same_as chain returning to its start is refused too, the key following the chains. The README's kind table is now held to CATEGORY's order, and nameparser/vocab/README.md states the rule; decisions/vocabulary-model records why. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #767 +/- ##
=======================================
Coverage 99.05% 99.05%
=======================================
Files 47 47
Lines 4642 4642
=======================================
Hits 4598 4598
Misses 44 44 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
… loop (#747) From the #767 review: - the order check runs last in check_pool, so an exclusion's reason (check 6) and a same_as typo (check 7) are seen before it; the en.toml exclusion test is back on en.toml - the message names the first entry outside a longest run already in order, so a word appended at the end is named, not its neighbours - a file's same_as path follows that file's entries only, so another file's chain no longer decides its order - the loop refusal walks every same_as written, across files and kinds, and names the loop rather than the chain into it - the und.toml example quoted the wrong way; und.toml:168 said "above" of a note 2,900 lines below Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…747) - the named place is against the run already in order, so following the messages takes exactly n less the longest in-order run moves (the first version gave a neighbour that was out of place itself: up to 22 moves where 10 would do); held over every order of six words - _heads leaves out a same_as naming its own word, so check_order called alone ends; the loop search is iterative, so a chain longer than the recursion limit is no RecursionError Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derek73
added a commit
that referenced
this pull request
Oct 20, 2026
…#750) (#769) * Add title kinds; classify the salutations and given-addressing titles (#750) Ten title kinds join the generic `title`: salutation, style, academic, religious, nobility, military, office, occupation, kinship and modifier, defined in nameparser/vocab/README.md. The salutations of every language and the 49 titles that address by the given name leave und.toml for the files of the languages whose names they occur in, six of them new (ak, et, id, ro, sw, vi); the module comments become `expands`, `note`, `issue`, `same_as` and, on a marked word only, `gender`. The title keep-outs that decisions/ prose held become `excluded` entries the build enforces, with their counts as name readings where counts were taken (malam, bay, shehu). The per-word counts of decisions/salutation-titles move onto each word's note, and the prose points at the query. Tests pin every check on the title category in a fine kind as well as the generic one, each row failing when its check is keyed on the kind. No compiled set moves: build --check is current and the snapshot matches. Part of #750. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the docs review of #750 part 1 shehu's argument against a graf-class reading of its counts is back in vocabulary-exclusions, with malam's contrast and both CLDR greps; the single-letter trap (bare د, κ) is stated apart from the transliteration rule; lord, lady and knight are named as not yet nobility; the forty-six refusal-only entries are dated to the tree they were counted on; und.toml's positional comments are described without a quote that moved; paní's note keeps that the unaccented pani is not shipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the second docs review of #750 part 1 The single-letter trap is decided per word, and the Bengali ড is named as the decided exception; the Arabic refusals no longer claim their article forms ship; shehu's bullet drops the digits its entry holds; the in-place correction to the #767 entry says what was wrong; the graf-class pointers say which words carry counts and which file holds herra's note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the PR review of #769 und.toml's Hebrew, Devanagari civil and Bengali comments no longer point at blocks this PR moved out, and the #606 header names its noble ranks; the Hebrew comment gives both boundary tests the mid-word quote fails; גב''s note gives the mechanism that applies to a word-final quote; bà's reason says it was refused as ông's pair without its own evidence; الدكتور's note says it speaks for the group. The README calls one kind per file a convention, not a check, and says its examples include words the later parts of #750 classify. Tests: an addresses exclusion's reason across title kinds (check 6) and a same_as across title kinds (check 7), each failing under its check keyed on the kind; and a title-kind exclusion leaving another category free, which fails if the exclusion over-widens to every reading of the word. The per-kind compile test's comment no longer overstates how cased is read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record the und.toml stale-pointer check in the vocab README (#750) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #747. #750 (the titles) waits for this, so its words land in a checked order.
What
tools/vocab/build.pygains check 10: every language file exceptund.tomllists its words in one canonical order, and the build refuses a file out of order, naming the entry to move and where it belongs (en.toml: 'snr' (generation) belongs after 'sr'. …). The entry it names is the first one outside the longest run already in order, and its place is given against that run, so following the messages takes the fewest moves and a word appended at the end of a file is the one named. It checks and never rewrites a file, because a comment above an entry belongs to it and must move with it, which no TOML reader can see. Check 10 runs last, so an exclusion's reason (check 6) or asame_asnaming no word (check 7) is reported first.The order:
CATEGORY's order. It already listed each category's kinds together and the README's kind table already followed it; an import-time assert now keeps the kinds of a category together, and the README test is now sensitive to the table's order. A word with several readings sorts under its first kind. An entry that only records a refusal (senior,fils) sorts under the kind it refuses; a name reading places an entry only when it has nothing else.same_aschain, starting from the word at the root, with the head first:jr,jnr,junior, thensr,snr. Because the whole chain is used,el-keeps its own spellings after it inside theal-group in ar.toml, where sorting by the direct target alone would have split them. Only the file's own entries are followed, so another file's chain never decides this file's order.z(aḍ-afteraz-), and that traditional and simplified spellings with nosame_asbetween them end up apart (課長,课长).und.tomlis exempt: its vetting comments point along the order of the Python lists it came from ("the bare-κ trap above"). Asame_aschain that loops back on itself (e.g.jr→jnr→jr) is refused in every file,und.tomlincluded. The check looks at everysame_aswritten, across files and kinds, because nothing holds a word to a single head.Data
The first commit sorts the 41 of 71 language files that were out of order on cc030b9. Whole entries move and nothing inside an entry changes. The second commit adds the check, so each commit builds.
Docs
nameparser/vocab/README.md: a new "Word order" section, and the "Adding a word" steps now point to it.decisions/vocabulary-model: a dated bullet recording why the order is checked rather than rewritten, why it uses the chain rather than the direct target, why codepoint order, and whyund.tomlis exempt.Verification
uv run python tools/vocab/build.py --check:_compiled.pyis current (byte-identical).tools/differential/snapshot.py: matches the snapshot.Full suite: 21,909 passed, 123 skipped, 3 xfailed. mypy and ruff are clean.
New rows in
tests/v2/test_vocab.pycover a swapped pair, kinds and categories out of order, a variant before its head, a variant away from its head, refusals placed outside their kind, and loops in a language file and inund.toml. The accepted shapes cover a head before its spellings, a refusal beside a name reading, a comment moving with its entry, the ar.toml chain, two scripts in one file, andund.tomlout of order passing.Review rounds.
/pr-review-toolkit:review-prran five reviewers, and dd7b335 fixes what they found:same_asloops passed;und.tomlpointed the wrong way.A second-round review of dd7b335 found that the given place could be beside an entry that was out of place itself (up to 22 moves where 10 would do), plus a hang when
check_orderis called on its own and aRecursionErroron a very long chain. 0a223d8 fixes all three. Each new row fails against the commit before its fix: some raise nothing, the rest name the wrong entry or the wrong loop.test_following_the_order_messages_takes_the_fewest_moveschecks the move count over every order of six words.Negative controls, measured: with
check_orderdisabled, every order row compiles, so the check is what refuses each one. Under a plain-text key, the two variant rows pass, so only the head-first grouping fails them. Under a direct-same_askey, the ar.toml chain shape fails.🤖 Generated with Claude Code