Skip to content

Read a semicolon between post-nominals as a comma (D15) - #871

Merged
derek73 merged 5 commits into
masterfrom
feat/semicolon-suffix-delimiter
Oct 25, 2026
Merged

derek73 merged 5 commits into
masterfrom
feat/semicolon-suffix-delimiter

Conversation

@derek73

@derek73 derek73 commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

D15 of the vocabulary phase 2 spec (#752): the semicolon becomes a default suffix delimiter, and a delimiter glued to a suffix word counts.

Behavior

Input 1.4.0 – 2.5.0 Now
John Smith, MD; PhD given MD;, family John Smith, suffix PhD given John, family Smith, suffix MD; PhD
John Smith, MD;PhD given MD;PhD suffix MD;PhD
Smith, MD; PhD given MD; family Smith, suffix MD; PhD
John Smith, MA; PhD given MA; suffix MA; PhD (reported, as MA PhD is)
John Smith, MD ; PhD title MD, given ; (2.5.0) suffix MD, PhD (the lone delimiter is dropped, as a declared one is)
Berg, Jan van; MD middle van; unchanged: a glued particle joins nothing, backward included (accepted, Derek)
John Smith MD; PhD family MD; unchanged: a semicolon before the first comma is rules/T1's question, out of scope

Design

  • The default is a constant beside _vocab.delimiter_cores, not a Policy.extra_suffix_delimiters default. That field is a set callers replace (the facade's suffix_delimiter replaces it too), so a default inside it would vanish for anyone declaring their own delimiter. No policy removes the semicolon, as none removes the comma.
  • A glued word reads as its stem. The stem is "the word without the delimiter". One predicate, _vocab.glued_stem, answers it, and four sites ask it:
    • classify tags the word as its stem, after the first comma only;
    • _comma.decide uses the stem in its shape tests;
    • is_wholly_suffix uses it in the run test;
    • segment's C2 title test uses it.
  • Why the stem is read everywhere. Reading it in the run test alone, as the spike did, put 365 reversed names right. Tagging the stem raised that to 412, and reading it in decide's shape tests (MBBS;) to 431.
  • Guards.
    • Behind a head that is not a whole name, the glued word takes the strict test, so Smith, V; MD keeps given V;, as Smith, V MD does.
    • A glued particle, connective or bound given name loses its joining tags, so nothing joins across the delimiter (Berg, de; Smith keeps given de;); P6's backward attachment goes with it, an accepted cost.
    • Every gluable core is tried in sorted order, so a word like MD/PhD; reads the same whatever the hash seed.
    • Only a core with no letter or digit glues: under " and ", Stand is not St with and glued on.

Measured against 638685b

Corpus Result
ClinicalTrials.gov officials (local, data 2026-10-21) 494 names hold a semicolon. 443 move: 431 reversed names put right, 7 titles now read as titles, 2 standing-alone semicolons now rendered as a comma, 3 others. All read by hand, none worse.
FEC (15 semicolon names), TSE, PSC agreement, committed snapshot, CLDR/GND agreement Nothing moves. The snapshot only gains the 12 new rules examples.
  • Twin sweep: 269,136 generated glued texts across seven comma heads and three orders. No text that read as its space twin on the base reads otherwise now.
  • Invariants: test_a_glued_semicolon_reads_as_its_space_twin (990 texts, 726 disagreed at base) and test_a_semicolon_standing_alone_in_a_tail_reads_as_its_comma_twin (129 texts, all disagreed at base).
  • Frame cost: reference name 307 → 305; Smith, John 161 → 158.

Docs

  • rules/C10 is renamed to "a suffix delimiter parts a suffix run" and rewritten, gaining a Precedence bullet and two Accepted bullets. Its rule-size record grew.
  • C1's map line and R1's separator sentence are amended.
  • decisions/C10 has a new entry.
  • AGENTS.md's spellings paragraph gains the glued-separator lesson.
  • Also updated: customize.rst, the Policy docstring and the release log.
  • A design-docs review found 8 issues, including two wrong readings (Stand under " and ", and Berg, Jan van; MD), all addressed. A second review of those fixes found a hash-seed-dependent reading (MD/PhD;), fixed, and a forward join across a glued particle, resolved by the decision above.

Decided

  • No opt-out switch: the semicolon is built in like the comma (Derek, 2026-10-24; recorded in decisions/C10).
  • A glued particle's tags are dropped, so Berg, Jan van; MD keeps 1.4.0's middle van; (Derek, 2026-10-24; rules/C10's Accepted bullet).

Left for release

  • migrate.rst entries are left for the 2.6.0 release step, as with the cycle's other PRs.

🤖 Generated with Claude Code

The semicolon is a suffix delimiter every policy has, beside the comma
(rules/C10). A delimiter written without a letter or digit, glued to
one end of a word after the first comma, leaves the word read as the
word without it, its text kept: "John Smith, MD; PhD" reads given
John, family Smith, suffix "MD; PhD", where 1.4.0 through 2.5.0
reversed the name. One predicate, _vocab.glued_stem, serves classify's
tags, _comma.decide's shape tests, is_wholly_suffix and segment's C2
title test; a glued connective or bound given name joins nothing
across the delimiter. A semicolon before the first comma stays a word
(rules/T1's question, out of scope).

ClinicalTrials.gov officials: 443 of 494 semicolon names move, 431 a
reversed name put right; nothing moves in the committed corpora, CLDR,
GND, FEC, TSE or PSC (decisions/C10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added enhancement docs Documentation fixes and updates tests labels Oct 24, 2026
@derek73 derek73 self-assigned this Oct 24, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.06%. Comparing base (638685b) to head (29f3a80).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #871      +/-   ##
==========================================
+ Coverage   99.05%   99.06%   +0.01%     
==========================================
  Files          47       47              
  Lines        4669     4730      +61     
==========================================
+ Hits         4625     4686      +61     
  Misses         44       44              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 3 commits October 24, 2026 16:49
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rder

glued_stem stopped at the first core the frozenset yielded, so a word
holding one core inside and another at its end ('MD/PhD;' under '/')
read two ways by hash seed. It now tries every gluable core in sorted
order; decide's flag loop no longer stops at the first core, and
segment's title test pre-checks for a core before calling glued_stem.
Found by the second review of #871.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kept, a particle with a semicolon glued after it chained forward across
it ('Berg, de; Smith' read family 'Berg de; Smith'). Dropping the tags
also drops P6's backward attachment, so 'Berg, Jan van; MD' reads middle
'van;' as every release from 1.4.0 did; recorded as accepted in
rules/C10 (Derek, 2026-10-24).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates enhancement tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant