[derek73]

What

tools/name_usage.py rebuilt each local corpus's word index on every run: 5.8 s of the FEC corpus's 7 s (after #834), and 1.6 s of ClinicalTrials.gov's. This PR does two things:

• Caches the index as wordindex.json beside each corpus in ~/.cache/nameparser-tools.
• Reads Local.records lazily, from raw.jsonl on first use, so a single-word lookup answered from the cache parses no records.

Measured 2026-10-22, a single-word name_usage.py --no-wikidata run, with output byte-identical to the pre-cache run:

	Time
before	12.5 s
the run that builds the caches (once per fetch or fold change)	14.8 s
cached	2.5 s
spaced word (Ph. D.), which still scans every name	8.6 s

How the cache stays honest

A stale index gives wrong counts that look right, so the key is computed, not versioned by hand. It combines:

• The records' sha256 from meta.json, so a refetch rebuilds.
• fold_fingerprint(), a hash over:
  • the code objects of _FOLD_CODE (word_key, _strippable, _keyed, _names, word_index) and of corpus_build.jsonl_lines. It hashes the code that is running, not the source on disk, so an edit made during a long run cannot store an old fold under a new key. Line numbers are left out, so moving code doesn't trigger a rebuild.
  • the constants they read (_TOKEN_SPLIT, _JOINERS, with sets sorted so the key is the same in every process);
  • WordIndex's field names;
  • unicodedata.unidata_version and the interpreter version.
• A test holding the hand-kept lists honest: test_the_fold_lists_everything_its_functions_call_and_read walks the global names the fold's functions use and fails on any that isn't listed.

Other choices:

• A cached file is used only when its key matches and every count is an int. Anything else is rebuilt, with a line on stderr.
• JSON, not pickle, so a planted cache file cannot run code.
• WordIndex holds plain dicts, built or cached alike, read with .get(key, 0). Rebuilding Counters on load had cost 0.37 s of 0.61 s.

Errors

• A damaged corpus found when the records are read now raises CorpusDamaged, a subclass of ValueError, so existing callers still catch it.
• name_usage guards each corpus's lookup but catches only CorpusDamaged and OSError. A damaged corpus fails its own row, while a bug in a lookup still raises.
• tally exits naming the fetch, as require() does.
• Both test modules point the cache at a scratch folder, so no test can write an index into a developer's real cache.

Review notes

• Review rounds: two. First a code reviewer; then test, silent-failure, comment and type reviewers. Every confirmed finding is fixed. The silent-failure review found the source-on-disk fingerprint race and the unchecked inner values; the test review found the missing no-sha guard test.
• Negative controls: all re-taken on the final tree, and each of the 15 guards' tests fails with it removed. Two controls from the first round turned out to be vacuous and their tests were fixed: the no-sha guard's test passed because the cache folder didn't exist, and nothing reached tally's damage path.
• Dropped: the first round's index read inside load_locals turned out to be redundant once each lookup was guarded (its control passed), so it was removed.
• Not changed: the spaced-word scan, still about 8.6 s on the FEC corpus.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.05%. Comparing base (7a76e8a) to head (11aecff).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #836   +/-   ##
=======================================
  Coverage   99.05%   99.05%           
=======================================
  Files          47       47           
  Lines        4659     4659           
=======================================
  Hits         4615     4615           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.