Repository navigation

gh-160036: Document `tokenize.TokenInfo` separately - #160037

Merged

lysnikolaou merged 5 commits into

python:mainfrom

johnslavik:tokenize-tokeninfo-docs

Oct 21, 2026

Merged

gh-160036: Document `tokenize.TokenInfo` separately#160037
lysnikolaou merged 5 commits into
python:mainfrom
johnslavik:tokenize-tokeninfo-docs

Commits on Oct 21, 2026
- Document tokenize.TokenInfo and its attributes
  johnslavik
  committed
  7e7b3b7
- Clarify TokenInfo field semantics and positions
  johnslavik
  committed
  d12f714
- Address review
  johnslavik
  committed
  94140f1
- Simplify the exact token type description
  johnslavik
  committed
  35e19fc
- Move TokenInfo version note below its description
  johnslavik
  committed
  7ceb205