Repository navigation
[heft-lint-plugin] Additional-file enumeration parses every file and re-lints other programs' files in solution builds #6151
Description
Summary
Since 1.3.0 ("Lint files selected by ESLint flat config even when they are not part of the TypeScript program"), @rushstack/heft-lint-plugin enumerates "additional" files with a second ESLint instance (Eslint.ts, #fileEnumerator.lintFiles(['.'])). Two things make this very slow in larger projects:
- The enumeration parses every selected file with the configured parser. Rules are disabled (
ruleFilter: () => false), but ESLint still parses each file. With a type-aware parser (@typescript-eslint/parserwithproject/projectService), each file outside the program also triggers TypeScript program creation. The enumerator also ignores the program's files through one escaped glob per file, which is slow to match when there are thousands. - In a solution build (
buildProjectReferenceswith several referenced tsconfigs), only the programs that TypeScript emitted in this run are reported to the lint plugin.allProgramFilenamestherefore contains only those programs' files, and every file of the unchanged programs is treated as an additional file:- it is parsed by the enumerator (point 1), and
- it is linted again as an additional file, with a lint cache (
_eslint-<tsconfig hash>.json) keyed by the file list of that particular combination, which is empty after a full build. So the first incremental build that changes only one program re-lints the files of all other programs.
Repro
Numbers from a project with three programs (main 2,971 files, a second program with 213, a generated program with 1,082 roots). ESLint 9.37, TypeScript 5.3, heft-lint-plugin 1.4.0, config/typescript.json with buildProjectReferences: true and a solution tsconfig referencing three tsconfigs.
Scenario (incremental heft run --only build) |
Lint time | Files linted |
|---|---|---|
| Full clean build | 122 s | 2,977 + 214 + 0 |
| Change touching only the generated program, cold cache | 144 s | 3,191 |
| Same, warm cache | 63 s | 0 (enumeration alone) |
| Change touching only the main program | 132 s | 3,191 |
Enumeration only (the #fileEnumerator.lintFiles(['.']) call), same project:
| Enumerator configuration | Files listed | Additional | Time |
|---|---|---|---|
| As shipped, only the changed program ignored | 3,190 | 3,190 | 47.5 s |
| As shipped, all programs ignored | 7 | 7 | 21.5 s |
| Stub parser, no ignore globs, program files dropped afterwards | 3,190 | 7 | 3.1 s |
A CPU profile of the warm case shows most of the time in ESLint's readAndVerifyFile / @typescript-eslint/typescript-estree getProgramAndAST, @eslint/config-array matching, and calculateConfigForFile for the per-file cache hash.
Proposed fix
- Enumerate without parsing: give the enumerator a stub parser for JS/TS files (it has no rules to run) and drop the
ignoresoverride;getExtraSourceFilesToLintAsyncalready removes program files afterwards. - When the programs come from the TypeScript plugin, add the root files of every program of the solution (the
projecttsconfig fromconfig/typescript.jsonand the tsconfigs it references in the project folder) toallProgramFilenames. Files of unchanged programs are linted with their own program when it changes, which is the behavior before 1.3.0 for program files. (Alternative: have the TypeScript plugin report every program of the solution, or expose them on its accessor.)
Checked on two projects (one with three programs, one with two):
- full builds: identical linted sets and findings, same additional files;
- a non-program
.tsfile selected by the flat config is still linted and reported; - incremental single-program changes lint only that program's changed files plus the additional files.
Expected effect on the repro: the generated-program-only incremental build goes from 144 s (cold) / 63 s (warm) of lint to a few seconds. Happy to open a PR with this change.
AI-generated with GitHub Copilot on behalf of Naman Kanakiya (@namankanakiya); measurements were run on real builds.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsNeeds triage
Activity