Skip to content

[heft-lint-plugin] Additional-file enumeration parses every file and re-lints other programs' files in solution builds #6151

Open

Description

Summary

Since 1.3.0 ("Lint files selected by ESLint flat config even when they are not part of the TypeScript program"), @rushstack/heft-lint-plugin enumerates "additional" files with a second ESLint instance (Eslint.ts, #fileEnumerator.lintFiles(['.'])). Two things make this very slow in larger projects:

  1. The enumeration parses every selected file with the configured parser. Rules are disabled (ruleFilter: () => false), but ESLint still parses each file. With a type-aware parser (@typescript-eslint/parser with project / projectService), each file outside the program also triggers TypeScript program creation. The enumerator also ignores the program's files through one escaped glob per file, which is slow to match when there are thousands.
  2. In a solution build (buildProjectReferences with several referenced tsconfigs), only the programs that TypeScript emitted in this run are reported to the lint plugin. allProgramFilenames therefore contains only those programs' files, and every file of the unchanged programs is treated as an additional file:
    • it is parsed by the enumerator (point 1), and
    • it is linted again as an additional file, with a lint cache (_eslint-<tsconfig hash>.json) keyed by the file list of that particular combination, which is empty after a full build. So the first incremental build that changes only one program re-lints the files of all other programs.

Repro

Numbers from a project with three programs (main 2,971 files, a second program with 213, a generated program with 1,082 roots). ESLint 9.37, TypeScript 5.3, heft-lint-plugin 1.4.0, config/typescript.json with buildProjectReferences: true and a solution tsconfig referencing three tsconfigs.

Scenario (incremental heft run --only build) Lint time Files linted
Full clean build 122 s 2,977 + 214 + 0
Change touching only the generated program, cold cache 144 s 3,191
Same, warm cache 63 s 0 (enumeration alone)
Change touching only the main program 132 s 3,191

Enumeration only (the #fileEnumerator.lintFiles(['.']) call), same project:

Enumerator configuration Files listed Additional Time
As shipped, only the changed program ignored 3,190 3,190 47.5 s
As shipped, all programs ignored 7 7 21.5 s
Stub parser, no ignore globs, program files dropped afterwards 3,190 7 3.1 s

A CPU profile of the warm case shows most of the time in ESLint's readAndVerifyFile / @typescript-eslint/typescript-estree getProgramAndAST, @eslint/config-array matching, and calculateConfigForFile for the per-file cache hash.

Proposed fix

  1. Enumerate without parsing: give the enumerator a stub parser for JS/TS files (it has no rules to run) and drop the ignores override; getExtraSourceFilesToLintAsync already removes program files afterwards.
  2. When the programs come from the TypeScript plugin, add the root files of every program of the solution (the project tsconfig from config/typescript.json and the tsconfigs it references in the project folder) to allProgramFilenames. Files of unchanged programs are linted with their own program when it changes, which is the behavior before 1.3.0 for program files. (Alternative: have the TypeScript plugin report every program of the solution, or expose them on its accessor.)

Checked on two projects (one with three programs, one with two):

  • full builds: identical linted sets and findings, same additional files;
  • a non-program .ts file selected by the flat config is still linted and reported;
  • incremental single-program changes lint only that program's changed files plus the additional files.

Expected effect on the repro: the generated-program-only incremental build goes from 144 s (cold) / 63 s (warm) of lint to a few seconds. Happy to open a PR with this change.

AI-generated with GitHub Copilot on behalf of Naman Kanakiya (@namankanakiya); measurements were run on real builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    • Status
      Needs triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions