Skip to content

test_runner: merging coverage across test processes drops never-run blocks, so coverage gates pass #66860

Open

Description

@steida

Version

v24.22.0, also v24.20.0 (the merge code is unchanged on main as of 2026-10-19)

Platform

Darwin 25.6.0 arm64 (the bug is in platform-independent JavaScript)

Subsystem

test_runner

What steps will reproduce the bug?

mod.mjs:

export const f = (a, b) => {
  if (a) {
    if (b) {
      return "never";
    }
    return "a";
  }
  return "other";
};

a.test.mjs:

import assert from "node:assert";
import { test } from "node:test";
import { f } from "./mod.mjs";

test("a", () => {
  assert.strictEqual(f(true, false), "a");
  assert.strictEqual(f(false, false), "other");
});

b.test.mjs:

import assert from "node:assert";
import { test } from "node:test";
import { f } from "./mod.mjs";

test("b", () => {
  assert.strictEqual(f(false, false), "other");
});
node --test --experimental-test-coverage --test-coverage-include=mod.mjs --test-coverage-branches=100 a.test.mjs
node --test --experimental-test-coverage --test-coverage-include=mod.mjs --test-coverage-branches=100 a.test.mjs b.test.mjs

How often does it reproduce? Is there a required condition?

Always. It needs two test processes (the default --test-isolation=process) where one runs a block but not a block nested in it, and the other runs neither.

What is the expected behavior? Why is that the expected behavior?

Both commands report the same coverage of mod.mjs, because b.test.mjs runs nothing that a.test.mjs does not: lines 4-5 uncovered, 80% branches, and the threshold fails with exit code 1. The first command does that, and so does the second with --test-isolation=none.

What do you see instead?

The second command reports 100% lines and branches and exits 0:

ℹ mod.mjs   | 100.00 |   100.00 |  100.00 |

Adding a test file that covers less made a failing coverage gate pass.

Additional information

The raw V8 coverage of f (from NODE_V8_COVERAGE), per process:

a: [{17,120,2}, {38,100,1}, {51,80,0}, {100,119,1}]
b: [{17,120,1}, {38,100,0}]
merged by the test runner: [{17,120,3}, {38,100,1}, {100,119,1}]

V8 leaves out a block whose count equals its parent's, so b reports the whole if (a) block (38-100) as 0, and a reports the nested if (b) block (51-80) as 0. Neither process ran 51-80, but the merge drops it.

mergeCoverageRanges in lib/internal/test_runner/coverage.js merges ranges as sets, keeping a zero range only when the other process reports the same range or a zero range containing it. Both checks miss here, in either file order:

  • a merged first: b's 38-100 matches the old 38-100 exactly, and the break ends the inner loop before the containment check reaches 51-80.
  • b merged first: a's 38-100 matches first and adds its count to the old 38-100 in place, so that range is no longer 0 when a's 51-80 is checked for containment.

The merge can drop zero ranges but never invents one, so the error always overstates coverage and thresholds pass. Line counts are also not summed unless ranges match exactly.

A merge that matches V8's own semantics sums, for each source position, the count of the innermost range that contains it in each process, as c8 does through mergeProcessCovs from @bcoe/v8-coverage.

We found this in Evolu, whose @evolu/sqlite-wasm package requires 100% line, branch, and function coverage from node:test across 26 test files. The gate passed while an out-of-memory branch in its database open ran in no test, which a single-file run showed. The same merge hid three more never-run blocks in Evolu's two coverage gates. It may belong on the coverage stabilization tracker, #63506.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions