Skip to content

buffer: indexOf/includes clamp byteOffset to 2^31-1 and return a negative index on buffers over 2 GiB #67008

Open

Description

@ozonedev27

Version

v24.22.1 (also v22.23.3)

Platform

Linux 5.15.167.4-microsoft-standard-WSL2 x86_64 GNU/Linux

Subsystem

buffer

What steps will reproduce the bug?

// Buffer.indexOf / includes with a byteOffset above 2^31 - 1
const G = 2 ** 30;
const buf = Buffer.alloc(3 * G);
buf[2.25 * G] = 0x0a; // a newline BEFORE byteOffset
buf[2.75 * G] = 0x0a; // the first newline AFTER byteOffset

const from = 2.5 * G;
const got = buf.indexOf(0x0a, from);
console.log(`indexOf(0x0a, ${from})   = ${got}  (expected ${2.75 * G})`);
console.log(`  returned index < byteOffset: ${got !== -1 && got < from}`);
console.log(`includes(0x0a, ${2.9 * G}) = ${buf.includes(0x0a, 2.9 * G)}  (expected false)`);
console.log(`subarray(from).indexOf + from = ${from + buf.subarray(from).indexOf(0x0a)}  (correct)`);

How often does it reproduce? Is there a required condition?

Always, when the buffer is larger than 2 GiB and byteOffset is above 2^31 - 1.

What is the expected behavior? Why is that the expected behavior?

indexOf(0x0a, 2684354560)   = 2952790016  (expected 2952790016)
  returned index < byteOffset: false
includes(0x0a, 3113851289.6) = false  (expected false)

indexOf should return the first match at or after byteOffset, and buffers this size are within buffer.constants.MAX_LENGTH.

What do you see instead?

indexOf(0x0a, 2684354560)   = -1879048192  (expected 2952790016)
  returned index < byteOffset: true
includes(0x0a, 3113851289.6) = true  (expected false)
subarray(from).indexOf + from = 2952790016  (correct)

Additional information

Two problems combine here:

  1. byteOffset appears to be clamped to 2^31 - 1, so the search starts at 2 GiB. It finds the newline at 2.25 GiB, which is before the requested offset, and includes returns true for a range that has no match.
  2. The returned index is then wrapped to int32: -1879048192 is 2415919104 - 2^32, the position of the 2.25 GiB newline. A negative non--1 result isn't a valid return value.

Searching a subarray gives the right answer, so only the offset argument and the return value are affected.

Bun 1.4.4 has the same offset clamp (without the negative return). Reported there separately.

Activity

  1. vedchaudhari commented on Oct 26, 2026

    @vedchaudhari
    Contributor

    The negative return values are already fixed on main by #66325, but the offset part is still broken.

    I think the int32 clamp in bidirectionalIndexOf() was a design decision back in 2016 (6c1e5ad), since buffers couldn't be bigger than 2 GiB then. MAX_LENGTH grew a lot after v14, but the clamp never got updated, so now it gives wrong results on bigger buffers. lastIndexOf() is affected too.

    I'll open a PR for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions