Skip to content

fs: support encoding option in glob - #66856

Open
XadillaX wants to merge 1 commit into
nodejs:mainfrom
XadillaX:fix-fs-glob-encoding-59202
Open

XadillaX wants to merge 1 commit into
nodejs:mainfrom
XadillaX:fix-fs-glob-encoding-59202

Conversation

@XadillaX

Copy link
Copy Markdown
Contributor

Summary

fs.glob(), fs.globSync(), and fs.promises.glob() always decoded
results as UTF-8. On a filename that is not valid UTF-8, the bytes were
lost at the native-to-JavaScript boundary and could not be recovered
with Buffer.from(), even though the rest of fs supports Buffer paths.

  • Add an encoding option (default 'utf8'). With 'buffer', matched
    paths are returned as Buffer objects preserving the raw filename
    bytes.
  • With withFileTypes, the Dirent name/parentPath and the values
    passed to exclude callbacks use the requested encoding.
  • Traversal and filesystem calls (including the lstat used for path
    callbacks) keep operating on the original raw paths; encoding is
    applied only when crossing into JavaScript.
  • Validate encoding and document the option for all three APIs.

Tests

  • New test/parallel/test-fs-glob-encoding.mjs covering all three
    entry points across encodings, Dirent fields, exclude callbacks,
    relative/root cwd, iterator batching, and (on Linux) filenames
    containing raw non-UTF-8 bytes including read-back through the
    returned Buffer paths.
  • Existing glob, path.glob, glob syntax, permission, and watch-ignore
    suites remain green.

Fixes: #59202

Add an `encoding` option to `fs.glob()`, `fs.globSync()`, and
`fs.promises.glob()`. The glob walker already operates on raw path
bytes; encode results at the native-to-JavaScript boundary instead so
that filenames which are not valid UTF-8 can be returned without loss
when `encoding` is `'buffer'`.

With `withFileTypes`, the `name` and `parentPath` properties of each
Dirent, as well as the values passed to `exclude` callbacks, use the
requested encoding. Filesystem operations, including the lstat used
for path callbacks, keep using the original paths. The default
encoding remains `'utf8'`.

Refs: nodejs#59202
Signed-off-by: XadillaX <i@2333.moe>
Assisted-by: Trae
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Oct 19, 2026
@codecov

codecov Bot commented Oct 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.84211% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.50%. Comparing base (1d5f8e6) to head (202605f).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/glob/node_glob.cc 81.53% 3 Missing and 9 partials ⚠️
src/glob/glob_walker.cc 87.50% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #66856   +/-   ##
=======================================
  Coverage   90.50%   90.50%           
=======================================
  Files         797      797           
  Lines      278196   278269   +73     
  Branches    53453    53470   +17     
=======================================
+ Hits       251772   251850   +78     
+ Misses      16797    16787   -10     
- Partials     9627     9632    +5     
Files with missing lines Coverage Δ
lib/internal/fs/glob.js 97.94% <100.00%> (+0.01%) ⬆️
src/glob/glob_walker.h 100.00% <ø> (ø)
src/glob/node_glob.h 42.85% <ø> (ø)
src/glob/glob_walker.cc 86.07% <87.50%> (+0.47%) ⬆️
src/glob/node_glob.cc 76.19% <81.53%> (+0.73%) ⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fs.globSync returns incorrect results with non-UTF8 encoded file names.

2 participants