Skip to content

lib: optimize TextEncoder.encode for short strings - #66831

Open
mertcanaltin wants to merge 4 commits into
nodejs:mainfrom
mertcanaltin:mert/textencoder-small-string
Open

mertcanaltin wants to merge 4 commits into
nodejs:mainfrom
mertcanaltin:mert/textencoder-small-string

Conversation

@mertcanaltin

Copy link
Copy Markdown
Member

This pr improvement short strings for TextEncoder.encode.

Benchmark results:

(.venv) ➜  node git:(mert/textencoder-small-string) ✗ npx node-benchmark-compare te-clean.csv                                                        
                                                                          confidence improvement accuracy (*)   (**)  (***)
util/text-encoder.js op='encode' type='ascii' n=1000000 len=256                    *      1.67 %       ±1.45% ±1.94% ±2.54%
util/text-encoder.js op='encode' type='ascii' n=1000000 len=32                   ***    127.18 %       ±1.93% ±2.58% ±3.40%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=256                 1.39 %       ±2.07% ±2.79% ±3.68%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=32         ***     85.44 %       ±2.43% ±3.27% ±4.31%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=256                -0.12 %       ±0.86% ±1.15% ±1.49%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=32         ***     74.83 %       ±1.45% ±1.94% ±2.54%

Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 6 comparisons, you can thus expect the following amount of false-positive results:
  0.30 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.06 false positives, when considering a   1% risk acceptance (**, ***),
  0.01 false positives, when considering a 0.1% risk acceptance (***)

I got help from Claude Opus 5.5. I reviewed the changes and ran the tests and benchmarks myself.

Assisted-by: Claude opus 5.5
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
@nodejs-github-bot nodejs-github-bot added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. labels Oct 18, 2026
Comment thread test/parallel/test-whatwg-encoding-custom-textencoder-short.js Outdated
mertcanaltin and others added 2 commits October 18, 2026 16:51
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
@mertcanaltin
mertcanaltin requested a review from aduh95 October 18, 2026 13:58
@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.51%. Comparing base (e2d5522) to head (cdf9f11).
⚠️ Report is 57 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66831      +/-   ##
==========================================
+ Coverage   90.50%   90.51%   +0.01%     
==========================================
  Files         797      797              
  Lines      277793   278237     +444     
  Branches    53366    53466     +100     
==========================================
+ Hits       251405   251838     +433     
+ Misses      16801    16765      -36     
- Partials     9587     9634      +47     
Files with missing lines Coverage Δ
lib/internal/encoding.js 96.75% <100.00%> (+0.08%) ⬆️

... and 61 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread lib/internal/encoding.js
@mertcanaltin
mertcanaltin requested a review from anonrig October 19, 2026 17:23
@aduh95 aduh95 added author ready PRs with CI started, the required approvals, and no outstanding review comments. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. commit-queue-squash PRs the Commit Queue should land as one squashed commit. labels Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Benchmark GHA (util / text-encoder): https://github.com/nodejs/node/actions/runs/39607879277

Results

[!WARNING]
Do not take GHA benchmark results as face value, always confirm them
using a dedicated machine, e.g. Jenkins CI.

Benchmark results:

                                                                               confidence improvement accuracy (*)    (**)   (***)
util/text-encoder.js op='encode' type='ascii' n=1000000 len=1024                              -2.59 %      ±17.90% ±23.59% ±30.27%
util/text-encoder.js op='encode' type='ascii' n=1000000 len=256                                2.01 %      ±22.07% ±29.09% ±37.33%
util/text-encoder.js op='encode' type='ascii' n=1000000 len=32                        ***    172.54 %      ±31.10% ±41.03% ±52.72%
util/text-encoder.js op='encode' type='ascii' n=1000000 len=8192                               0.66 %      ±16.04% ±21.14% ±27.13%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=1024                     2.12 %      ±18.63% ±24.56% ±31.51%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=256                     -1.85 %      ±19.74% ±26.02% ±33.39%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=32              ***    140.33 %      ±25.69% ±33.87% ±43.49%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=8192                     0.51 %       ±7.77% ±10.24% ±13.14%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=1024                    -1.62 %      ±16.32% ±21.51% ±27.60%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=256                     -1.19 %      ±18.34% ±24.17% ±31.01%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=32              ***    132.39 %      ±22.45% ±29.60% ±37.99%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=8192                    -0.16 %       ±7.29%  ±9.61% ±12.33%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=1024                          -0.30 %      ±17.92% ±23.63% ±30.31%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=256                            1.12 %      ±20.39% ±26.88% ±34.49%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=32                            -4.04 %      ±17.34% ±22.85% ±29.33%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=8192                           0.84 %       ±8.20% ±10.81% ±13.87%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=1024                 0.86 %      ±13.22% ±17.42% ±22.36%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=256                 -0.60 %      ±16.01% ±21.11% ±27.08%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=32                   2.46 %      ±19.34% ±25.49% ±32.70%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=8192                 0.50 %       ±5.69%  ±7.50%  ±9.62%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=1024                 0.03 %       ±9.57% ±12.61% ±16.19%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=256                  2.47 %      ±12.39% ±16.33% ±20.96%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=32                   2.54 %      ±17.87% ±23.56% ±30.23%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=8192                 0.04 %       ±6.00%  ±7.91% ±10.15%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 24 comparisons, you can thus
expect the following amount of false-positive results:
  1.20 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.24 false positives, when considering a   1% risk acceptance (**, ***),
  0.02 false positives, when considering a 0.1% risk acceptance (***)

[!WARNING]
Do not take GHA benchmark results as face value, always confirm them
using a dedicated machine, e.g. Jenkins CI.

@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 21, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-squash PRs the Commit Queue should land as one squashed commit. encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants