Skip to content

stream: add sync iterable fast path to pipeTo#63318

Open
trivikr wants to merge 2 commits into
nodejs:mainfrom
trivikr:stream-iter-pipeto-sync-source
Open

stream: add sync iterable fast path to pipeTo#63318
trivikr wants to merge 2 commits into
nodejs:mainfrom
trivikr:stream-iter-pipeto-sync-source

Conversation

@trivikr
Copy link
Copy Markdown
Member

@trivikr trivikr commented May 15, 2026

Description

This adds a pipeTo() fast path for synchronous iterable sources when there
are no transforms, no abort signal, and the writer supports writeSync().

Previously, pipeTo() always normalized through from(), which routes sync
iterables through the async normalization path. The new guarded path writes
incrementally from the sync iterator while preserving async fallback behavior
for values that still require async normalization.

Benchmark

                                                                               confidence improvement accuracy (*)    (**)   (***)
streams/iter-throughput-pipeto.js n=5 datasize=1048576 api='iter-sync-source'         ***     18.03 %       ±3.54%  ±4.86%  ±6.64%
streams/iter-throughput-pipeto.js n=5 datasize=16777216 api='iter-sync-source'                 7.09 %      ±19.35% ±27.78% ±40.79%
streams/iter-throughput-pipeto.js n=5 datasize=67108864 api='iter-sync-source'        ***     10.86 %       ±3.77%  ±5.23%  ±7.26%

Assisted-by: openai:gpt-5.5

trivikr added 2 commits May 14, 2026 20:44
Avoid normalizing sync iterable sources through from() when pipeTo()
has no transforms or signal and the writer can accept sync writes.
This keeps writes incremental while preserving async fallback for
values that still need it.

Signed-off-by: Kamat, Trivikram <[email protected]>
Assisted-by: openai:gpt-5.5
@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance
  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. labels May 15, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

❌ Patch coverage is 85.45455% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.07%. Comparing base (2edd842) to head (004031e).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/streams/iter/pull.js 85.45% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #63318      +/-   ##
==========================================
+ Coverage   90.04%   90.07%   +0.02%     
==========================================
  Files         714      714              
  Lines      225338   225388      +50     
  Branches    42598    42628      +30     
==========================================
+ Hits       202897   203009     +112     
+ Misses      14236    14155      -81     
- Partials     8205     8224      +19     
Files with missing lines Coverage Δ
lib/internal/streams/iter/pull.js 85.10% <85.45%> (-0.06%) ⬇️

... and 38 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@trivikr trivikr self-assigned this May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants