Streams

Processes data in chunks as it arrives, instead of loading an entire file or response into memory at once.

                createReadStream(path) / createWriteStream(path) / stream.on('data', handler)
              

Purpose

A stream processes data incrementally, in chunks, as it becomes available β€” reading a large file, receiving an HTTP request body, or piping data between sources β€” without ever holding the entire data set in memory simultaneously.

Syntax

import { createReadStream, createWriteStream } from 'node:fs';

const input = createReadStream('large-file.txt');
const output = createWriteStream('copy.txt');

input.on('data', (chunk) => output.write(chunk));
input.on('end', () => output.end());
input.on('error', (err) => console.error('Stream error:', err));

// Or, more idiomatically, letting Node manage backpressure automatically:
input.pipe(output);

Behavior notes

  • A data event delivers a Buffer (raw bytes) by default, unless an encoding is set β€” decoding each chunk to text individually can split a multi-byte character across chunk boundaries, which stream.setEncoding('utf-8') handles correctly by buffering partial characters internally.
  • .pipe() connects a readable stream directly to a writable one and automatically manages backpressure β€” pausing the readable side if the writable side can’t keep up, preventing unbounded memory growth.
  • Always attach an error listener β€” an unhandled stream error can crash the process.
  • Async iteration (for await (const chunk of stream)) is also supported on a readable stream, offering a more modern alternative to manual data/end event listeners.

Common mistakes

  • Manually forwarding chunks (input.on('data', chunk => output.write(chunk))) without handling backpressure, when .pipe() would handle it automatically and more safely.
  • Omitting an error listener, letting a stream failure crash the process unhandled.

Runtime notes

Node-specific; the Web Streams API (ReadableStream/WritableStream) is the browser (and increasingly cross-runtime) equivalent, with a different but conceptually similar interface.

Taught in these lessons

Related reference