Cleanup _writableState and _readableState access across codebase

@isaacs

This is a meta-issue to keep to track of all usages of _writableState and _readableState outside of streams source. These properties are considered private and should not be used unless absolutely necessary. Usage of them can indicate a few things:

the code can be rewritten using existing documented API to achieve the same result;
streams lack some consumer functionality and new public API should be introduced;
streams lack some implementor functionality and new protected API should be introduced;
documentation needs to be added for some parts of private state for implementors;
it is an optimization that is and always be possible only in core.

The list of all _writableState and _readableState usages:

`src/node.js`

L564 stdin._readableState.reading = false. Added in commit: bb56dcc by @isaacs; (src: nix stdin _readableState.reading manipulation #454)
L573 stdin._readableState.reading = false. Added in commit: bb56dcc by @isaacs; (src: nix stdin _readableState.reading manipulation #454)

`lib/_debug_agent.js`

L87 this._readableState.objectMode = true (_debug_agent: use readableObjectMode option for client stream #270);

`lib/_http_server.js`

L348 socket._readableState.flowing = null (TODO(isaacs): Need a way to reset a stream to fresh state IE, not flowing, and not explicitly paused.). Added in 967b5db by @isaacs;
L408 var needPause = socket._writableState.needDrain. Added in 085dd30 by @isaacs ;
L445 req._readableState.resumeScheduled. Not sure where this originated, but in 2efe4ab @indutny added oldMode check here;

`lib/_tls_legacy.js`

L421 this._writableState.finished;
L508 self._readableState.length > 0;

`lib/_tls_wrap.js`

L311 self._writableState.errorEmitted (Cleanup stream state in net #465);
L313 self._writableState.errorEmitted = true (Cleanup stream state in net #465);
L350 socket._readableState.length;

`lib/child_process.js`

L1061 stream._readableState.flowing (child_process: remove redundant condition #511);

`lib/crypto.js`

whole LazyTransform thing. Is it really necessary? Maybe it should go to stream? Maybe it should be public? Maybe transforms should be lazy by default?;
L56 this._writableState.decodeStrings = false;
L57 this._writableState.defaultEncoding = 'binary';
L90 var encoding = this._readableState.encoding || 'buffer' (crypto: remove use of this._readableState #610);

`lib/fs.js`

L1624 allocNewPool(this._readableState.highWaterMark);

`lib/net.js`

`lib/zlib.js`

List of used properties:

`_readableState`

reading;
objectMode;
flowing (boolean?) is used to determine which mode readable stream is in; can be true, false or null; `null is the initial state which means that is implicitly paused;
resumeScheduled;
length;
encoding;
highWaterMark;
ended;
endEmitted;

`_writableState`

needDrain;
ended;
ending;
finished;
errorEmitted;
decodeStrings;
defaultEncoding;
length;
getBuffer();

/cc @chrisdickinson

@chrisdickinson

This is great! Thanks for compiling a comprehensive list. The most egregious overreaches, I think, are any of the places we reach in and directly modify state. Informational access is less bad.

I'd like to encourage folks to pick these off one at a time. For changes that might require a new internal API, it'd be great to briefly discuss plans here first before cutting a PR.

On Jan 15, 2015, at 8:18 AM, Vladimir Kurchatkin notifications@github.com wrote:

This is a meta-issue to keep to track of all usages of _writableState and _readableState outside of streams source. These properties are considered private and should not be used unless absolutely necessary. Usage of them can indicate a few things:

the code can be rewritten using existing documented API to achieve the same result;
streams lack some consumer functionality and new public API should be introduced;
streams lack some implementor functionality and new protected API should be introduced;
documentation needs to be added for some parts of private state for implementors;
it is an optimization that is and always be possible only in core.
The list of all _writableState and _readableState usages:

src/node.js

L564 stdin._readableState.reading = false;
L573 stdin._readableState.reading = false;
lib/_debug_agent.js

L87 this._readableState.objectMode = true (#270);
lib/_http_server.js

L348 socket._readableState.flowing = null (TODO(isaacs): Need a way to reset a stream to fresh state IE, not flowing, and not explicitly paused.);
L408 var needPause = socket._writableState.needDrain;
L445 req._readableState.resumeScheduled;
lib/_tls_legacy.js

L421 his._writableState.finished;
L508 self._readableState.length > 0;
lib/_tls_wrap.js

L311 self._writableState.errorEmitted;
L313 self._writableState.errorEmitted = true;
L350 socket._readableState.length;
lib/child_process.js

L1061 stream._readableState.flowing;
lib/crypto.js

whole LazyTransform thing. Is it really necessary? Maybe it should go to stream? Maybe it should be public? Maybe transforms should be lazy by default?;
L56 this._writableState.decodeStrings = false;
L57 this._writableState.defaultEncoding = 'binary';
L90 var encoding = this._readableState.encoding || 'buffer';
lib/fs.js

L1624 allocNewPool(this._readableState.highWaterMark);
lib/net.js

L162 this._writableState.decodeStrings = false;
L174 this._readableState.flowing = false;
L196 this._readableState.ended;
L226 self._readableState.ended;
L242 this._readableState.ended = true (comment: ended should already be true, since this is called after the EOF errno and onread has eof'ed);
L243 this._readableState.endEmitted;
L362 this._writableState.length;
L392 this._readableState.endEmitted;
L405 socket._writableState.length;
L415 if (this._writableState.finished);
L429 self._writableState.errorEmitted;
L433 self._writableState.errorEmitted = true;
L535 self._readableState.length === 0;
L715 state.getBuffer();
L842 this._readableState.reading = false;
L843 this._readableState.ended = false;
L844 this._readableState.endEmitted = false;
L845 this._writableState.ended = false;
L846 this._writableState.ending = false;
L847 this._writableState.finished = false;
L848 this._writableState.errorEmitted = false;
/cc @chrisdickinson

—
Reply to this email directly or view it on GitHub.

_debug_agent: use `readableObjectMode` option

Of note: I'm not opposed to all non-streams access to _readableState, just accesses that are directly inspecting a property or manipulating state. APIs private to io.js itself are okay to hang off of ReadableState, but I'd like to make sure that there's a clear API there, and not just random attribute access/manipulation.

We should also probably inspect userland modules to find out what people are doing with internals.

As a heads up, I edited your issue comment to link #454 for the node.js _readableState use (sorry sorry). In the future, I can ping you instead if you'd like to keep the list maintained!

@chrisdickinson

@chrisdickinson fine by me. But think it's a good idea to reference this issue so we can see the timeline of all relevant stuff here

We might add lib/zlib.js:

L423: ws.ended
L426: ws.ending
L429: ws.needDrain
L460: ws.ending || ws.ended
L461: ws.length
L479: ws.length

crypto: remove use of this._readableState

streams: make streams lazy by default

Cross posting this from nodejs/readable-stream#109:

To amend the intent of this issue a bit – the goal is not to absolutely remove all use of _{{writ,read}able,transform}State, but to remove all places where:

subclasses directly manipulate private state, or...
state is being preserved in the base classes for the exclusive use of a subclass.

Ultimately, the places where the subclasses access those state objects should be catalogued and we should look into the best way to deal with that access – whether that's promotion into a public or protected "reflection" API for subclasses.

This issue hasn't seen any activity in almost a year. Is it still active and useful? Or not so much?

net: remove _readableState from debug statement

(The stalled bot doesn't appear to be working, so I've manually closed. Feel free to reopen if this is no longer stalled)

Navigation Menu

Repository navigation

Cleanup `_writableState` and `_readableState` access across codebase #445

`src/node.js`

`lib/_debug_agent.js`

`lib/_http_server.js`

`lib/_tls_legacy.js`

`lib/_tls_wrap.js`

`lib/child_process.js`

`lib/crypto.js`

`lib/fs.js`

`lib/net.js`

`lib/zlib.js`

`_readableState`

`_writableState`

168 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Cleanup _writableState and _readableState access across codebase #445

Description

src/node.js

lib/_debug_agent.js

lib/_http_server.js

lib/_tls_legacy.js

lib/_tls_wrap.js

lib/child_process.js

lib/crypto.js

lib/fs.js

lib/net.js

lib/zlib.js

_readableState

_writableState

Activity

chrisdickinson commented on Jan 15, 2015

chrisdickinson commented on Jan 15, 2015

vkurchatkin commented on Jan 15, 2015

chrisdickinson commented on Jan 15, 2015

vkurchatkin commented on Jan 15, 2015

chrisdickinson commented on Jan 15, 2015

chrisdickinson commented on Jan 30, 2015

Trott commented on Jan 19, 2016

168 remaining items

lubas569 commented on Dec 19, 2022

avivkeller commented on Sep 24, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Cleanup `_writableState` and `_readableState` access across codebase #445

`src/node.js`

`lib/_debug_agent.js`

`lib/_http_server.js`

`lib/_tls_legacy.js`

`lib/_tls_wrap.js`

`lib/child_process.js`

`lib/crypto.js`

`lib/fs.js`

`lib/net.js`

`lib/zlib.js`

`_readableState`

`_writableState`