iojs.exe randomly terminates on Windows

The iojs.exe process in question is running one of http and https servers.

The problem seems to occur if a separate iojs process is running, and is subsequently terminated, e.g. manually by pressing Ctrl+C within a Command Prompt window, then the aforementioned process would terminate as well, whether it is running in the background or in (a separate) Command Prompt window.

@alexlamsl

@alexlamsl can you give us any other info? io.js and windows versions? Memory usage?

@Fishrock123

Sorry @Fishrock123 - mistyped TAB then Enter and submitted a mostly empty comment above (=.=")

When the iojs process terminates unexpected, most of the time there isn't any output to stdin or stderr - so far only once have I see the following output:

Assertion failed: (req_.data) == (this), file c:\workspace\iojs+release\nodes\iojs-win2008r2-release-x64\src\req-wrap-inl.h, line 29

I have encountered this issue on io.js 1.7.1 and 1.8.1 - I have not tried any earlier versions.

@alexlamsl

@alexlamsl does it happen if you have any two iojs processes running and quite one, or only when one is running http?

In addition, as I first suspected this issue might have something to do with https, I ran other iojs processes to continuously hit the first process via https. It ran fine after churning through ~50GB of data in both occasions (takes about 3 hours), then only when I press Ctrl+C on the tester process would I get the original process to terminate.

I tried running the tester process on a separate PC over the network, and cannot reproduce this problem so far.

Under normal circumstances, the https process would run in the background forever, and every now and then some scheduled iojs process would run to perform some http requests to external resources (NOT to the background https process), and they are expected to exit gracefully.

I suspect there might be occasions when those schedule processes terminate in an erroneous condition, and triggers this random termination behaviour on the https process.

Running just iojs (the prompt mode?) and press Ctrl+C twice to quite does not trigger the https process to terminate.

Something else that is probably irrelevant as well - when I listen to clientError events in an attempt to debug, I get these every now and then:

{ [Error: Parse Error] bytesParsed: 0, code: 'HPE_INVALID_METHOD' }
{ _connecting: false,
  _hadError: false,
  _handle: null,
  _parent: null,
  _host: null,
  _readableState:
   { objectMode: false,
     highWaterMark: 16384,
     buffer: [],
     length: 0,
     pipes: null,
     pipesCount: 0,
     flowing: true,
     ended: false,
     endEmitted: false,
     reading: true,
     sync: false,
     needReadable: true,
     emittedReadable: false,
     readableListening: false,
     defaultEncoding: 'utf8',
     ranOut: false,
     awaitDrain: 0,
     readingMore: true,
     decoder: null,
     encoding: null,
     resumeScheduled: false },
  readable: false,
  domain: null,
  _events:
   { end: [ [Object], [Function: socketOnEnd] ],
     finish: [Function: onSocketFinish],
     _socketEnd: [Function: onSocketEnd],
     drain: [ [Function: ondrain], [Function: socketOnDrain] ],
     timeout: [Function],
     error: [Function: socketOnError],
     close: [Function: serverSocketCloseListener],
     data: [Function: socketOnData],
     connect: { [Function: g] listener: [Function] } },
  _eventsCount: 9,
  _maxListeners: undefined,
  _writableState:
   { objectMode: false,
     highWaterMark: 16384,
     needDrain: false,
     ending: false,
     ended: false,
     finished: false,
     decodeStrings: false,
     defaultEncoding: 'utf8',
     length: 0,
     writing: false,
     corked: 0,
     sync: true,
     bufferProcessing: false,
     onwrite: [Function],
     writecb: null,
     writelen: 0,
     bufferedRequest: null,
     lastBufferedRequest: null,
     pendingcb: 0,
     prefinished: false,
     errorEmitted: true },
  writable: false,
  allowHalfOpen: true,
  destroyed: true,
  bytesRead: 38,
  _bytesDispatched: 0,
  _pendingData: null,
  _pendingEncoding: '',
  server:
   { domain: null,
     _events:
      { request: [Function],
        connection: [Function: connectionListener],
        clientError: [Object] },
     _eventsCount: 3,
     _maxListeners: undefined,
     _connections: 0,
     _handle:
      { fd: -1,
        reading: false,
        owner: [Circular],
        onread: null,
        onconnection: [Function: onconnection],
        writeQueueSize: 0 },
     _usingSlaves: false,
     _slaves: [],
     _unref: false,
     allowHalfOpen: true,
     pauseOnConnect: false,
     httpAllowHalfOpen: false,
     timeout: 120000,
     _connectionKey: '6::::80' },
  _idleTimeout: -1,
  _idleNext: null,
  _idlePrev: null,
  _idleStart: 4540549,
  parser:
   { '0': [Function: parserOnHeaders],
     '1': [Function: parserOnHeadersComplete],
     '2': [Function: parserOnBody],
     '3': [Function: parserOnMessageComplete],
     _headers: [],
     _url: '',
     socket: [Circular],
     incoming: null,
     maxHeaderPairs: 2000,
     onIncoming: [Function: parserOnIncoming] },
  _paused: false,
  read: [Function],
  _consuming: true }

Memory usage is around 60~250MB, on a machine that has 64GB, running Windows Server 2012 R2

If I were to run 3 tester processes simultaneously on the same machine as the https process, quitting one of them would trigger this issue.

However, the other 2 tester processes would continue to run, albeit not being able to connect to anything obviously, as the https process has (unexpectedly) terminated.

I just tried forceful termination of iojs processes which I spawn and does not use http - does not reproduce this issue.

@Fishrock123

@Fishrock123 any other combination of tests you would like me to try?

I will try and see if I can narrow something down over the weekend - but without any console output, I find a bit hard to even give an educated guess atm.

Just a wild guess, but could you remove the node.exe hardlink and rename iojs.exe to node.exe and check if it still occurs?

I once had a similar issue on Linux with a node process backgrounded through the shell, and launching another node would terminate the first. Launching the second process with iojs did not kill it.

@silverwind

@silverwind on Windows, iojs.exe and node.exe looks like two byte-by-byte identical files (as opposed to hardlinks)

Anyway, I tried removing node.exe, then renaming iojs.exe to node.exe, then run the tests:

// leave this one running
// https server that does fs.createReadStream(...).pipe(response)
> node .


// run for ~5 mins, then Ctrl+C
// hits first server with 10 simultaneous, continuous https requests
// ~400req/s, ~20MB/s
> node test.js

Problem still persists (in that the first process sometimes terminates, and when it does it may print an assertion error).

Assertion failed: (req_.data) == (this), file c:\workspace\iojs+release\nodes\iojs-win2008r2-release-x64\src\req-wrap-inl.h, line 29

@indutny

@indutny I cherrypicked your commit, rebuilt and the server is still crashing but due to an other error HEAP[iojs.exe]: HEAP: Free Heap block 000000C5AD74CA50 modified at 000000C5AD74CA98 after it was freed

@mathiask88

@mathiask88 you were using the patch for latest libuv master, https://github.com/proxy/gist.github.com/saghul/52677dba652ab8e78051 is for the version io.js uses (#1563).

Will try figure it out today on io.js level. I think that df28a73 should work after some tweaking.

@silverwind

@silverwind Ah, ok thanks. Then I did everything right ;) However with the libuv patch this problem seems to be gone.

@saghul

I think @saghul intends to include the original fix for libuv/libuv#339 in libuv 1.5.0 (#1563 (comment)), maybe it' best to wait on that before applying more 'hacks'. 😉

And it landed: libuv/libuv@f880023.

deps: update libuv to 1.5.0

Should be fixed by 04cc03b. Closing, holler if I should reopen it.

I verified my test case again, seems to be fixed for real this time.

Verified on iojs 2.0.1. Thanks everyone!

stream_base: dispatch reqs in the stream impl

deps: update libuv to 1.5.0

@silverwind

@silverwind Nah, you didn't.

Assertion failed: (consumed_) == (false), file c:\workspace\iojs+release\nodes\w
in2008r2-release-x64\src\stream_base.h, line 203

Node 5.0.0

@polkovnikov-ph I believe you are hitting a different bug - may be file a new issue to track it?

Navigation Menu

Repository navigation

iojs.exe randomly terminates on Windows #1512

49 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

iojs.exe randomly terminates on Windows #1512

Description

Activity

Fishrock123 commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

Fishrock123 commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

alexlamsl commented on Apr 23, 2015

silverwind commented on Apr 24, 2015

alexlamsl commented on Apr 25, 2015

49 remaining items

mathiask88 commented on May 5, 2015

silverwind commented on May 5, 2015

indutny commented on May 5, 2015

mathiask88 commented on May 5, 2015

silverwind commented on May 5, 2015

silverwind commented on May 5, 2015

bnoordhuis commented on May 6, 2015

silverwind commented on May 6, 2015

alexlamsl commented on May 8, 2015

reverofevil commented on Jan 19, 2016

alexlamsl commented on Jan 19, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions