Skip to content

child process : unexpected exit code when process is out of memory #12271

Closed
Closed
child process : unexpected exit code when process is out of memory#12271
Labels
memoryIssues and PRs related to Node.js memory management or memory footprint.processIssues and PRs related to the process subsystem.wipIssues and PRs that are still a work in progress.

Description

@abenhamdine
abenhamdine
opened on Apr 7, 2017

[edit by @refack]
suggested fix #12271 (comment)
[end edit]

Version: 7.8.0
Platform: Windows 10 64
Subsystem: Child processes

I fork some child processes to do some intensive calculations.
Because I likely have some memory leaks, I pass the following arguments to the forked processes :

 execArgv: process.execArgv.concat(['--expose-gc', '--max-executable-size=192', '--max-old-space-size=256', '--max-semi-space-size=2'])

and as expected, the processes run out of memory after a given time, with the following error :

<--- Last few GCs --->

[13196:000001C7DD2B3520]   120623 ms: Mark-sweep 249.8 (275.1) -> 249.8 (275.1) MB, 367.6 / 0.2 ms  deserialize GC in old space requested
[13196:000001C7DD2B3520]   121001 ms: Mark-sweep 249.8 (275.1) -> 249.8 (275.1) MB, 376.2 / 0.4 ms  deserialize GC in old space requested
[13196:000001C7DD2B3520]   121428 ms: Mark-sweep 249.8 (275.1) -> 249.8 (275.1) MB, 426.4 / 0.4 ms  deserialize GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000001994CFA66A1 <JS Object>
    1: createContext [vm.js:~41] [pc=0000029DC91AFC50](this=00000344694A26E9 <an Object with map 0000015E4FEAACB9>,sandbox=00000164890109D1 <an Object with map 000002F68947DEB1>)
    2: createSandbox(aka createSandbox) [C:\node-projects\payroll-app\server\payrollEngine\PayrollEngine.js:~750] [pc=0000029DC91C93C4](this=000001C562102201 <null>,bul=000000F3E1CB4639 <an Object with map 000000BA66...

FATAL ERROR: deserialize context Allocation failed - process out of memory

DEBUG 17:02:28 : manageCalcProcess.onChildExit() : child process 13196 exited with code 3

What has surprised me is that the child process exits with a code 3.

Per docs, code 3 seems pretty vague, and not related to memory shortage.

I wonder if it's intentional, and if it would be more useful to have a specific exit code in that case.

Edit : Perhaps it's because of the vm.createContext() which mess the code ?

Activity

  1. abenhamdine
    changed the title [-]child process : exit code when process is out of memory[/-] [+]child process : unexpected exit code when process is out of memory[/+] on Apr 7, 2017
  2. Fishrock123
    added
    processIssues and PRs related to the process subsystem.
    questionIssues asking questions about Node.js.
    on Apr 7, 2017
  3. vsemozhetbyt
    added
    memoryIssues and PRs related to Node.js memory management or memory footprint.
    on Apr 7, 2017

  4. Trott commented on May 27, 2017

    @Trott
    Trott
    on May 27, 2017
    Member

    @bnoordhuis @cjihrig @indutny

  5. richardlau commented on May 28, 2017

    @richardlau
    richardlau
    on May 28, 2017
    Member

    Suspect this is #12823 (comment)

  6. bnoordhuis commented on May 28, 2017

    @bnoordhuis
    bnoordhuis
    on May 28, 2017
    Member

    Yes, that sounds about right. Node kills itself with a SIGABRT signal on out-of-memory conditions. On UNIX systems, you get an exit code of 128 + signal 6 = 134 but on Windows it's exit code 3, presumably due to how the CRT emulates raise(SIGABRT).

  7. gireeshpunathil commented on May 28, 2017

    @gireeshpunathil
    gireeshpunathil
    on May 28, 2017
    Member

    Agree that Windows aborts map with exit code 3 - assigned by the CRT, not node.

    At the same time, as @abenhamdine pointed out, the exit code listing does not seem to reflect this properly. Currently number 3 is defined for parse error. Either this needs to cover parse error | windows aborts, or signalled terminations in windows need to coin a different exit code to reflect the fact.

  8. gireeshpunathil commented on May 28, 2017

    @gireeshpunathil
    gireeshpunathil
    on May 28, 2017
    Member

    /cc @refack

  9. refack commented on May 28, 2017

    @refack
    refack
    on May 28, 2017
    Contributor

    MSDN hints at ways to override this:
    https://msdn.microsoft.com/en-us/library/xdkz3x12.aspx
    https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/set-abort-behavior

  10. bnoordhuis commented on May 29, 2017

    @bnoordhuis
    bnoordhuis
    on May 29, 2017
    Member

    That doesn't seem to let you override the exit code though. Perhaps the simplest fix is this:

    diff --git a/src/util.h b/src/util.h
index 175db0c..7791304 100644
--- a/src/util.h
+++ b/src/util.h
@@ -95,7 +95,7 @@ template <typename T> using remove_reference = std::remove_reference<T>;
 
 // Windows 8+ does not like abort() in Release mode
 #ifdef _WIN32
-#define ABORT_NO_BACKTRACE() raise(SIGABRT)
+#define ABORT_NO_BACKTRACE() _exit(134)
 #else
 #define ABORT_NO_BACKTRACE() abort()
 #endif

  11. gireeshpunathil commented on May 29, 2017

    @gireeshpunathil
    gireeshpunathil
    on May 29, 2017
    Member

    @bnoordhuis - does this have the risk of node silently exiting on assertion failures? no coredumps and no messages?

  12. bnoordhuis commented on May 29, 2017

    @bnoordhuis
    bnoordhuis
    on May 29, 2017
    Member

    I don't think so. The error message is printed before ABORT_NO_BACKTRACE() is invoked and core dumps aren't a thing on Windows, right?

  13. refack commented on May 29, 2017

    @refack
    refack
    on May 29, 2017
    Contributor

    I did a similar thing for @indutny for his mini-test

  14. gireeshpunathil commented on May 29, 2017

    @gireeshpunathil
    gireeshpunathil
    on May 29, 2017
    Member

    @bnoordhuis - thanks. I did not know that when node aborts in Windows, it does without a dump! Probably due to the JVM background I took it for granted that it does, just like any other OS.

    Looking at what happens in Java, I see that they trap SIGABRT within the process and invoke MiniDumpWriteDump to capture one, and then unset the trap, re-send the signal to let it die.

    That raises another question: does that mean that --abort-on-uncaught-exception is redundant in Windows? uncaught exception anyways captures the callstack, and with this flag, what is being provided additionally?

  15. 3 remaining items

  16. jkantr commented on Jun 21, 2017

    @jkantr
    jkantr
    on Jun 21, 2017
    Contributor

    I see this has been here unreferenced for awhile... and technically would be a first contribution to core for me :) Can I take it up?

  17. jkantr commented on Jun 21, 2017

    @jkantr
    jkantr
    on Jun 21, 2017
    Contributor

    @refack it would appear actually this change would mean the common test API would have to be altered, specifically nodeProcessAborted():

    https://github.com/nodejs/node/blob/76340e3f1007998c7cb9d69fa1a42d42663ca6c2/test/common/index.js#L609,L615

    As a bonus, altering nodeProcessAborted will also allow to write a proper test for this as well I believe heh. But maybe it requires some more discussion?

    I'm up for it, just making sure I'm barking down the right path :)

  18. refack commented on Jun 21, 2017

    @refack
    refack
    on Jun 21, 2017
    Contributor

    actually this change would mean the common test API would have to be altered, specifically nodeProcessAborted():

    Makes sense. Good catch.

  19. refack
    added
    wipIssues and PRs that are still a work in progress.
    and removed
    good first issueIssues that are suitable for first-time contributors.
    on Jun 21, 2017

  20. refack commented on Jun 21, 2017

    @refack
    refack
    on Jun 21, 2017
    Contributor

    @jkantr, I've flagged this as in progress indicating you "claimed it".

  21. jkantr commented on Jun 21, 2017

    @jkantr
    jkantr
    on Jun 21, 2017
    Contributor

    Sounds good, should be able to cobble something together in a few

  22. Trott
    removed
    questionIssues asking questions about Node.js.
    on Aug 7, 2017
  23. refack
    closed this as completedin #13947on Aug 8, 2017

  24. nicojs commented on Jul 31, 2018

    @nicojs
    nicojs
    on Jul 31, 2018
    Contributor

    Question: how could I detect that a child process ran out-of-memory. Should I check on the exit code (134) or should I scan the stderr output for "JavaScript heap out of memory"? Or is there a better way?

  25. jlenon7 commented on Sep 22, 2021

    @jlenon7
    jlenon7
    on Sep 22, 2021
    Member

    @nicojs same doubt here. Nico, can you please explain how did you get this exit code 134 ? When I listen to the event child.on('exit', (code) => console.log(code)) I only got the exit code 0 and signal SIGABRT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    memoryIssues and PRs related to Node.js memory management or memory footprint.processIssues and PRs related to the process subsystem.wipIssues and PRs that are still a work in progress.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions