investigate flaky test-domain-error-types

Test: test-domain-error-types
Platform: ubuntu1804_sharedlibs_debug
Console Output:

not ok 655 parallel/test-domain-error-types
  ---
  duration_ms: 0.812
  severity: crashed
  exitcode: -11
  stack: |-

Build Links: https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_debug_x64/26102/consoleText

This one has been happening pretty relentlessly. It feels superfluous to add more examples, but here's one anyway.

https://ci.nodejs.org/job/node-test-commit-linux-containered/26170/nodes=ubuntu1804_sharedlibs_debug_x64/consoleText

not ok 735 parallel/test-domain-error-types
  ---
  duration_ms: 0.913
  severity: crashed
  exitcode: -11
  stack: |-
  ...

(Seems to only happen in debug builds but happens there a lot.)

The first time this appears to happen in v8 canary build 3 days ago: https://ci.nodejs.org/job/node-test-commit-node-v8/765/

That's a sub-task of https://ci.nodejs.org/job/node-update-v8-canary/1714/. Weirdly, the main task is green even though the subtask is red.

@nodejs/v8

The first time this shows up in an actual pull request CI seems to be https://ci.nodejs.org/job/node-test-commit-linux-containered/26102/ which is a CI run for #37991 which is surely unrelated. So...something in V8 that slipped in at some point?

(Or maybe some other change messed with garbage collection or something in a way that surfaces a bug that was masked previously?)

From #37970 (comment):

I'm fairly certain this is causing #38063 (frequent test-domain-error-types failures on debug build in CI).

Here are 8 clean debug CI runs on e0eb515 the commit immediately preceding this one in master.

But here it is failing on f0bf373, the commit in this PR:

https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_debug_x64/26244/

I'm going to run it a bunch more times to confirm that wasn't a fluke.

Conversation to be continued in #38063, although I'll circle back here if I find out I'm wrong and this PR isn't the cause of that issue. I'm pretty sure it is, though.

Here are 8 more CI runs to verify that f0bf373 is indeed the change where this issue starts showing up:

Now they're all green. Starting to seem like I was wasting my time bisecting because the issue was a CI issue and not an issue with the test?

Optimistically closing.

Argh, just did a resume build on node-daily-master and it showed up again.

Crashed again today. https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_debug_x64/27078/testReport/(root)/test/parallel_test_domain_error_types/

Failed again today: https://ci.nodejs.org/job/node-test-commit-linux-containered/27102/

This now has been consistently failing https://ci.nodejs.org/job/node-test-commit-arm-debug/ (now on the equinix-ubuntu1804_sharedlibs_container-arm64 containers)

test: remove test-domain-error-types flaky designation

Navigation Menu

Repository navigation

investigate flaky test-domain-error-types #38063

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

investigate flaky test-domain-error-types #38063

Description

Activity

Trott commented on Apr 6, 2021

Trott commented on Apr 6, 2021

Trott commented on Apr 6, 2021

Trott commented on Apr 6, 2021

Trott commented on Apr 6, 2021

Trott commented on Apr 7, 2021

Trott commented on Apr 7, 2021

Trott commented on Apr 7, 2021

Trott commented on Apr 7, 2021

Trott commented on Apr 7, 2021

Trott commented on May 18, 2021

mhdawson commented on May 19, 2021

joyeecheung commented on Sep 1, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions