Skip to content

test: test-crypto-keygen.js should be split into multiple tests to avoid timeout in CI #49202

Closed
Closed
test: test-crypto-keygen.js should be split into multiple tests to avoid timeout in CI#49202
Labels
cryptoIssues and PRs related to the crypto subsystem.testIssues and PRs related to Node.js core tests and test infrastructure.

Description

@joyeecheung
joyeecheung
opened on Aug 16, 2023

It looks like #41206 is coming back to the CI, see https://ci.nodejs.org/job/node-test-binary-windows-js-suites/22596/RUN_SUBSET=2,nodes=win2012r2-COMPILED_BY-vs2019-x86/testReport/junit/(root)/parallel/test_crypto_keygen_/ It's a test with 1800+ lines that is crammed with computation-intensive crypto tests that seem to be logically separable. On my local machine which is quite powerful it still takes >5s to run. It would be good to split it into multiple tests to avoid timing out in the CI (I could do it if no one picks this up, but I feel like maybe people from @nodejs/crypto have better ideas about how to split them properly and give the split tests proper names - if I am doing it I would probably just name them test-crypto-keygen-1.js, test-crypto-keygen-2.js..)

Activity

  1. joyeecheung
    added
    good first issueIssues that are suitable for first-time contributors.
    on Aug 16, 2023

  2. pluris commented on Aug 17, 2023

    @pluris
    pluris
    on Aug 17, 2023
    Contributor

    If I want to reproduce it, can I run it with the command below?

    python.exe tools/test.py -J --repeat=1000 test/parallel/test-crypto-keygen.js

  3. joyeecheung commented on Aug 17, 2023

    @joyeecheung
    joyeecheung
    on Aug 17, 2023
    MemberAuthor

    You can just run it with out/Release/node test/parallel/test-crypto-keygen.js and see how long it takes and try to split it to make the split tests run faster, the time out only reproduces on very slow machines, running it many times would not make a difference if you run it on a fast machine.

  4. joyeecheung commented on Aug 17, 2023

    @joyeecheung
    joyeecheung
    on Aug 17, 2023
    MemberAuthor

    https://ci.nodejs.org/job/node-test-binary-windows-js-suites/22633/RUN_SUBSET=2,nodes=win2016-COMPILED_BY-vs2022-x86/console

    And test-crypto-dh.js too (which crams 19 crypto.createDiffieHellman() calls in one test

  5. joyeecheung commented on Aug 17, 2023

    @joyeecheung
    joyeecheung
    on Aug 17, 2023
    MemberAuthor

    I am going to do it to deflake the CI sooner than later.

  6. joyeecheung
    removed
    good first issueIssues that are suitable for first-time contributors.
    on Aug 17, 2023

  7. pluris commented on Aug 17, 2023

    @pluris
    pluris
    on Aug 17, 2023
    Contributor

    I was checking to give it a try, but you've already fixed it. That's a good job. 👍

  8. joyeecheung commented on Aug 18, 2023

    @joyeecheung
    joyeecheung
    on Aug 18, 2023
    MemberAuthor
    ok 57 parallel/test-crypto-secure-heap
  ---
  duration_ms: 1981.18900
  ...
ok 58 parallel/test-crypto-dh-constructor
  ---
  duration_ms: 4528.98300
  ...
ok 59 parallel/test-crypto-classes
  ---
  duration_ms: 5376.81000
  ...
ok 60 parallel/test-crypto-dh
  ---
  duration_ms: 9867.30400
  ...
ok 61 parallel/test-crypto-keygen
  ---
  duration_ms: 9152.01800

    Here is a list of crypto tests that took more than 1s for me locally (probably would be much slower in the CI on slow machines, in the failures referenced above, over 2 minutes)

  9. VoltrexKeyva
    added
    testIssues and PRs related to Node.js core tests and test infrastructure.
    cryptoIssues and PRs related to the crypto subsystem.
    on Aug 22, 2023
  10. nodejs-github-bot
    added 3 commits that reference this issue on Aug 31, 2023

    test: split test-crypto-keygen.js

    7224940

    test: split JWK async elliptic curve keygen tests

    b9a6ab5

    test: reduce length in crypto keygen tests

    b781eaf
  11. nodejs-github-bot
    added a commit that references this issue on Sep 7, 2023

    test: split test-crypto-dh to avoid timeout on slow machines in the CI

    e0fb3f7
  12. UlisesGascon
    added a commit that references this issue on Sep 10, 2023

    test: split test-crypto-keygen.js

    e04a260

  13. 7 remaining items

  14. targos
    added a commit that references this issue on Nov 15, 2023

    test: split test-crypto-dh to avoid timeout on slow machines in the CI

    65b41eb
  15. targos
    added 3 commits that reference this issue on Nov 27, 2023

    test: split test-crypto-keygen.js

    344592d

    test: split JWK async elliptic curve keygen tests

    958e114

    test: reduce length in crypto keygen tests

    67cb6f9

  16. shnooshnoo commented on Nov 28, 2023

    @shnooshnoo
    shnooshnoo
    on Nov 28, 2023

    hi @joyeecheung seems like this was fixed and can be closed now, can it?

  17. joyeecheung commented on Nov 28, 2023

    @joyeecheung
    joyeecheung
    on Nov 28, 2023
    MemberAuthor

    I think only the two largest tests from #49202 (comment) have been split, but then I think so far I haven't seen the other less large ones timing out in the CI, so maybe it's good for now. We can split some more if any of them time out again.

  18. joyeecheung
    closed this as completedon Nov 28, 2023
  19. sercher
    added 8 commits that reference this issue on Apr 25, 2024

    test: split test-crypto-dh to avoid timeout on slow machines

    134d45b

    test: split test-crypto-keygen.js

    6ebbd6c

    test: split JWK async elliptic curve keygen tests

    2aa755e

    test: reduce length in crypto keygen tests

    dc50f7f

    test: split test-crypto-dh to avoid timeout on slow machines

    92eabdb

    test: split test-crypto-keygen.js

    580db80

    test: split JWK async elliptic curve keygen tests

    2447b77

    test: reduce length in crypto keygen tests

    2bfb541
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cryptoIssues and PRs related to the crypto subsystem.testIssues and PRs related to Node.js core tests and test infrastructure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions