Skip to content

Can no longer validate signatures on NodeJS binaries due to needing approval on keyserver #58904

Description

@the-gabe

Version

No response

Platform


Subsystem

No response

What steps will reproduce the bug?

gpg --keyserver hkps://keys.openpgp.org --recv-keys C0D6248439F1D5604AAFFB4021D900FFDB233756 # Antoine du Hamel now fails with key 21D900FFDB233756: new key but contains no user ID - skipped

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

The key is approved on the keyserver

What do you see instead?

broken

Additional information

No response

Activity

  1. the-gabe commented on Jun 30, 2025

    @the-gabe
    Author

    Hey @aduh95 I guess you might be able to resolve this by doing a verification email for your signing key at openpgp.org. Thanks!

    Mitigating currently by using the Ubuntu keyserver instead.

  2. aduh95 commented on Jun 30, 2025

    @aduh95
    Contributor

    Not really, it seems that OpenPGP doesn't support having two keys associated with the same email address – which makes sense if you're trying to use it to send me an encrypted message, but is unfortunate if you only want to verify my past signatures.

  3. the-gabe commented on Jun 30, 2025

    @the-gabe
    Author

    That's unfortunate. It would be nice to have an alternate suggested means of key obtainment documented within the README for this scenario.

  4. aduh95 commented on Jun 30, 2025

    @aduh95
    Contributor

    I definitely agree, I've suggested exactly that in #58877 (comment). I'm sorry for the inconvenience

  5. ruant commented on Jun 30, 2025

    @ruant

    A workaround that works for me at least is:

    curl https://github.com/raw/nodejs/release-keys/HEAD/keys/C0D6248439F1D5604AAFFB4021D900FFDB233756.asc | gpg --import

    instead of relaying on the opengpg keyserver.

    Out of curiosity, why oh why doesn't the nodejs organization have it's own key that does all the signings?
    It's a complete mess with all these "private" gpg keys belonging to maintainers that come and go.

  6. richardlau commented on Jun 30, 2025

    @richardlau
    Member

    Out of curiosity, why oh why doesn't the nodejs organization have it's own key that does all the signings? It's a complete mess with all these "private" gpg keys belonging to maintainers that come and go.

    Because then you have to manage access to the shared key. You'd either have to allow the private part of the key to exist on multiple machines (belonging to the releasers) or on a shared machine (whereby if it was compromised the attacker would be able to sign releases). With the current set up, the private part of the signing key is only known by the releaser it belongs to. That also gives some accountability -- I cannot sign releases with another releaser's key (or vice versa).

    We've had releasers lose their laptops which meant we only needed to invalidate one key -- in the setup with a single signing key we'd need to revoke the shared key in those cases.

  7. ruant commented on Jun 30, 2025

    @ruant
  8. MikeMcC399 commented on Jul 1, 2025

    @MikeMcC399
    Contributor

    it seems that OpenPGP doesn't support having two keys associated with the same email address – which makes sense if you're trying to use it to send me an encrypted message, but is unfortunate if you only want to verify my past signatures.

    Confirmed in the hkps://keys.openpgp.orgs FAQ Can I verify more than one key for some email address?

    An email address can only be associated with a single key. When an address is verified for a new key, it will no longer appear in any key for which it was previously verified. Non-identity information will still be distributed for all keys.

    @aduh95

    Do you know if this also applies to keyserver.ubuntu.com? If the Ubuntu keyserver is more tolerant and would continue to allow importing the original key C0D6248439F1D5604AAFFB4021D900FFDB233756, then the following step used in https://github.com/nodejs/docker-node/blob/main/Dockerfile-debian.template could be re-ordered to prefer keyserver.ubuntu.com

    Currently it's like this, and we're using this in Cypress Docker image Dockerfiles as well:

          gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys "$key" || \
          gpg --batch --keyserver keyserver.ubuntu.com --recv-keys "$key" ; \

    This is a problem for the Cypress Docker image build process, because they are not only built when a new Node.js is released.

    If the command gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys C0D6248439F1D5604AAFFB4021D900FFDB233756 is executed, a skip message is output, no key is imported, and the return code is success 0, so keyserver.ubuntu.com is never used.

    Reversing the order in the above sequence, using keyserver.ubuntu.com before hkps://keys.openpgp.org appears to work. I'm not sure if there are any "gotchas" for the change, but I'm going to propose it in the Cypress Docker repo, as otherwise release of a new Cypress Docker image for the planned new Cypress release today will be blocked.

  9. MikeMcC399 commented on Jul 1, 2025

    @MikeMcC399
  10. MikeMcC399 commented on Jul 1, 2025

    @MikeMcC399
    Contributor

    From https://github.com/nodejs/node/blob/main/README.md#release-keys

    Juan José Arboleda soyjuanarbol@gmail.com DD792F5973C6DE52C432CBDAC77ABFA00DDBF2B7

    replaced the previous key

    Juan José Arboleda soyjuanarbol@gmail.com 61FC681DFB92A079F1685E77973F295594EC4689

    The key 61FC681DFB92A079F1685E77973F295594EC4689 was then no longer served from hkps://keys.openpgp.org

    14:09:04  + gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys 61FC681DFB92A079F1685E77973F295594EC4689
    14:09:05  gpg: key 973F295594EC4689: new key but contains no user ID - skipped
    14:09:05  gpg: Total number processed: 1
    14:09:05  gpg:           w/o user IDs: 1
    

    preventing verification of Node.js 18.12.1 released by Juan José Arboleda on Nov 04, 2022.

    This indicates that it is a recurring problem, even though the frequency is low.

  11. aduh95 commented on Jul 1, 2025

    @aduh95
    Contributor

    Do you know if this also applies to keyserver.ubuntu.com?

    I don't know, I've read in https://superuser.com/a/1485255 that it wasn't the case, but also, I'm not too keen on testing that hypothesis for now. I haven't uploaded my new key there for now.

    The README needs to have working instructions. Possibly Node.js should prefer using keyserver.ubuntu.com instead of hkps://keys.openpgp.org?

    See #58877 (comment), IMO the solution is not to pick a different keyserver, it should be to recommend using a solution the project has control over (i.e. nodejs/release-keys).

  12. MikeMcC399 commented on Jul 2, 2025

    @MikeMcC399
    Contributor

    @aduh95

    It would be good to see PR #58877 merged, so at least the non-working line in the README instructions section Release keys referring to the key C0D6248439F1D5604AAFFB4021D900FFDB233756 is replaced. The displayed instruction no longer works, and there is no expectation that this key on the hkps://keys.openpgp.org keyserver can be reactivated:

    gpg --keyserver hkps://keys.openpgp.org --recv-keys C0D6248439F1D5604AAFFB4021D900FFDB233756 # Antoine du Hamel
  13. aduh95 commented on Jul 2, 2025

    @aduh95
    Contributor

    I'm not sure what good would that do, until I am to sign another release, it doesn't change anything what key is listed for me.

    The displayed instruction no longer works

    gpg still exits with exit code 0, so I doubt that this is breaking anyone – what's broken is when one tries to verify a release signed with C0D6248439F1D5604AAFFB4021D900FFDB233756, which is not going to change by landing #58877.

  14. MikeMcC399 commented on Jul 2, 2025

    @MikeMcC399
    Contributor

    @aduh95

    gpg still exits with exit code 0, so I doubt that this is breaking anyone – what's broken is when one tries to verify a release signed with C0D6248439F1D5604AAFFB4021D900FFDB233756, which is not going to change by landing #58877.

    This actually breaks any usage of the Docker image cypress/factory for versions >= 4.3.0 and < 5.11.2 attempting to build a custom Cypress Docker image including any Node.js version signed with this key, such as Node.js 22.17.0, the Active LTS version at this time. The cypress/factory build process attempts to verify Node.js releases as part of the build process, and fails the build if the verification fails.

    A workaround was implemented in cypress/factory:5.11.2 to first try importing from the keyserver keyserver.ubuntu.com and falling back on hkps://keys.openpgp.org if that fails. That fix was released yesterday.

    Admittedly, updating the README would not change the above, as the key usage is hard-coded into the cypress/factory Docker image. This is more about having the cypress/factory Docker image build process aligned with working and official Node.js recommendations.

  15. 4 remaining items

  16. MikeMcC399 commented on Jul 4, 2025

    @MikeMcC399
  17. pas256 commented on Jul 5, 2025

    @pas256

    That solution in nodejs/docker-node#2252 is really slick.

  18. MikeMcC399 commented on Jul 7, 2025

    @MikeMcC399
    Contributor

    @pas256

    That solution in nodejs/docker-node#2252 is really slick.

    Yes, that's a great way to reliably import all keys programmatically!

    I also went through all the keys and have made a suggestion in #58979 about continuing to use keyservers and especially documenting how to import keys in the category: "Other keys used to sign some previous releases"

    I could only find one key that couldn't be imported from either hkps://keys.openpgp.org or keyserver.ubuntu.com and that was 1C050899334244A8AF75E53792EF661D867B9DFA which nodejs/release-keys@5457e01 comments as "Add old key for Danielle Adams (used to sign v15.2.0 release)" - relating to the non-LTS 15.x release line that has been unsupported since June 2021.

  19. MikeMcC399 commented on Jul 8, 2025

    @MikeMcC399
    Contributor

    The solution in nodejs/docker-node#2252 has now been merged, so I posted a script to #58979 (comment) based on this, which imports all keys listed on Release keys (present and past). There are currently 27 of them.

  20. del-leehopper commented on Jul 8, 2025

    @del-leehopper

    Can someone please advise how I can verify the signature of the SHASUMS256.txt for v22.17.0? As you probably guessed, I am getting the "Bad Signature" error. I tried downloading the public keys from keyserver.ubuntu.com also to no avail.

    I have read through this thread, but I am none the wiser.

    Thanks.

  21. aduh95 commented on Jul 8, 2025

    @aduh95
    Contributor

    @del-leehopper here's bash script that downloads the linux tarball and verifies the signature:

    set -exo pipefail
    
    VERSION=v22.17.0
    TAR="node-$VERSION-linux-x64.tar.xz"
    
    PUBRING_REV=3ead0a2d07b13e469fd97ef39facf6d31993fb71
    PUBRING_SHASUM=4f8664db3ba7311589efe3697881155f8ba4bb5d36fe84bdfa7e77359408b1bf
    
    PUBRING=$(mktemp)
    curl -fsLo "$PUBRING" "https://github.com/nodejs/release-keys/raw/$PUBRING_REV/gpg-only-active-keys/pubring.kbx"
    shasum -a 256 "$PUBRING" | grep -qF "$PUBRING_SHASUM"
    
    curl -fso "$TAR" "https://nodejs.org/dist/$VERSION/$TAR"
    
    curl -fs "https://nodejs.org/dist/${VERSION}/SHASUMS256.txt.asc" \
    | gpgv --keyring="${PUBRING}" --output - \
    | grep -F "$TAR" \
    | shasum -c -
    
    rm "$PUBRING"
  22. MikeMcC399 commented on Jul 9, 2025

    @MikeMcC399
    Contributor

    @del-leehopper

    Using the instructions from the README > Verifying binaries, except importing the signature from keyserver.ubuntu.com, the following worked for me:

    docker run -it --rm debian

    at bash prompt

    apt-get update && apt-get install -y gnupg ca-certificates curl
    curl -O https://nodejs.org/dist/v22.17.0/SHASUMS256.txt
    curl -O https://nodejs.org/dist/v22.17.0/SHASUMS256.txt.sig
    gpg --keyserver keyserver.ubuntu.com --recv-keys C0D6248439F1D5604AAFFB4021D900FFDB233756
    gpg --verify SHASUMS256.txt.sig SHASUMS256.txt
    /# gpg --verify SHASUMS256.txt.sig SHASUMS256.txt
    gpg: Signature made Wed Jun 25 00:12:40 2025 UTC
    gpg:                using RSA key C0D6248439F1D5604AAFFB4021D900FFDB233756
    gpg: Good signature from "Antoine du Hamel <duhamelantoine1995@gmail.com>" [unknown]
    gpg: WARNING: This key is not certified with a trusted signature!
    gpg:          There is no indication that the signature belongs to the owner.
    Primary key fingerprint: C0D6 2484 39F1 D560 4AAF  FB40 21D9 00FF DB23 3756
    

    If it's not working for you, then I suggest you post the instructions you used and your logs.

  23. aduh95 commented on Jul 18, 2025

    @aduh95
    Contributor

    I've opened #59113 to hopefully address this once and for all, so whatever releasers do with public servers do not affect the project.

  24. MikeMcC399 commented on Jul 19, 2025

    @MikeMcC399
    Contributor

    @aduh95

    I've opened #59113 to hopefully address this once and for all, so whatever releasers do with public servers do not affect the project.

    Any change in recommended process which avoids having to consider and manage individual Node.js release keys in downstream repos is very welcome!

    If you have a drop-in migration replacement for the script used in https://github.com/nodejs/docker-node/blob/main/Dockerfile-debian.template, which currently uses keyservers, that would be very much appreciated. We base our Node.js installation script used in the cypress/factory Docker image on the process used in nodejs/docker-node as a kind of implicit guidance.

    This may be off-topic for this issue. In which case I would wait for your #59113 PR to be approved and merged and then raise it as a separate issue in nodejs/docker-node.

    Edit: nodejs/docker-node#2265 opened for the related topic of upgrading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions