Repository navigation
fix: use cibuildwheel's CA bundle in cross venvs - #3020
Conversation
agriyakhetarpal
left a comment
There was a problem hiding this comment.
I confirm that the Pyodide fix makes sense at least (both SSL_CERT_FILE and PIP_CERT), but would like the Android/iOS code overall to get additional eyes, if possible, as there might be context that could be missed. Thanks!
|
I was able to verify Android locally with an agent. My |
|
I have Xcode installed and can verify with iOS and my agent. Will run in a while. |
|
iOS checks out, and the reasoning seems sound. I tested with Python 3.14 and iOS 27.1 Simulator (24A94425). |
|
@henryiii here's what my agent found on your PR after it completed the iOS runner. Some good follow-ups here, I think: Still reachable after PR 3020
Covered by the PR without it saying so
Small notes
Sources: Google blog on the Chrome Root Store removal, uv advisory GHSA-hq85-5cpj-mr62, OpenJDK 8u402 release notes adding the eMudhra roots. I'm happy to take point 4 :D |
|
On 1, guessing this is for android or pyodide? iOS can't be built from Linux. :) On 2, I don't know what we should do with host. I'd rather think I'd treat it like running something before cibuildwheel runs, and leave it up to the user/host. On 3, it says we can't fix that. On 4, up to you. If someone pins older packages, we don't have to try to always fix that, but we can attempt to if you'd like. |
joerick
left a comment
There was a problem hiding this comment.
I'm getting a bit confused here... I don't believe we're actually doing anything wrong by trusting the OS's cert file here. Isn't it the user or Apple's job to fix their /etc/ssl/cert.pem rather than us?
Setting SSL_CERT_FILE to certifi might fix a security hole today, but could easily open another one in the future. And if it does, that becomes our responsibility.
|
It just consistently uses our cert file, rather than ignoring it cross compiling. When you are not cross compiling, |
|
Also, this is still only making pip use certifi, not other tools, before-build, etc. |
|
I just reread the description and I see that by doing the sys.platform overriding we hitting a peculiar edge case where tools are getting confused, inviting this behaviour. I still think that we're into the realm of patching things that aren't our responsibility here, but since the sys.platform monkeying is in our purview I can see the argument. It would be better IMO if android env tweak is only applied to
SSL_CERT_FILE is used quite widely I believe. e.g. curl will read it. And this is setting the variables when the venv is created. So I think it will affect before-build and repair (FWIW) too. |
freakboy3742
left a comment
There was a problem hiding this comment.
The iOS and Android changes generally make sense to me; the only caveat I'd flag is the same one picked up by @joerick - that the environment being modified is the build environment, which is then inherited by the test/runtime environment (android_env on Android, test_env on iOS).
That means we're overriding the certificates on the build machine; that means system certificate stores are being overridden. That won't matter for most users/builds, as the updated certifi certs will be just as accurate as the system (if not more so) - but if the user is in an environment that has a corporate VPN cert, they might not be able to run pip installs.
On Android, iOS, and Pyodide, pip runs with a patched sys.platform, so truststore uses its OpenSSL backend. On macOS hosts this reads /etc/ssl/cert.pem, which still has the compromised eMudhra root. Set PIP_CERT to our certifi bundle in these environments. Assisted-by: ClaudeCode:claude-opus-5-5
With truststore (pip 24.2+), PIP_CERT only adds certs, so the host's OpenSSL CA file was still trusted. SSL_CERT_FILE replaces it. Keep PIP_CERT for pip without truststore. Assisted-by: ClaudeCode:claude-opus-5-5
build_env has the real sys.platform, so it does not need them. Assisted-by: ClaudeCode:claude-opus-5-5
Only pip in the pyodide venv has a patched sys.platform. Assisted-by: ClaudeCode:claude-opus-5-5
5a649f4 to
7c86eeb
Compare
|
Tightened it up on Android/Pyodide. iOS I can't really tighten up without a refactor to split the environment.
I meant before-all, sorry. before-all tends to run non-pip things, before-build often is where you run pip. 馃 AI text below (could have mistakes) 馃 This table shows where
On Android, "test run" is the Gradle runner, which runs in |
ryanking13
left a comment
There was a problem hiding this comment.
I haven't verified locally but the Pyodide-side change makes sense to me.
|
Got an okay from @joerick on Discord, so going to put this in. Anyone can work around it by setting the variables like |
馃 AI text below 馃
Follow-up to #3017 and #3019. On Android, iOS, and Pyodide, pip runs in a cross venv with a patched
sys.platform. truststore chooses its backend fromplatform.system(), so it uses the OpenSSL backend instead of the macOS or Windows one. This backend reads OpenSSL's default CA file. On macOS hosts, that file is/etc/ssl/cert.pem(for python-build-standalone, and for a python.org framework build that has nocert.pem). That file still has the compromised eMudhra root (emSign ECC Root CA - G3). Apple blocks the bad intermediates through revocation in the Security framework, but this PEM file does not get those blocks.This PR sets two variables to cibuildwheel's certifi bundle, only in the environments that have a patched
sys.platform:SSL_CERT_FILE: truststore then loads this file instead of the host's CA file.PIP_CERTalone is not sufficient here, because with truststore it only adds certs. With pip 26.2.2 and aPIP_CERTthat has one unrelated CA, pip still connects to PyPI.PIP_CERT: for pip without truststore (for example, pip < 24.2 pinned by the user). This path uses requests, which ignoresSSL_CERT_FILE.Scope per platform:
android_env(before-build, build, before-test, test install).build_envhas the realsys.platform, so environment setup, repair, and the test runner do not get the variables.pyodide venv. The build environment is a normal host venv.Values set by the user take priority.
uv also reads
SSL_CERT_FILEin its default mode, so this also covers the uv frontend on Android (uv < 0.13.4 has the bad root in its bundled webpki store).The Android and iOS test projects check in
setup.pythat both variables are set. The Pyodide test checks them inbefore-test. I ran the Pyodide tests locally: they fail without the fix and pass with it. Android and iOS were verified locally in the comments below.