Repository navigation
launching python subprocess can fail: rules_python==1.7.0 + bootstrap_impl=system_python 聽#3437
Description
馃悶 bug report
Affected Rule
py_binary() / py_test() that invokes python as subprocess. When using --@rules_python//python/config_settings:bootstrap_impl=system_python (default)
--@rules_python//python/config_settings:bootstrap_impl=script does not reproduce this issue.
Is this a regression?
Did this behavior use to work in the previous version? Yes
Description
I have had python executables where they launched a python subprocess as part of them. A basic one for example looked like
# Python code
import sys
# do some preprocess
subprocess.check_output([sys.executable, "-m", "datamodel_code_generator", ...
# do some post process# BUILD code
py_binary(
name = "pydantic_generator",
deps = ["@pypi//:datamodel_code_generator", ...]
)With rules_python==1.7.0 I get an error that no module named datamodel_code_generator found.
馃敩 Minimal Reproduction
https://github.com/shayanhoshyari/issue-reports/tree/main/rules_python/subprocess_call
馃敟 Exception or Error
/private/var/tmp/_bazel_hoshyari/5bf493163e144814d5d8244a8cb0233d/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/test.runfiles/rules_python++python+python_3_13_aarch64-apple-darwin/bin/python3: No module named datamodel_code_generator
馃實 Your Environment
Operating System:
Reproduces on both ubuntu (tried 22.04) MacOS (tried Sonoma)
Output of bazel version:
bazel 8.2.1
Rules_python version:
1.7.0
Anything else relevant?
Seems this is due to b8e32c4
1.6.3and older usedPYTHONPATHso it propagated in subprocess calls.1.7.0starts using the combination- bootstrap stage 1 that calls
- bootstrap stage 2 that adds
iiitosite.addsitepackage, then runs main entrypiont withrunpy. - siteinit.py that adds the new imports.
The issue is that adding folders via site.addsitepackage do not propagate via subprocess.call. So when the main script launches a new python, it is not aware of ii and hence iii.
bootstrap_impl=script does not have this issue as sys.executable resolves to a different python (test.venv/bin/python) that always is aware of iii even when ii is out of picture.
(I suspect if we launch with -S even bootstrap_impl=script will stop working)
Fix ideas:
- use same venv python for
system_pythontoo - use sitecustomize + PYTHONPATH in bootstrap stage 1 instead of
site.addsitepackagein bootstrap stage 2. This should propagate.
Activity
[-]`rules_python>==1.7.0` launching python subprocess is not picking up imports[/-][+]`rules_python>==1.7.0` + `bootstrap_impl=system_python` launching python subprocess can fail [/+][-]`rules_python>==1.7.0` + `bootstrap_impl=system_python` launching python subprocess can fail [/-][+]launching python subprocess can fail: `rules_python==1.7.0` + `bootstrap_impl=system_python` [/+]shayanhoshyari commentedon Dec 1, 2025
Seems #3422 is relevant. But it might not be the same issue, as this is not strictly about adding the runfiles root.
rickeylev commentedon Dec 19, 2025
With the two bootstrap sharing more code now, this is pretty feasible now. The system_python bootstrap is basically just a stage1 bootstrap, just implemented in python instead of bash.
Hm, actually, looking at the python_bootstrap_template.txt, it seems the code for that is already there? Ah, I see -- L516 in py_executable.bzl skips creating a full venv for bootstrap=system_python. Looks like it just creates the site-packages parts. I think there was an obscure CI failure I couldn't figure out? In any case, that's probably the line that needs to be changed.
I had considered this, but was concerned that it'd collide with a user's sitecustomize (e.g. one coming from a docker image that a py_binary gets installed into).
To some extent, yes, this is true. However, in stage2, the site.addsitepackage() call is invoked if the venv's site-package directory doesn't appear to be on sys.path, which works around the issue. This won't help direct invocations of sys.executable, though.
Also, a non-bazel venv will break with -S, too; venvs rely on the "find the site-packages directory" logic that having the "site" functionality enabled triggers.
subproccessModule Can't Import Deps #3517 as a duplicate of this issueluispadron commentedon Jan 21, 2026
+1 though I'm seeing this just by updating to Bazel 9 and not making any changes to my rules_python version. I can repro on both Bazel 8 and 9 if i use rules_python 1.7+
kocolosk commentedon Apr 16, 2026
Any further thoughts on a preferred workaround here? We just bumped into this when attempting to upgrade from rules_python 1.6.3 as well.
shayanhoshyari commentedon Apr 17, 2026
@kocolosk d0b9baa should address this. Seems released on version 2: https://rules-python.readthedocs.io/en/latest/changelog.html
It uses the solution @rickeylev mentioned above
shayanhoshyari commentedon Apr 17, 2026
@rickeylev double checking, can this issue be closed now?