ci: work around Pixi samples-source lockfile oscillation by rwgk · Pull Request #3059 · NVIDIA/cuda-python

rwgk · 2026-10-09T05:00:34Z

Description

Temporarily unblock Pixi freshness and source-build CI without regenerating the
lockfiles, upgrading Pixi, or letting published wheels replace our local packages.

Why a lockfile refresh cannot fix this

Pixi 0.73.0 checks PyPI requirements that the samples environment explicitly
excludes with false-marker overrides. Its repair path then removes the local
Conda source references; a subsequent lock command adds them back. Repeated
refreshes therefore do not converge. The source-build job also fails because
--locked checks the whole workspace, including this unrelated environment.

The reduced reproducer confirms two upstream defects: override markers are not
honored during freshness traversal, and PyPI-only refreshes drop partial Conda
source records. Both reproduce on Pixi 0.81.0 and source HEAD
38ceaa89ac03649aaaefb919d1e501ca3d074d88; this is distinct from the earlier
dependency-ordering problem (Pixi #7000).

See also: Pixi #7215, with the
reduced reproducer, suggested two-part fix, regression tests, and validation logs.

Temporary workaround

Source-build CI uses PIXI_FROZEN=true, including nested Pixi commands. It
keeps the committed dependency solution and still builds the checkout's source
packages, without triggering the broken workspace-wide freshness check.
Freshness CI recognizes only Pixi 0.73.0 exiting 1 and removing exactly the
nine samples local-source references across the three platforms. Every
other lockfile byte must remain identical; manifest overrides, paths, and
source-record definitions must also match.
For PRs, both the original and repaired lockfiles must match the base results
exactly, and no pixi.toml, pyproject.toml, or Pixi-version input may change.
This exception is therefore limited to the inherited defect, not PR-induced
changes. Recognized cases emit a warning and restore the committed references.
All other stale or operational-error cases still fail. The refresh workflow
remains strict and does not open a PR for unstable lockfiles.

Validation

CI-tool tests: 126 passed, plus 36 passing subtests, including 30 new
detector cases covering exact matches, unrelated changes, and error handling.
The real freshness step checks all six workspaces successfully: five are
fresh; cuda_core matches the narrowly guarded inherited defect.
Local and remote source smoke
pass: 1,615 pathfinder tests passed, five skipped, successful bindings/core
imports, and native Cython extension compilation and placement.
Remote freshness check
passes with the expected warning and the committed lockfile restored.
Full CI matrix:
all 118 executed checks passed, with five intentional skips. One Windows
Python 3.14 / CUDA 13.4.2 / L4 MCDM test was still queued for a runner when
monitoring stopped; its result has not been verified.
Code hooks and changed-document link checks pass. An all-files local link
scan finds eight pre-existing 404s in unrelated sample READMEs.

Removing the workaround

Once a released Pixi passes repeated lock/check commands on the reproducer and
all CUDA Python workspaces without changing their bytes, remove the detector
and freshness exception and restore PIXI_LOCKED=true. Also verify that
selective PyPI updates retain local source records. No lockfiles or manifests
are changed by this PR.

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

copy-pr-bot · 2026-10-09T05:00:38Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

rwgk · 2026-10-09T05:01:01Z

/ok to test f6387f0

github-actions · 2026-10-09T05:18:25Z

Doc Preview CI
Preview removed because the pull request was closed or merged.

coderabbitai · 2026-10-09T06:34:43Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA/cuda-python/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a84f1332-4478-4088-9f3e-df5adf5001c8

📥 Commits

Reviewing files that changed from the base of the PR and between a2b1e31 and f6387f0.

📒 Files selected for processing (5)

.github/workflows/ci-pixi-lockfile-freshness-check.yml
.github/workflows/ci-pixi-source-test.yml
ci/README.md
ci/tools/check_pixi_samples_source_pruning.py
ci/tools/tests/test_check_pixi_samples_source_pruning.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Summary

Summary by CodeRabbit

Bug Fixes
- CI now handles a known Pixi 0.73.0 lockfile issue involving removed sample-source references without treating that specific case as a stale-lockfile failure. Other lockfile changes remain subject to existing checks.
- Qualifying maintenance pull requests and non-PR runs receive a warning, and the committed lockfile is restored.
Documentation
- CI guidance now describes the temporary lockfile handling and when the exception can be removed.

Walkthrough

The CI adds a detector for a specific Pixi 0.73.0 lockfile change that removes samples environment source references. The freshness workflow applies a conditional exception to matching changes, while source-build CI uses frozen mode.

Changes

Pixi Lockfile Exception

Layer / File(s)	Summary
Pruning detector and validation `ci/tools/check_pixi_samples_source_pruning.py`, `ci/tools/tests/test_check_pixi_samples_source_pruning.py`	The checker validates the expected lockfile and manifest configuration. Tests cover recognized changes, rejected inputs, and CLI error outcomes.
CI freshness handling `.github/workflows/ci-pixi-lockfile-freshness-check.yml`, `.github/workflows/ci-pixi-source-test.yml`, `ci/README.md`	The freshness workflow conditionally warns and restores the committed lockfile for an eligible match. Source-build CI sets `PIXI_FROZEN=true`. The README documents the exception and its removal criteria.

Priority: ⬇️ Low

Change: Bug fix

Merge Risk: ⚪ Minimal · up to f6387

The guarded lockfile exception preserves the base comparison, and no actionable merge-blocking issue remains after normal checks.

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

juenglin · 2026-10-09T15:44:26Z

Is there a pixi issue to watch?

Removed preview folders for the following PRs: - PR #3024 - PR #3059 - PR #3062 - PR #3063 - PR #3065 - PR #3066 - PR #3067

ci: work around Pixi samples-source lockfile oscillation

f6387f0

rwgk added this to the cuda.core next milestone Oct 9, 2026

rwgk added the CI/CD CI/CD infrastructure label Oct 9, 2026

rwgk self-assigned this Oct 9, 2026

rwgk marked this pull request as ready for review October 9, 2026 06:32

jpascucci-nv approved these changes Oct 9, 2026

View reviewed changes

jpascucci-nv merged commit df752de into NVIDIA:main Oct 9, 2026
131 checks passed

rwgk deleted the rwgk/ci-pixi-source-lock-workaround branch October 9, 2026 15:23

jpascucci-nv mentioned this pull request Oct 9, 2026

Work around pixi lockfile oscillation in lockfile refresh as well. #3062

Merged

1 of 2 tasks

juenglin reviewed Oct 9, 2026

View reviewed changes

Comment thread ci/README.md

juenglin Oct 9, 2026

Copy link
Copy Markdown

Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a pixi issue to watch?

This was referenced Oct 9, 2026

ci: temporarily pause Pixi lockfile maintenance #3065

Merged

PyPI override markers make source-package locks stale; PyPI-only refresh drops source records prefix-dev/pixi#7215

Open

github-actions Bot pushed a commit that referenced this pull request Oct 10, 2026

Clean up PR preview folders for 7 closed/merged PRs

a485805

Removed preview folders for the following PRs: - PR #3024 - PR #3059 - PR #3062 - PR #3063 - PR #3065 - PR #3066 - PR #3067

Navigation Menu

Repository navigation

ci: work around Pixi samples-source lockfile oscillation - #3059

ci: work around Pixi samples-source lockfile oscillation#3059
jpascucci-nv merged 1 commit into
NVIDIA:mainfrom
rwgk:rwgk/ci-pixi-source-lock-workaround

rwgk commented Oct 9, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Oct 9, 2026

Uh oh!

rwgk commented Oct 9, 2026

Uh oh!

github-actions Bot commented Oct 9, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 9, 2026

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

Uh oh!

juenglin Oct 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

rwgk commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Why a lockfile refresh cannot fix this

Temporary workaround

Validation

Removing the workaround

Checklist

Uh oh!

copy-pr-bot Bot commented Oct 9, 2026

Uh oh!

rwgk commented Oct 9, 2026

Uh oh!

github-actions Bot commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 9, 2026

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

Uh oh!

juenglin Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rwgk commented Oct 9, 2026 •

edited

Loading

github-actions Bot commented Oct 9, 2026 •

edited

Loading