Skip to content

Commit 22b3472

Browse files
committed
feat: address conda#896 corrupt-cache failure mode
Builds on the mamba 2.6.0 cache-layout fix to also defend against the corrupt ``repodata_record.json`` entries that mamba/micromamba 2.1.1-2.3.3 wrote to the package cache (mamba-org/mamba#4052, fixed upstream in mamba-org/mamba#4110 / mamba 2.6.0). The 2.3.3 partial fix recovered ``depends`` and ``constrains`` but still wrote ``timestamp: 0`` and ``license: ""`` -- enough to trip our detector. Mitigations: 1. *Corruption detection + heal in disk fallback.* ``_is_mamba_2x_corrupt_record`` matches the ``timestamp == 0`` + empty ``license`` signature. ``_heal_corrupt_record`` overlays the record with ``info/index.json`` (extracted at install time, never affected by the bug). WARNING per heal. 2. *Rich-LINK corruption guard.* The ``_link_action_as_fetch`` fast path bypasses ``_get_repodata_record`` and therefore the corruption detector. We re-check the corruption signature in the fast path so a LINK whose metadata carries the bug routes to disk fallback (where ``_heal_corrupt_record`` can recover from ``info/index.json``) rather than synthesizing a corrupt FETCH directly. Mamba 2.6.0+ is supposed to heal cache records before emitting them in LINK; we no longer depend on that external invariant. 3. *pkgs_dirs leak detection.* Reports paths that mamba/conda merged in from condarc despite the env-var override. 4. *Degraded-path warning.* Single WARNING per platform when ``_reconstruct_fetch_actions`` takes the disk-fallback branch. 5. *Reverse-propagation of categories from .dependencies.* Solver auto-installs (pip, wheel) aren't transitive deps of anything requested. Reverse-prop: an orphan inherits the union of categories of the packages it depends on. ``_truncate_main_category`` runs after reverse-prop so a package inheriting both ``main`` and ``dev`` collapses to ``{main}`` -- v1 serialization doesn't emit the package twice. 6. *Hard-fail on residual orphans.* Anything still without a category after reverse-prop is the corruption-induced silent-vanishing failure mode. Raises ``OrphanLockedDependencyError`` rather than silently drop. 7. *Lockfile-side heal of carry-forward corruption (per-entry).* ``_heal_locked_dependencies_from_cache`` runs in ``update_specs_for_arch`` *before* ``fake_conda_environment`` and ``to_fetch_action()``. The cache is a *per-entry* discriminator: - Cache says non-empty depends -> heal in place. The contradiction (cache vs lockfile) is per-entry proof of corruption for that entry. - Cache says empty depends -> legit-empty, leave alone. - Cache has no entry -> ambiguous. Healable-elsewhere is NOT evidence about ambiguous-here: partial caches are normal (a user may have ``libzlib`` cached but not ``tzdata``). We WARN with prescriptive text and let the orphan check downstream cover the silent-vanishing variant. We do NOT hard-fail on the ``healed > 0 + ambiguous`` mixed case -- that predicate false-positived on the ordinary scenario where one corrupt entry is healable while a legit-empty leaf is simply not yet in the cache. The warning is honest about a residual limitation: a corrupt entry that is itself a requested or otherwise-reachable root may carry forward unflagged when its missing transitive deps happen to be reachable via other paths in the graph. The repair guidance is to regenerate the lockfile from sources on a known-clean cache; ``conda-lock install`` is demoted to a diagnostic-only note for investigating carry-forward metadata, with an explicit warning not to install a corrupt lockfile into a real environment (any packages already vanished during v1 serialization are unrecoverable). End-to-end test backed by the PR conda#862 reproduction harness: ``test_pr862_corrupt_2_1_1_cache_does_not_drop_packages`` walks the full committed reproduction pipeline minus Docker, parametrized over both ``[mamba]`` and ``[conda-standalone]``. It warms a real cache, overlays the corrupt ``info/{index,repodata_record}.json`` from ``2.1.1-pkgs.tar.gz``, runs ``conda-lock lock`` against the hybrid cache, renders the explicit lockfile (stage 05), and asserts structural lockfile equivalence + explicit-URL equivalence + empty- deps headroom against a clean control. Plus unit tests for: corruption signature detection, heal-from-index, rich-LINK corruption guard, heal-in-place + WARN on ambiguous (includes the explicit healed-libzlib + legit-empty-tzdata-leaf scenario the predicate now correctly handles), legit-empty-deps acceptance, partial-cache-of-legit-empty does NOT raise, pkgs_dirs leak warning, degraded-path warning, mixed-root reverse propagation with main truncation, hard-fail on unrecoverable orphans, auto-install orphan inheritance. To make the conda-standalone parametrization exercise both CI and local ``pixi run pytest``, ``conda-standalone`` is added to: - ``environments/dev-environment.yaml`` (CI source of truth) - ``environments/conda-lock-python-3.10.yaml``, ``environments/conda-lock-python-3.14.yaml`` (relocked) - ``pixi.toml`` (local dev source of truth) ``pixi.lock`` will need a one-time ``pixi lock`` to refresh the solved package set; this could not be done in the sandbox where the change was developed. CI is unaffected -- it consumes the conda-lock environment files above, not pixi. The PR conda#862 stage scripts are excluded from mypy/ty/pre-commit -- they are Docker-based one-shot data generators, not runtime code.
1 parent c2f92f6 commit 22b3472

38 files changed

Lines changed: 24693 additions & 7000 deletions

.pre-commit-config.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,8 +60,12 @@ repos:
6060
^conda_lock/_vendor/.*\.pyi$
6161
| ^tests/test-local-pip/setup\.py$
6262
| ^tests/test-pip-repositories/fake-private-package.*-1\.0\.0/setup\.py$
63+
| ^tests/test-corrupt-repodata/
6364
)
6465
# First exclude is due to:
6566
# conda_lock/_vendor/conda/__init__.py: error: Duplicate module named "conda_lock._vendor.conda" (also at "conda_lock/_vendor/conda.pyi")
6667
# Second two excludes are due to:
6768
# tests/test-pip-repositories/fake-private-package-1.0.0/setup.py: error: Duplicate module named "setup" (also at "tests/test-local-pip/setup.py")
69+
# Last exclude: tests/test-corrupt-repodata/ contains stage scripts from
70+
# the PR #862 reproduction harness; the committed fixtures are what our
71+
# tests consume, the scripts are reproduction scaffolding.

conda_lock/conda_solver.py

Lines changed: 467 additions & 2 deletions
Large diffs are not rendered by default.

conda_lock/errors.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,21 @@ class ChannelAggregationError(CondaLockError):
2121
"""
2222
Error thrown when lists of channels cannot be combined
2323
"""
24+
25+
26+
class OrphanLockedDependencyError(CondaLockError):
27+
"""
28+
Raised when ``apply_categories`` plus reverse propagation leaves
29+
planned packages with no category.
30+
31+
Such packages would silently vanish from the on-disk lockfile (the
32+
v1 serialization emits one entry per category, so an empty category
33+
set produces no entries) and the resulting environment would install
34+
fewer packages than the solver actually planned.
35+
36+
The usual root cause is a corrupt ``repodata_record.json`` from
37+
mamba/micromamba versions 2.1.1-2.3.3 (mamba-org/mamba#4052,
38+
mamba-org/mamba#4110) leaving the package -- *or one of its
39+
dependents* -- with empty ``depends``, which breaks the dependency
40+
walk both forwards and in reverse. See conda/conda-lock#896.
41+
"""

0 commit comments

Comments
 (0)