Commit 22b3472
committed
feat: address conda#896 corrupt-cache failure mode
Builds on the mamba 2.6.0 cache-layout fix to also defend against the
corrupt ``repodata_record.json`` entries that mamba/micromamba 2.1.1-2.3.3
wrote to the package cache (mamba-org/mamba#4052, fixed upstream in
mamba-org/mamba#4110 / mamba 2.6.0). The 2.3.3 partial fix recovered
``depends`` and ``constrains`` but still wrote ``timestamp: 0`` and
``license: ""`` -- enough to trip our detector.
Mitigations:
1. *Corruption detection + heal in disk fallback.*
``_is_mamba_2x_corrupt_record`` matches the ``timestamp == 0`` +
empty ``license`` signature. ``_heal_corrupt_record`` overlays the
record with ``info/index.json`` (extracted at install time, never
affected by the bug). WARNING per heal.
2. *Rich-LINK corruption guard.*
The ``_link_action_as_fetch`` fast path bypasses
``_get_repodata_record`` and therefore the corruption detector. We
re-check the corruption signature in the fast path so a LINK whose
metadata carries the bug routes to disk fallback (where
``_heal_corrupt_record`` can recover from ``info/index.json``)
rather than synthesizing a corrupt FETCH directly. Mamba 2.6.0+ is
supposed to heal cache records before emitting them in LINK; we no
longer depend on that external invariant.
3. *pkgs_dirs leak detection.*
Reports paths that mamba/conda merged in from condarc despite the
env-var override.
4. *Degraded-path warning.*
Single WARNING per platform when ``_reconstruct_fetch_actions``
takes the disk-fallback branch.
5. *Reverse-propagation of categories from .dependencies.*
Solver auto-installs (pip, wheel) aren't transitive deps of
anything requested. Reverse-prop: an orphan inherits the union of
categories of the packages it depends on. ``_truncate_main_category``
runs after reverse-prop so a package inheriting both ``main`` and
``dev`` collapses to ``{main}`` -- v1 serialization doesn't emit
the package twice.
6. *Hard-fail on residual orphans.*
Anything still without a category after reverse-prop is the
corruption-induced silent-vanishing failure mode. Raises
``OrphanLockedDependencyError`` rather than silently drop.
7. *Lockfile-side heal of carry-forward corruption (per-entry).*
``_heal_locked_dependencies_from_cache`` runs in
``update_specs_for_arch`` *before* ``fake_conda_environment`` and
``to_fetch_action()``. The cache is a *per-entry* discriminator:
- Cache says non-empty depends -> heal in place. The contradiction
(cache vs lockfile) is per-entry proof of corruption for that
entry.
- Cache says empty depends -> legit-empty, leave alone.
- Cache has no entry -> ambiguous. Healable-elsewhere is NOT
evidence about ambiguous-here: partial caches are normal (a user
may have ``libzlib`` cached but not ``tzdata``). We WARN with
prescriptive text and let the orphan check downstream cover the
silent-vanishing variant. We do NOT hard-fail on the
``healed > 0 + ambiguous`` mixed case -- that predicate
false-positived on the ordinary scenario where one corrupt entry
is healable while a legit-empty leaf is simply not yet in the
cache.
The warning is honest about a residual limitation: a corrupt entry
that is itself a requested or otherwise-reachable root may carry
forward unflagged when its missing transitive deps happen to be
reachable via other paths in the graph. The repair guidance is to
regenerate the lockfile from sources on a known-clean cache;
``conda-lock install`` is demoted to a diagnostic-only note for
investigating carry-forward metadata, with an explicit warning not
to install a corrupt lockfile into a real environment (any
packages already vanished during v1 serialization are
unrecoverable).
End-to-end test backed by the PR conda#862 reproduction harness:
``test_pr862_corrupt_2_1_1_cache_does_not_drop_packages`` walks the
full committed reproduction pipeline minus Docker, parametrized over
both ``[mamba]`` and ``[conda-standalone]``. It warms a real cache,
overlays the corrupt ``info/{index,repodata_record}.json`` from
``2.1.1-pkgs.tar.gz``, runs ``conda-lock lock`` against the hybrid
cache, renders the explicit lockfile (stage 05), and asserts
structural lockfile equivalence + explicit-URL equivalence + empty-
deps headroom against a clean control.
Plus unit tests for: corruption signature detection, heal-from-index,
rich-LINK corruption guard, heal-in-place + WARN on ambiguous
(includes the explicit healed-libzlib + legit-empty-tzdata-leaf
scenario the predicate now correctly handles), legit-empty-deps
acceptance, partial-cache-of-legit-empty does NOT raise, pkgs_dirs
leak warning, degraded-path warning, mixed-root reverse propagation
with main truncation, hard-fail on unrecoverable orphans, auto-install
orphan inheritance.
To make the conda-standalone parametrization exercise both CI and
local ``pixi run pytest``, ``conda-standalone`` is added to:
- ``environments/dev-environment.yaml`` (CI source of truth)
- ``environments/conda-lock-python-3.10.yaml``,
``environments/conda-lock-python-3.14.yaml`` (relocked)
- ``pixi.toml`` (local dev source of truth)
``pixi.lock`` will need a one-time ``pixi lock`` to refresh the
solved package set; this could not be done in the sandbox where the
change was developed. CI is unaffected -- it consumes the conda-lock
environment files above, not pixi.
The PR conda#862 stage scripts are excluded from mypy/ty/pre-commit --
they are Docker-based one-shot data generators, not runtime code.1 parent c2f92f6 commit 22b3472
38 files changed
Lines changed: 24693 additions & 7000 deletions
File tree
- conda_lock
- environments
- tests
- test-corrupt-repodata
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
| 63 | + | |
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
67 | 68 | | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
0 commit comments