detect conda#896 corruption per-entry, not by absolute threshold

The previous parse-time detector warned only when more than 10
empty-deps conda entries appeared on a single platform. The reviewer
correctly pointed out two failure modes:

1. Small environments with fewer than 11 packages can be just as
   corrupt; the threshold silently passes them through.
2. In practice corruption affects only some fraction of entries, so
   any pure count-based heuristic is fundamentally fragile.

Replace the threshold with a per-entry classifier that exploits a
mathematical invariant: a conda package whose build string identifies
it as built against a Python ABI MUST declare a python dependency.
There is no legitimate ``pyhd*``, ``pypy*``, or ``py<digit>...`` build
with empty ``depends``. So a single such entry with empty
``dependencies`` is proof of corruption -- threshold-free, works on
lockfiles of any size.

Verified against the committed PR conda#862 fixtures: 48 of 77 corrupt
empty-deps entries match the python-built signal (zero false
positives in the clean baseline -- ``_libgcc_mutex``, ``tzdata``,
``python_abi``, ``nlohmann_json-abi`` all have non-``py*`` builds).

For the residual native-binary entries (e.g. ``openssl``, ``krb5``)
where lockfile-only evidence cannot prove corruption per-entry, fall
back to a small absolute count (>3) of unclassified empty-deps
entries on a single platform after stripping the well-known
legit-empty allowlist (``_*`` ABI mutexes, ``tzdata``,
``ca-certificates``, ``*-abi``, ``*_abi``). The threshold here is
deliberately low so even a small carrier lockfile trips, with the
allowlist absorbing the conservative side.

Tests cover both signals and the negative cases:
- ``test_detector_warns_on_single_pynoarch_empty_entry``: 1
  ``babel-pyhd8ed1ab_0`` with empty deps, lockfile of size 1,
  triggers warning (the small-corrupt case the reviewer flagged).
- ``test_detector_warns_on_single_arch_python_empty_entry``: same
  for ``mypy-py314h*``.
- ``test_detector_quiet_on_legit_empty_leaves``: 5 well-known
  legit-empty packages, no warning.
- ``test_detector_warns_on_small_native_lib_corruption``: 4 native
  libs (>3 unclassified threshold) trips the fallback.
- ``test_detector_quiet_on_two_native_empties_below_threshold``: 2
  unclassified empties stay quiet to avoid false positives in tiny
  envs.
- The two existing fixture-based tests still pin the large-lockfile
  behavior end-to-end (corrupt and clean PR conda#862 fixtures).

https://claude.ai/code/session_01F6kPbdaqoKb9XARYdbRVhd

Author: claude

conda_lock/lockfile/__init__.py

@@ -1,5 +1,6 @@
 import logging
 import pathlib
+import re
 
 from collections import defaultdict
 from collections.abc import Collection, Mapping, Sequence
@@ -169,63 +170,169 @@ def dep_name(*, manager: str, dep: str, mapping_url: str) -> str:
     _truncate_main_category(planned)
 
 
-# A handful of conda packages legitimately ship with no runtime
-# dependencies (``tzdata``, ``python_abi``, ``_libgcc_mutex``,
-# ``_openmp_mutex``, ``nlohmann_json-abi``, ``ca-certificates`` ...)
-# so seeing a small number of empty-deps entries per platform is
-# normal. The mamba/micromamba 2.1.1-2.3.3 cache corruption bug
-# (mamba-org/mamba#4052, conda/conda-lock#896) zeros ``depends`` for
-# *most* installed packages, producing many tens of empty-deps
-# entries per platform. Use that gap as a heuristic threshold so a
-# corrupt carrier lockfile is flagged everywhere it's parsed --
-# including read-only paths like ``conda-lock render`` and
-# ``conda-lock install`` that don't go through the cache-backed
-# heal in ``update_specs_for_arch``.
-_LEGIT_EMPTY_DEPS_PER_PLATFORM_THRESHOLD = 10
+# --- Detection of conda/conda-lock#896 corrupt-cache lockfiles ----------
+#
+# When mamba/micromamba 2.1.1-2.3.3 writes a corrupt ``repodata_record.json``
+# (mamba-org/mamba#4052, fix mamba-org/mamba#4110), every conda-lock entry
+# generated against that cache loses its ``depends`` (and other) metadata.
+# In the resulting lockfile, that surfaces as a conda package entry whose
+# ``dependencies`` mapping is empty.
+#
+# Empty ``dependencies`` *can* be legitimate -- a handful of packages
+# (``tzdata``, ``python_abi``, ``_libgcc_mutex``, ``_openmp_mutex``,
+# ``nlohmann_json-abi``, ``ca-certificates`` ...) really do ship with
+# no runtime deps. So we cannot just count empties; we have to classify
+# them. The strongest classifier is the build string in the package
+# URL: a build string starting with ``py`` (noarch ``pyhd...``,
+# ``pypy...``, or arch ``py310...``/``py314...``) identifies the
+# package as built against a specific Python ABI. Such a package is
+# *required* to declare a ``python`` dependency; an empty
+# ``dependencies`` mapping is therefore mathematically impossible for
+# a healthy entry. That gives us a per-entry hard signal that is
+# threshold-free and works on lockfiles of any size -- a single
+# ``babel-pyhd8ed1ab_0`` with empty deps is enough to flag.
+#
+# A second, weaker signal covers native-binary packages whose
+# corruption *is* impossible to distinguish from a legit-empty leaf
+# without external metadata (``openssl`` corrupted vs. ``_libgcc_mutex``
+# legit). For those we strip the well-known legit-empty allowlist and
+# fall back to a small absolute count so even a tiny carrier lockfile
+# trips the warning.
+
+_PYTHON_BUILD_RE = re.compile(r"^(py|pypy)(\d|h)")
+"""Build strings starting with ``py<digit>`` (e.g. ``py314h5bd0f2a_0``)
+or ``pyh*`` / ``pypy*`` (e.g. ``pyhd8ed1ab_0``, ``pypy39_0``) identify
+a package as built against a specific Python ABI."""
+
+_LEGIT_EMPTY_NAMES = frozenset({"tzdata", "ca-certificates"})
+_UNCLASSIFIED_EMPTY_DEPS_THRESHOLD = 3
+"""Fallback threshold for native-binary empty-deps entries that are not
+covered by the per-entry hard signal and not on the allowlist. Set
+small (3) so even a small corrupt lockfile trips the warning; the
+hard signal handles everything noarch-python."""
+
+
+def _build_string_from_url(url: str) -> str | None:
+    """Extract the build string from a conda package URL.
+
+    URLs look like ``.../<subdir>/<name>-<version>-<build>.{conda,tar.bz2}``.
+    Returns ``None`` if the URL does not parse as a conda artifact.
+    """
+    fn = url.rsplit("/", 1)[-1]
+    for ext in (".conda", ".tar.bz2"):
+        if fn.endswith(ext):
+            stem = fn[: -len(ext)]
+            break
+    else:
+        return None
+    parts = stem.rsplit("-", 2)
+    if len(parts) != 3:
+        return None
+    return parts[2]
+
+
+def _looks_like_legit_empty(name: str) -> bool:
+    """Names of well-known packages that legitimately ship with no
+    runtime ``depends``.
+
+    These are data-only packages (``tzdata``, ``ca-certificates``),
+    ABI-mutex packages (``_libgcc_mutex``, ``_openmp_mutex``), and
+    ABI-marker packages (``python_abi``, ``nlohmann_json-abi``).
+    """
+    if name in _LEGIT_EMPTY_NAMES:
+        return True
+    if name.startswith("_"):
+        return True
+    if name.endswith(("-abi", "_abi")) or "-abi-" in name or "_abi-" in name:
+        return True
+    return False
+
+
+def _is_python_built(entry: LockedDependency) -> bool:
+    """``True`` if the entry's build string identifies it as built
+    against a Python ABI -- in which case empty ``dependencies`` is
+    proof of corruption."""
+    build = _build_string_from_url(entry.url)
+    return build is not None and bool(_PYTHON_BUILD_RE.match(build))
 
 
 def _warn_if_lockfile_looks_corrupt(lockfile: Lockfile, source: pathlib.Path) -> None:
-    """Heuristic warning when a parsed lockfile has the empty-``depends``
-    fingerprint of mamba/micromamba 2.1.1-2.3.3 cache corruption.
-
-    This is the *visibility* layer for the
-    ``conda-lock render``/``install``/lock-no-solve paths, where
-    ``update_specs_for_arch``'s cache-backed heal never runs. It's a
-    heuristic, not proof: we cannot tell legit-empty (``tzdata``)
-    from corrupt-empty (a package whose ``depends`` got zeroed) from
-    the lockfile alone -- only the count gap distinguishes them.
-    Warns; never raises (read-only callers can't necessarily
+    """Emit a single WARNING when a parsed lockfile shows the
+    empty-``depends`` fingerprint of mamba/micromamba 2.1.1-2.3.3
+    cache corruption (conda/conda-lock#896, mamba-org/mamba#4052,
+    mamba-org/mamba#4110).
+
+    This is the *visibility* layer for read-only paths --
+    ``conda-lock render``, ``conda-lock install``, and a no-solve
+    ``conda-lock lock`` -- where ``update_specs_for_arch``'s
+    cache-backed heal never runs.
+
+    Detection is two-tiered:
+
+    1. **Per-entry hard signal**: any conda entry whose build string
+       starts with ``py`` (noarch ``pyhd*`` or arch ``py310*``...) and
+       whose ``dependencies`` mapping is empty is *definitionally*
+       corrupt -- no python ABI build can have zero declared
+       dependencies. This signal is threshold-free; one such entry
+       triggers the warning. It catches small corrupt lockfiles that
+       a count-based heuristic would miss.
+
+    2. **Fallback for native-binary entries**: empty-deps entries
+       that are *not* python-built and *not* on the well-known
+       legit-empty allowlist (``_*``, ``*-abi``, ``tzdata``, ...) are
+       counted; if more than ``_UNCLASSIFIED_EMPTY_DEPS_THRESHOLD`` of
+       them appear on any single platform, we warn. This covers
+       corruption of ``openssl``/``krb5``/etc., where lockfile
+       evidence alone cannot prove corruption per-entry.
+
+    Warns; never raises (read-only callers cannot necessarily
     re-solve).
     """
-    by_platform: dict[str, list[str]] = defaultdict(list)
+    definite_per_platform: dict[str, list[str]] = defaultdict(list)
+    unclassified_per_platform: dict[str, list[str]] = defaultdict(list)
     for entry in lockfile.package:
-        if entry.manager == "conda" and not entry.dependencies:
-            by_platform[entry.platform].append(entry.name)
-    suspicious = {
+        if entry.manager != "conda" or entry.dependencies:
+            continue
+        if _is_python_built(entry):
+            definite_per_platform[entry.platform].append(entry.name)
+        elif not _looks_like_legit_empty(entry.name):
+            unclassified_per_platform[entry.platform].append(entry.name)
+
+    has_definite = any(definite_per_platform.values())
+    suspicious_unclassified = {
         p: ns
-        for p, ns in by_platform.items()
-        if len(ns) > _LEGIT_EMPTY_DEPS_PER_PLATFORM_THRESHOLD
+        for p, ns in unclassified_per_platform.items()
+        if len(ns) > _UNCLASSIFIED_EMPTY_DEPS_THRESHOLD
     }
-    if not suspicious:
+    if not has_definite and not suspicious_unclassified:
         return
-    summary = "; ".join(
-        f"{platform}: {len(names)} empty-deps entries (sample: {sorted(names)[:5]})"
-        for platform, names in sorted(suspicious.items())
-    )
+
+    parts: list[str] = []
+    if has_definite:
+        for platform, names in sorted(definite_per_platform.items()):
+            parts.append(
+                f"{platform}: {len(names)} python-built package(s) with "
+                f"empty 'dependencies' (sample: {sorted(names)[:5]}) -- "
+                f"these cannot legitimately have zero deps"
+            )
+    for platform, names in sorted(suspicious_unclassified.items()):
+        parts.append(
+            f"{platform}: {len(names)} non-allowlisted empty-deps "
+            f"entries (sample: {sorted(names)[:5]})"
+        )
+    summary = "; ".join(parts)
+
     logger.warning(
-        "Lockfile %s contains far more empty-'dependencies' entries "
-        "on at least one platform than the legitimate baseline of ~5 "
-        "(tzdata, python_abi, _libgcc_mutex, _openmp_mutex, "
-        "nlohmann_json-abi). This is the fingerprint of a lockfile "
-        "generated against a corrupt mamba/micromamba 2.1.1-2.3.3 "
-        "cache (see conda/conda-lock#896 / mamba-org/mamba#4110); "
-        "some packages may already have vanished from the lockfile "
-        "during v1 serialization. Per-platform: %s. The safe repair "
-        "is to regenerate from sources on a known-clean cache "
-        "(`mamba clean -a` then `conda-lock lock -f <your sources>`). "
-        "Do NOT install this lockfile into a production environment "
-        "without regenerating: packages already lost during v1 "
-        "serialization are unrecoverable from it.",
+        "Lockfile %s shows the fingerprint of a lockfile generated "
+        "against a corrupt mamba/micromamba 2.1.1-2.3.3 cache "
+        "(see conda/conda-lock#896 / mamba-org/mamba#4110): %s. "
+        "Some packages may already have vanished from the lockfile "
+        "during v1 serialization. The safe repair is to regenerate "
+        "from sources on a known-clean cache (`mamba clean -a` then "
+        "`conda-lock lock -f <your sources>`). Do NOT install this "
+        "lockfile into a production environment without "
+        "regenerating: packages already lost during v1 serialization "
+        "are unrecoverable from it.",
         source,
         summary,
     )