fix: fail on unreachable planned conda packages

After ``apply_categories`` walks forward from each requested input
spec, every planned package should have at least one category. A
package without any category is an orphan: the lockfile dependency
graph cannot explain why this package is in the plan.

Orphans are not tolerated. They would silently vanish from the v1
lockfile output (which emits one entry per category, so a package
with no category produces zero entries) and the resulting
environment would install fewer packages than the solver actually
planned. This is the silent-vanishing variant of the
conda#896 chain that cache-side and lockfile-side heal
do not catch -- it surfaces only after categorization.

Add ``conda_lock.solver.graph_integrity.assert_no_orphaned_conda_packages``
which owns the entire orphan policy:

- Definition of an orphaned planned package.
- No reverse-propagation of categories. ``apply_categories``'
  forward walk is the only categorization path. Reverse-prop
  hides the broken-graph signal in single-category projects
  (every orphan with any categorized dependency would silently
  inherit ``main``, and the hard-fail would never fire on real
  conda#896-style breakage), and ``"P depends on X (main)"`` does not
  prove ``"main needs P"``; a permissive variant would launder
  dev-only solver artifacts into main installs.
- Why ``pip`` is not normally an orphan even though nothing the
  user explicitly requests transitively depends on it: both
  conda's ``add_pip_as_python_dependency`` and libmamba's
  repo-load injection mutate ``python``'s declared dependencies
  to include ``pip`` at metadata-load time. So a healthy
  forward walk from ``python`` reaches ``pip`` in normal
  operation, and an orphaned ``pip`` indicates broken solver
  metadata rather than the expected shape.
- ``CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE=1`` escape hatch:
  intentionally ugly, demotes the hard-fail to a loud WARNING and
  assigns orphans to ``main`` so they survive v1 serialization.
  This is for implementation-bug emergencies (a buggy custom
  solver, missing ``add_pip_as_python_dependency`` injection on
  an exotic channel) not a configuration knob. The WARNING
  spells out the over-install consequence: every ``conda-lock
  install`` invocation -- even ones without
  ``--dev-dependencies`` or ``-e <category>`` -- will install
  the orphans, potentially over-installing dev-only solver
  artifacts into production environments.

Add ``OrphanLockedDependencyError`` in ``conda_lock.errors``;
``solve_conda`` calls
``assert_no_orphaned_conda_packages(planned, platform)`` after
``apply_categories``. The module docstring on
``conda_lock.conda_solver`` introduces the ``solver/*`` split as
a whole so a new reader sees the layering on first opening the
file.

Six component tests in ``tests/component/test_graph_integrity.py``
pin the policy contract:

- ``test_solve_conda_accepts_pip_via_python_add_pip_dependency``:
  the *normal* shape. ``python.depends == ["pip"]`` makes ``pip``
  forward-reachable; ``pip`` inherits ``python``'s requested
  category without rescue.
- ``test_solve_conda_hard_fails_when_python_metadata_omits_pip``:
  the abnormal-metadata shape. ``python.depends == []`` makes
  ``pip`` unreachable; the orphan check hard-fails. The test
  docstring names this as broken solver metadata, not normal
  solver auto-install behavior.
- ``test_solve_conda_envvar_demotes_orphan_to_warning``: pins
  ``CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE=1`` semantics. WARNING
  fires (explicitly mentioning ``main``,
  ``--dev-dependencies`` / ``-e <category>``, and the envvar
  name so the message can't silently weaken), orphans are
  assigned to ``main``, ``solve_conda`` returns instead of
  raising.
- ``test_solve_conda_hard_fails_on_unrecoverable_orphan``: the
  classic conda#896-corrupt-cache shape -- ``zlib`` planned with
  empty depends, no categorized package transitively requires
  it -- hard-fails with regenerate-from-sources guidance and
  the envvar escape named in the error message.
- ``test_solve_conda_orphan_via_dep_breakage_still_hard_fails``:
  multi-category corrupt-cache variant.
- ``test_solve_conda_passes_when_dependency_graph_is_intact``:
  positive sanity check.

Author: claude

conda_lock/conda_solver.py

@@ -1,3 +1,29 @@
+"""Orchestration entry points for conda-lock's solver pipeline.
+
+The narrow responsibilities split out of this module live under
+``conda_lock.solver``:
+
+- ``solver.repodata_cache``: URL normalization, cache path
+  derivation, record identity checks, mamba 2.1.1-2.3.3 stub-record
+  detection, and ``info/index.json``-based healing.
+- ``solver.dry_run``: normalize a solver's ``--dry-run --json``
+  output into a uniform shape with rich FETCH actions, falling back
+  to disk when the LINK metadata is sparse.
+- ``solver.lockfile_heal``: repair carry-forward empty
+  ``dependencies`` from the local cache before
+  ``fake_conda_environment`` propagates them.
+- ``solver.graph_integrity``: forward-reachability check on the
+  planned package set, with the
+  ``CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE`` escape hatch.
+
+What's left here is glue: drive the conda/mamba subprocess for the
+fresh-solve and update paths, translate the dryrun into
+``LockedDependency`` shapes, run categorization, and assert graph
+integrity. The fake-prefix machinery used by ``--update`` also
+lives here because it is a peer of the subprocess invocation, not
+a cache or graph concern.
+"""
+
 import json
 import logging
 import os
@@ -27,6 +53,7 @@
 from conda_lock.models.dry_run_install import DryRunInstall, LinkAction
 from conda_lock.models.lock_spec import Dependency, VersionedDependency
 from conda_lock.solver.dry_run import reconstruct_fetch_actions
+from conda_lock.solver.graph_integrity import assert_no_orphaned_conda_packages
 from conda_lock.solver.lockfile_heal import heal_locked_dependencies_from_cache
 from conda_lock.tempdir_manager import temporary_directory
 
@@ -150,6 +177,12 @@ def solve_conda(
         mapping_url=mapping_url,
     )
 
+    # Forward-reachability check on the planned package set. The
+    # policy, escape hatch, and the rationale for not
+    # reverse-propagating categories all live in
+    # ``conda_lock.solver.graph_integrity``.
+    assert_no_orphaned_conda_packages(planned, platform)
+
     return planned
 
 
@@ -331,14 +364,14 @@ def update_specs_for_arch(
         # elsewhere is not evidence about an ambiguous entry; partial
         # caches are normal.
         #
-        # Two failure modes follow. The orphan check downstream catches
-        # the silent-vanishing variant (corrupt entry's transitive deps
-        # become orphans). It does NOT catch the "categorized
-        # corrupt-carrier" variant where the corrupt entry is itself a
-        # requested or otherwise-reachable root and its missing
-        # transitive deps are reachable via other paths; the lockfile
-        # remains internally inconsistent and re-locks may silently
-        # drift. The WARNING below is visibility for that
+        # Two failure modes follow. ``assert_no_orphaned_conda_packages``
+        # downstream catches the silent-vanishing variant (corrupt
+        # entry's transitive deps become orphans). It does NOT catch
+        # the "categorized corrupt-carrier" variant where the corrupt
+        # entry is itself a requested or otherwise-reachable root and
+        # its missing transitive deps are reachable via other paths;
+        # the lockfile remains internally inconsistent and re-locks may
+        # silently drift. The WARNING below is visibility for that
         # harder-to-detect case so an operator can regenerate from
         # sources.
         logger.warning(
@@ -547,7 +580,6 @@ def make_fake_python_binary(prefix: str) -> None:
             This was called as:
                 {cmd}
             '''
-
             print(stderr_message, file=sys.stderr, flush=True, end='')
 
             if "-m pip" in cmd:

conda_lock/errors.py

@@ -21,3 +21,28 @@ class ChannelAggregationError(CondaLockError):
     """
     Error thrown when lists of channels cannot be combined
     """
+
+
+class OrphanLockedDependencyError(CondaLockError):
+    """
+    Raised when planned packages cannot be reached from the
+    requested input specs through declared ``dependencies`` edges.
+
+    Such packages would silently vanish from the on-disk lockfile (the
+    v1 serialization emits one entry per category, so an empty category
+    set produces no entries) and the resulting environment would install
+    fewer packages than the solver actually planned. An orphan is
+    proof that the lockfile dependency graph is broken: every package
+    in the plan must be reachable from some input spec.
+
+    The usual root cause is a corrupt ``repodata_record.json`` from
+    mamba/micromamba versions 2.1.1-2.3.3 (mamba-org/mamba#4052,
+    mamba-org/mamba#4110) leaving the package -- *or one of its
+    dependents* -- with empty ``depends``, which breaks the forward
+    dependency walk. See conda/conda-lock#896.
+
+    Emergency escape: ``CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE=1`` demotes
+    the error to a loud warning and assigns orphans to ``main`` so
+    they survive v1 serialization. This is intentionally ugly and is
+    not a supported configuration knob.
+    """

conda_lock/solver/graph_integrity.py

@@ -0,0 +1,162 @@
+"""Graph-integrity invariants for solved conda plans.
+
+After ``apply_categories`` walks forward from each requested input
+spec, every planned package should have at least one category. A
+package without any category is an orphan: the lockfile dependency
+graph cannot explain why this package is in the plan.
+
+Orphans are *not* tolerated. They would silently vanish from the v1
+lockfile output (which emits one entry per category, so a package
+with no category produces zero entries) and the resulting
+environment would install fewer packages than the solver actually
+planned. ``assert_no_orphaned_conda_packages`` is the guard that
+prevents that.
+
+This module owns the entire orphan policy in one place:
+
+- the definition of an orphaned planned package;
+- why we do **not** reverse-propagate categories (would hide the
+  broken-graph signal in single-category projects, and would
+  launder dev-only solver artifacts into main installs);
+- why ``pip`` is not normally an orphan
+  (``add_pip_as_python_dependency`` in conda; equivalent
+  injection in libmamba), so an orphaned ``pip`` indicates
+  abnormal solver metadata rather than expected behavior;
+- the ``CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE=1`` escape hatch and
+  the over-install consequence of promoting orphans to ``main``.
+
+The orphan check is one of the central invariants of the
+lockfile, not a local cleanup step; it lives here so a future
+reader sees that on first opening the file.
+"""
+
+import logging
+import os
+
+from conda_lock.errors import OrphanLockedDependencyError
+from conda_lock.lockfile.v2prelim.models import LockedDependency
+
+
+logger = logging.getLogger(__name__)
+
+
+_ORPHAN_ESCAPE_ENVVAR = "CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE"
+
+
+def assert_no_orphaned_conda_packages(
+    planned: dict[str, LockedDependency], platform: str
+) -> None:
+    """Raise ``OrphanLockedDependencyError`` if any planned package is
+    unreachable from the requested input specs.
+
+    Called after ``apply_categories``: any conda entry still without a
+    category at this point was not reached by the forward walk from
+    the requested roots through declared ``.dependencies`` edges.
+    That is the broken-graph signal we hard-fail on.
+
+    No reverse-propagation. ``apply_categories``' forward walk is the
+    only categorization path. The reasons are policy-level,
+    independent of whether the orphan happens to be a
+    "solver auto-install" like ``pip``:
+
+    - Reverse-prop hides the broken-graph signal. In single-category
+      projects (only ``main`` requested), every orphan with any
+      categorized dependency would silently inherit ``main`` and the
+      hard-fail would never fire on real conda/conda-lock#896-style
+      breakage.
+    - "P depends on X (main)" does not prove "main needs P"; it
+      only correlates. A permissive variant would launder dev-only
+      solver artifacts into main installs.
+
+    In a healthy conda/mamba dryrun this should never fire even for
+    ``pip``: both ``conda.core.subdir_data.add_pip_as_python_dependency``
+    and libmamba's repo-load injection mutate ``python``'s declared
+    dependencies to include ``pip`` at metadata-load time, and
+    libmamba's install API additionally injects ``pip`` as an
+    explicit root request whenever the user asks for ``python``. So
+    a forward walk from ``python`` reaches ``pip`` in normal
+    operation, and an orphaned ``pip`` is a sign of broken solver
+    metadata rather than the expected shape.
+
+    Escape hatch: ``CONDA_LOCK_ALLOW_ORPHANED_LOCKFILE=1`` demotes
+    the hard-fail to a loud WARNING and assigns orphans to ``main``
+    so they survive v1 serialization. This is intentionally ugly
+    and not documented as a stable interface; it exists for
+    implementation-bug emergencies and one-off recovery (a buggy
+    custom solver, missing dependency injection on an exotic
+    channel, etc.). Promoting orphans to ``main`` will cause every
+    ``conda-lock install`` invocation -- including ones that do
+    NOT pass ``--dev-dependencies`` or ``-e <category>`` -- to
+    install them, potentially over-installing dev-only solver
+    artifacts into production environments.
+    """
+    orphans = sorted(name for name, dep in planned.items() if not dep.categories)
+    if not orphans:
+        return
+
+    message = (
+        f"{len(orphans)} planned conda package(s) on platform "
+        f"{platform} are unreachable from the requested input "
+        f"specs through declared `dependencies` edges: {orphans}. "
+        f"The lockfile dependency graph is broken: these "
+        f"packages would silently vanish from the v1 lockfile "
+        f"output (which emits one entry per category, so a "
+        f"package with no category produces zero entries) and "
+        f"the resulting environment would install fewer "
+        f"packages than the solver actually planned.\n\n"
+        f"In a healthy conda/mamba dryrun every planned package "
+        f"is forward-reachable from some requested spec. Even "
+        f"``pip`` -- often described as a 'solver auto-install' "
+        f"-- is normally reachable because conda and libmamba "
+        f"both inject ``pip`` into ``python``'s declared "
+        f"dependencies at repodata-load time "
+        f"(``add_pip_as_python_dependency``). An orphan therefore "
+        f"indicates either: (1) corrupt ``repodata_record.json`` "
+        f"metadata from mamba/micromamba 2.1.1-2.3.3 (see "
+        f"conda/conda-lock#896 / mamba-org/mamba#4110) leaving "
+        f"the package -- or one of its dependents -- with empty "
+        f"``depends``, breaking the forward walk; or (2) a "
+        f"non-standard solver / channel / metadata source that "
+        f"omits the usual dependency injections.\n\n"
+        f"To resolve: regenerate the lockfile from sources on a "
+        f"known-clean cache (`mamba clean -a` then `conda-lock "
+        f"lock -f <your sources> ...`), or add the orphaned "
+        f"package(s) as explicit input specs in the relevant "
+        f"category. Re-running ``--update`` against the same "
+        f"input lockfile after only clearing the local cache "
+        f"will not recover packages that already vanished "
+        f"during a previous v1 serialization.\n\n"
+        f"Emergency escape hatch: set "
+        f"``{_ORPHAN_ESCAPE_ENVVAR}=1`` to demote "
+        f"this error to a warning and continue. Orphaned "
+        f"packages will be assigned category ``main`` so they "
+        f"survive v1 serialization. WARNING: assigning to "
+        f"``main`` means a dev-only solver artifact (something "
+        f"the solver pulled in only because a dev-category root "
+        f"asked for it) will be installed by every "
+        f"``conda-lock install`` invocation, including ones "
+        f"that did NOT pass ``--dev-dependencies`` or "
+        f"``-e <category>``. This can over-install dev-only "
+        f"packages into production environments. The escape "
+        f"hatch is a band-aid for implementation-bug "
+        f"emergencies, not a supported configuration -- use it "
+        f"only if you understand the consequences and have "
+        f"verified that promoting the orphans to ``main`` is "
+        f"acceptable for your install paths."
+    )
+    if os.environ.get(_ORPHAN_ESCAPE_ENVVAR) == "1":
+        logger.warning(
+            "%s=1 set; demoting orphaned-lockfile error to a "
+            "warning and promoting orphans to category 'main'. "
+            "This will cause every ``conda-lock install`` "
+            "invocation -- including ones that do NOT pass "
+            "``--dev-dependencies`` or ``-e <category>`` -- to "
+            "install these packages, which may include dev-only "
+            "solver artifacts. %s",
+            _ORPHAN_ESCAPE_ENVVAR,
+            message,
+        )
+        for name in orphans:
+            planned[name].categories.add("main")
+        return
+    raise OrphanLockedDependencyError(message)