fix: invalidate text index after document deletion

[Iams4kura]

Summary

Fixes #1140.

• Fixes: After Docs.delete removes a document and its Text objects, the associated texts remain searchable in texts_index and every later retrieval over-fetches to filter tombstoned results.
• Root cause: Docs.delete only recorded the removed document key in deleted_dockeys; it never invalidated texts_index, so stale vectors persisted and retrieve_texts compensated with an increasingly expensive tombstone filter.

Regression evidence

• Before: uv run pytest tests/test_paperqa.py::test_delete_invalidates_texts_index -q exited 1

• After: uv run pytest tests/test_paperqa.py::test_delete_invalidates_texts_index -q exited 0

Verification

• uv run pytest tests/test_paperqa.py -k 'delete_invalidates_texts_index or none_values or docdetails_deserialization or docdetails_doc_id_roundtrip or text_comparison or context_comparison' -q
• uv run prek run --files src/paperqa/docs.py tests/test_paperqa.py
• uv run pylint src/paperqa/docs.py
• uv run refurb --version && uv run refurb src/paperqa/docs.py tests/test_paperqa.py

Scope

• 2 files changed, +48 / -7 lines

[Copilot]

Pull request overview

This PR addresses issue #1140 by ensuring that deleting a document invalidates the in-memory/searchable text index so deleted texts do not remain retrievable, and by removing the prior tombstone-based workaround that over-fetched search results.

Changes:

• Removes the deleted_dockeys workaround and stops over-fetching/filters in Docs.retrieve_texts.
• Updates Docs.delete to invalidate the vector index after a deletion.
• Adds a regression test covering index invalidation and post-delete retrieval behavior across vector store implementations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
src/paperqa/docs.py	Clears texts_index on delete and simplifies retrieval by removing tombstone-based over-fetch/filter logic.
tests/test_paperqa.py	Adds regression test ensuring deletions invalidate the vector index and retrieval returns only retained texts.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.