Skip to content

Commit af2799d

Browse files
dzianissokalaudzianissokalau
committed
Fix confirmed documentation drift
1 parent b852ded commit af2799d

7 files changed

Lines changed: 214 additions & 29 deletions

‎DOCUMENTATION_DRIFT_FIX_REQUIREMENTS.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Documentation Drift Fix Requirements
2+
3+
Created: 2026-05-26
4+
5+
Source analysis:
6+
7+
- `DOCUMENTATION_DRIFT_REVIEW_SUMMARY.md`
8+
- `/Users/dzianissokalau/Downloads/async-research-documentation-review.md`
9+
10+
## Goal
11+
12+
Fix the confirmed documentation and metadata drift from the documentation audit
13+
without reopening stale or already-resolved findings from the external report.
14+
15+
The fix is intentionally small: normalize remaining user-facing references onto
16+
the public `async-research` CLI, remove misleading legacy location wording, and
17+
point package metadata at the current roadmap index.
18+
19+
## Scope
20+
21+
### Required Fixes
22+
23+
1. Replace stale public helper script names in
24+
`src/async_research_workflow/docs/autonomy_readiness_plan.md`.
25+
The readiness checklist should name public CLI commands, not old helper
26+
filenames.
27+
2. Replace the remaining `simulate_scheduled_week.py` prose reference in
28+
`src/async_research_workflow/docs/operational_readiness_runbook.md`.
29+
3. Replace repeated `implemented in docs/examples` wording in
30+
`src/async_research_workflow/docs/feedback_hardening_plan.md` with wording
31+
that reflects the current package layout.
32+
4. Update the `Roadmap` project URL in `pyproject.toml` from a delivered
33+
historical roadmap to the current roadmap index.
34+
5. Add a focused documentation-reference regression guard so user-facing docs do
35+
not reintroduce the stale public helper names:
36+
- `run_acceptance_suite.py`
37+
- `run_autonomy_benchmark.py`
38+
- `simulate_scheduled_week.py`
39+
- `autonomy_readiness_gate.py`
40+
41+
### Non-Goals
42+
43+
- Do not rewrite the package docs index. The current `Package Resources`
44+
section already reflects the package layout.
45+
- Do not expand the root README command map. It already covers the broad public
46+
CLI surface.
47+
- Do not remove every direct `python -m async_research_workflow.scripts...`
48+
invocation. Some helper calls are intentionally advanced/internal.
49+
- Do not change framework behavior, command semantics, schemas, starter
50+
templates, or state-transition logic.
51+
- Do not treat historical roadmap or review artifacts as current operator
52+
instructions unless they are in the active package docs surfaced to users.
53+
54+
## Acceptance Criteria
55+
56+
- No current user-facing docs instruct users to run the four stale public helper
57+
script filenames listed above.
58+
- The operational readiness runbook describes `async-research simulate-week`
59+
as the rehearsal command.
60+
- The feedback hardening implementation backlog no longer says features were
61+
implemented in `docs/examples`.
62+
- `pyproject.toml` points the `Roadmap` project URL at `roadmaps/README.md`.
63+
- The doc-reference regression test fails if one of the stale public helper
64+
filenames returns to current package docs, starter docs, root README, or
65+
roadmap docs outside explicitly historical contexts.
66+
67+
## Verification
68+
69+
Run:
70+
71+
```bash
72+
git diff --check
73+
.venv/bin/python -m unittest tests.test_doc_references tests.test_project_metadata tests.test_cli_help
74+
```
75+
76+
Run full discovery only if the patch touches implementation code. This fix is
77+
docs, metadata, and targeted test coverage only.

‎DOCUMENTATION_DRIFT_REVIEW_SUMMARY.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Documentation Drift Review Summary
2+
3+
Created: 2026-05-26
4+
5+
Source report:
6+
`/Users/dzianissokalau/Downloads/async-research-documentation-review.md`
7+
8+
## Summary
9+
10+
The attached documentation-to-implementation audit appears to be based on an
11+
older public snapshot of the repository, not the current workspace. Most of its
12+
largest claims are stale or already fixed.
13+
14+
I verified the current framework and documentation with:
15+
16+
- `.venv/bin/async-research --help`
17+
- `pyproject.toml`
18+
- `README.md`
19+
- `src/async_research_workflow/docs/README.md`
20+
- `src/async_research_workflow/docs/operational_readiness_runbook.md`
21+
- current tests and CI configuration
22+
- `.venv/bin/python -m unittest tests.test_doc_references tests.test_project_metadata tests.test_cli_help`
23+
24+
The 32 verification tests above pass.
25+
26+
## Claims That Are Not Real Anymore
27+
28+
| Report claim | Current reality |
29+
| --- | --- |
30+
| Version is `0.1.0a1`. | False. Current version is `0.3.0a1` in `pyproject.toml`, `src/async_research_workflow/__init__.py`, tests, fixtures, and README. |
31+
| Tests are basically one import/version test. | False. There are 79 top-level test files under `tests/`, covering CLI, console, interaction modes, runtime, data foundations, knowledge library, idea catalog, deliverables, docs, packaging, and more. |
32+
| CI only tests Python 3.11 and 3.12. | False. `.github/workflows/ci.yml` tests 3.11, 3.12, and 3.13. The package job also uses 3.13. |
33+
| `docs/README.md` says scripts, schemas, and templates all live under `examples/`. | False now. `src/async_research_workflow/docs/README.md` has a `Package Resources` section with separate links for examples, schemas, templates, benchmark cases, domain packs, and starter templates. |
34+
| Docs have broken `docs/examples/...` links. | Not supported by the current repo checks. `tests.test_doc_references` passes and explicitly guards against stale `examples/scripts/` references. |
35+
| The operational runbook mainly tells users to run `async_research_workflow/examples/scripts/...`. | False now. The triage and readiness blocks use public `async-research ...` commands. |
36+
| README under-documents the CLI. | Mostly false now. `README.md` has a broad command map covering mode, workflow, queue, prompts, schedules, decision, source, data, library, runtime, eval, evidence memory, model routing, scaling, brief, cost, batch, metrics, accepted, outcomes, deliverable, anti-context, reflection, review, revision, result acceptance, analysis, exploration, idea, experiment, benchmark, and simulate-week. |
37+
| `CHANGELOG` and `CONTRIBUTING` are missing. | False. `CHANGELOG.md` and `CONTRIBUTING.md` both exist. `tests.test_project_metadata` enforces release hygiene files. |
38+
39+
## Real Drift To Fix
40+
41+
| Priority | Real issue | What to fix |
42+
| ---: | --- | --- |
43+
| P0 | `src/async_research_workflow/docs/autonomy_readiness_plan.md` still has readiness checklist items like `run_acceptance_suite.py`, `simulate_scheduled_week.py`, and `autonomy_readiness_gate.py research_ops`. | Replace with public commands: `async-research acceptance-suite`, `async-research benchmark`, `async-research simulate-week research_ops`, and `async-research readiness research_ops --dry-run`. |
44+
| P0 | `src/async_research_workflow/docs/operational_readiness_runbook.md` still has prose saying ``simulate_scheduled_week.py` is a no-op rehearsal`. | Rename that prose to the public command: `async-research simulate-week`. |
45+
| P1 | `src/async_research_workflow/docs/feedback_hardening_plan.md` repeatedly says items were "implemented in docs/examples." | Update wording to "implemented in package docs/scripts/templates and public CLI wrappers," or mark the section clearly as historical. |
46+
| P1 | Some docs still use direct `python -m async_research_workflow.scripts...` helper calls. Many are legitimate advanced/internal helpers, but the boundary is uneven. | Audit remaining direct helper calls. Keep only those labeled `advanced/internal`; replace any public-wrapper equivalent with `async-research ...`. |
47+
| P1 | `pyproject.toml` has `Project-URL: Roadmap` pointing to `roadmaps/delivered_public_alpha_hardening_roadmap.md`. | Point it to `roadmaps/README.md` or another current roadmap index rather than an old delivered roadmap. |
48+
| P2 | The external audit process itself is stale. | Future review prompts should tell reviewers to audit the current branch/workspace, not an old GitHub tree snapshot. |
49+
50+
## What Not To Fix
51+
52+
- Do not rewrite `src/async_research_workflow/docs/README.md` based on the
53+
report's old `examples/` claim. The current package-resource section is
54+
already much closer to the implementation.
55+
- Do not expand README solely for the command-coverage claim. The current
56+
command map is already broad.
57+
- Do not add test coverage solely because the report saw one test file. The
58+
current test suite is much larger.
59+
- Do not remove every direct `python -m async_research_workflow.scripts...`
60+
invocation blindly. Some are intentional advanced/internal helper paths.
61+
62+
## Recommended Fix Order
63+
64+
1. Replace stale user-facing script filenames in
65+
`autonomy_readiness_plan.md`.
66+
2. Replace the lingering `simulate_scheduled_week.py` prose in
67+
`operational_readiness_runbook.md`.
68+
3. Update the "implemented in docs/examples" wording in
69+
`feedback_hardening_plan.md`.
70+
4. Update the `Roadmap` project URL in `pyproject.toml`.
71+
5. Add or extend documentation-reference guards so future stale public-helper
72+
names are caught unless clearly marked as historical or advanced/internal.
73+
74+
## Bottom Line
75+
76+
The report's major drift claims are mostly already fixed. The remaining real
77+
work is smaller: clean up a few stale script-name mentions, clarify
78+
advanced/internal helper usage, and update the package metadata roadmap URL. No
79+
major framework implementation drift was confirmed from this audit.

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ Homepage = "https://github.com/dzianissokalau/async-research"
3737
Repository = "https://github.com/dzianissokalau/async-research"
3838
Issues = "https://github.com/dzianissokalau/async-research/issues"
3939
Changelog = "https://github.com/dzianissokalau/async-research/blob/main/CHANGELOG.md"
40-
Roadmap = "https://github.com/dzianissokalau/async-research/blob/main/roadmaps/delivered_public_alpha_hardening_roadmap.md"
40+
Roadmap = "https://github.com/dzianissokalau/async-research/blob/main/roadmaps/README.md"
4141

4242
[project.scripts]
4343
async-research = "async_research_workflow.cli:main"

‎src/async_research_workflow/docs/autonomy_readiness_plan.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -464,10 +464,10 @@ This order makes the workflow measurable and safe before adding more autonomous
464464

465465
The workflow is ready for controlled real loops when:
466466

467-
- `run_acceptance_suite.py` passes;
468-
- `run_autonomy_benchmark.py` passes;
469-
- `simulate_scheduled_week.py` passes;
470-
- `autonomy_readiness_gate.py research_ops` exits `0` or `2`;
467+
- `async-research acceptance-suite` exits `0`;
468+
- `async-research benchmark` exits `0`;
469+
- `async-research simulate-week research_ops` exits `0`;
470+
- `async-research readiness research_ops --dry-run` exits `0` or `2`;
471471
- no unresolved high-severity `needs_human` items remain;
472472
- source register has approved seed sources;
473473
- freshness windows are configured for accepted evidence;

‎src/async_research_workflow/docs/feedback_hardening_plan.md‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -946,27 +946,27 @@ synthesizer: weekly
946946

947947
| Priority | Item | Status |
948948
| --- | --- | --- |
949-
| P0 | atomic locking protocol | implemented in docs/examples |
950-
| P0 | state transition validator | implemented in docs/examples |
951-
| P0 | schema validation after writes | implemented in docs/examples |
952-
| P0 | structural reviewer isolation | implemented in docs/examples |
953-
| P0 | portable repo root variable | implemented in docs/examples |
954-
| P1 | revision counters | implemented in docs/examples |
955-
| P1 | algorithmic review aggregation | implemented in docs/examples |
956-
| P1 | health monitor | implemented in docs/examples |
957-
| P1 | dynamic tier escalation | implemented in docs/examples |
958-
| P1 | mission-weighted idea scoring | implemented in docs/examples |
959-
| P1 | accepted outputs index | implemented in docs/examples |
960-
| P2 | schema versioning | implemented in docs/examples |
961-
| P2 | prompt/framework versioning | implemented in docs/examples |
962-
| P2 | human decision log | implemented in docs/examples |
963-
| P2 | metrics baseline/history | implemented in docs/examples |
964-
| P2 | claim-strength re-evaluation | implemented in docs/examples |
965-
| P2 | cross-task anti-context injection | implemented in docs/examples |
966-
| P3 | batch job lifecycle | implemented in docs/examples |
967-
| P3 | programmatic cost tracking | implemented in docs/examples |
968-
| P3 | dynamic killability thresholds | implemented in docs/examples |
969-
| P3 | data source audit register | implemented in docs/examples |
949+
| P0 | atomic locking protocol | implemented in package docs/scripts/templates |
950+
| P0 | state transition validator | implemented in package docs/scripts/templates |
951+
| P0 | schema validation after writes | implemented in package docs/scripts/templates |
952+
| P0 | structural reviewer isolation | implemented in package docs/scripts/templates |
953+
| P0 | portable repo root variable | implemented in package docs/scripts/templates |
954+
| P1 | revision counters | implemented in package docs/scripts/templates |
955+
| P1 | algorithmic review aggregation | implemented in package docs/scripts/templates |
956+
| P1 | health monitor | implemented in package docs/scripts/templates |
957+
| P1 | dynamic tier escalation | implemented in package docs/scripts/templates |
958+
| P1 | mission-weighted idea scoring | implemented in package docs/scripts/templates |
959+
| P1 | accepted outputs index | implemented in package docs/scripts/templates |
960+
| P2 | schema versioning | implemented in package docs/scripts/templates |
961+
| P2 | prompt/framework versioning | implemented in package docs/scripts/templates |
962+
| P2 | human decision log | implemented in package docs/scripts/templates |
963+
| P2 | metrics baseline/history | implemented in package docs/scripts/templates |
964+
| P2 | claim-strength re-evaluation | implemented in package docs/scripts/templates |
965+
| P2 | cross-task anti-context injection | implemented in package docs/scripts/templates |
966+
| P3 | batch job lifecycle | implemented in package docs/scripts/templates |
967+
| P3 | programmatic cost tracking | implemented in package docs/scripts/templates |
968+
| P3 | dynamic killability thresholds | implemented in package docs/scripts/templates |
969+
| P3 | data source audit register | implemented in package docs/scripts/templates |
970970

971971
## What Not To Change
972972

‎src/async_research_workflow/docs/operational_readiness_runbook.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -631,6 +631,7 @@ async-research health research_ops --dry-run
631631
If any command fails, do not run autonomous jobs until the failure is resolved or
632632
intentionally paused with a human decision row.
633633

634-
`simulate_scheduled_week.py` is a no-op rehearsal: it creates a temporary ops
635-
copy, uses fixture/model-free outputs, and should report `external_api_calls=0`.
636-
Use `--keep-work-dir` only when debugging the simulated artifacts.
634+
`async-research simulate-week` is a no-op rehearsal: it creates a temporary
635+
ops copy, uses fixture/model-free outputs, and should report
636+
`external_api_calls=0`. Use `--keep-work-dir` only when debugging the simulated
637+
artifacts.

‎tests/test_doc_references.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,12 @@
8181
r"python -m\s+async_research_workflow\.scripts\.run_acceptance_suite\b"
8282
),
8383
}
84+
STALE_PUBLIC_HELPER_COMMANDS = {
85+
"run_acceptance_suite.py": "async-research acceptance-suite",
86+
"run_autonomy_benchmark.py": "async-research benchmark",
87+
"simulate_scheduled_week.py": "async-research simulate-week",
88+
"autonomy_readiness_gate.py": "async-research readiness",
89+
}
8490
INTERNAL_HELPER_MODULES = (
8591
"validate_json_artifact",
8692
"validate_transition",
@@ -314,6 +320,28 @@ def test_docs_use_public_cli_for_promoted_commands(self) -> None:
314320

315321
self.assertEqual([], failures)
316322

323+
def test_docs_do_not_use_stale_public_helper_script_names(self) -> None:
324+
failures: list[str] = []
325+
for path in iter_documentation_files():
326+
if "roadmaps/automation" in path.as_posix() and "/reviews/" in path.as_posix():
327+
continue
328+
text = path.read_text(encoding="utf-8")
329+
for helper_name, public_command in STALE_PUBLIC_HELPER_COMMANDS.items():
330+
search_from = 0
331+
while True:
332+
index = text.find(helper_name, search_from)
333+
if index == -1:
334+
break
335+
context = line_context_for_offset(text, index)
336+
if not has_historical_roadmap_label(context):
337+
line = line_number_for_offset(text, index)
338+
failures.append(
339+
f"{path.relative_to(ROOT)}:{line} uses stale {helper_name}; use {public_command}"
340+
)
341+
search_from = index + len(helper_name)
342+
343+
self.assertEqual([], failures)
344+
317345
def test_knowledge_library_docs_use_canonical_boundary(self) -> None:
318346
docs = [
319347
ROOT / "README.md",

0 commit comments

Comments
 (0)