Skip to content

Clean up baseline persistence and comparison tests - #40

Merged
charlesmartin14 merged 3 commits into
mainfrom
agent/baseline-minor-cleanup
Aug 9, 2026
Merged

Clean up baseline persistence and comparison tests#40
charlesmartin14 merged 3 commits into
mainfrom
agent/baseline-minor-cleanup

Conversation

@charlesmartin14

Copy link
Copy Markdown
Member

Scope

This is the narrowly scoped minor-cleanup pass from the baseline code review. It deliberately does not change optimizer mathematics, hyperparameters, experiment selection policy, notebook code, committed output notebooks, or existing baseline results.

Changes

  • add atomic CSV and compressed-NPZ replacement helpers for MNIST progress artifacts
  • use those helpers for per-epoch performance, spectral, WeightWatcher-detail, optimizer-group, and ESD-history persistence
  • remove the ineffective locals() assignment loop from MNIST resume handling while retaining the explicit truncation logic
  • tighten validation for seeds, optional evaluation limits, SGD dampening, Muon parameter names/Nesterov momentum, and WeightWatcher bounds/method names
  • rename the comparison result field to paired_terminal_differences, with a backward-compatible paired_final_differences property
  • add synthetic MNIST comparison tests, including complete-suite execution and duplicate-grid rejection
  • add atomic-persistence and configuration-validation tests

Validation

  • git diff --check
  • python -m compileall -q baseline/rg_baselines baseline/tests
  • python -m unittest discover -s baseline/tests -v
  • 62 tests passed
  • dependency consistency passed with pip check

The temporary workflow used to apply and validate the patch was removed; the PR's net diff contains only the six intended baseline source/test files.

Deferred to the next review

The larger findings remain intentionally untouched: one-head completed-run fingerprint validation, stricter test-holdout policy, convergence censoring, paired one-head contrasts, interruption-equivalence testing, Muon known-answer tests, and the ViT monkeypatch refactor.

@charlesmartin14
charlesmartin14 marked this pull request as ready for review August 9, 2026 21:36
@charlesmartin14
charlesmartin14 merged commit e0f6f9a into main Aug 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant