perf!: remove the zero-gradient-ratio metric

[samsja]

Summary

Removes the zero_grad_ratio metric entirely: count_zero_gradient_elements / get_zero_gradient_ratio, both trainer call sites, the optim/zero_grad_ratio log key, and the trainer_zero_grad_ratio Prometheus gauge.

Why remove it:

1. The metric appears broken: it reported zero_grad_ratio = 1.0 (all gradients zero) on every validation run — Qwen3-30B-A3B actively training on 8×H200 — so it is likely not reading FSDP2 DTensor gradients correctly.
2. It's pure per-step overhead: a Python loop with a host→device scalar copy and a count_nonzero kernel per parameter tensor (thousands on MoE models), two all-reduces, and an .item() sync, every step.

Measured cost (Qwen3-30B-A3B, 8×H200, fake data, no offload, paired same-session runs, median of steps 2–5):

	seq 8K	seq 16K
with scan	3.47 s	5.84 s
without scan	3.18 s	6.23 s
scan cost	0.29 s (8% of step)	within noise

An earlier revision of this PR claimed ~1.4 s/step based on a cross-session comparison; that did not replicate in paired runs and is withdrawn — the honest number is ~8% at short sequences on many-tensor models, shrinking into noise as compute grows. The primary motivation is the metric's unreliable output.

🤖 Generated with Claude Code

---

Note

Low Risk
Observability-only removal with no changes to optimizer, loss, or gradient math; dashboards/alerts referencing optim/zero_grad_ratio or trainer_zero_grad_ratio will stop updating.

Overview
Removes the zero_grad_ratio observability path end-to-end: count_zero_gradient_elements / get_zero_gradient_ratio in trainer/utils.py, the post–grad-clip computation in RL and SFT trainers, the optim/zero_grad_ratio monitor key, and the trainer_zero_grad_ratio Prometheus gauge plus MetricsServer.update(..., zero_grad_ratio=...).

Training steps no longer pay the per-parameter gradient scan (Python loop, count_nonzero, all-reduces, sync) that existed only to populate that metric. Related unused imports (Iterable, nn, DTensor) are dropped with the helper code.

Note: docs/training.md still documents optim/zero_grad_ratio; that file is unchanged in this diff.

^{Reviewed by Cursor Bugbot for commit 95261f8. Bugbot is set up for automated code reviews on this repo. Configure here.}