Skip to content

chore!: drop verifiers v0 environment support - #3219

Open
hallerite wants to merge 3 commits into
mainfrom
chore/drop-verifiers-v0-envs
Open

chore!: drop verifiers v0 environment support#3219
hallerite wants to merge 3 commits into
mainfrom
chore/drop-verifiers-v0-envs

Conversation

@hallerite

@hallerite hallerite commented Aug 8, 2026

Copy link
Copy Markdown
Member

Summary

  • remove legacy v0 environment config, loading, and server-side index scheduling
  • standardize orchestration on verifiers v1 tasksets, typed task data, and Env.run
  • simplify rollout permits, cancellation, buffering, and metrics around the v1 episode model
  • update surviving fixtures for v1 tasksets
  • preserve current main behavior for externally managed per-source env servers

Why

Verifiers v1 makes the taskset client-owned and allows one episode to emit multiple traces. Keeping the v0 group/index path alongside it duplicated scheduling and accounting behavior. Prime's maintained configs already use v1 tasksets, so this change makes that model the only supported path.

Breaking change

Legacy [legacy] environment blocks are no longer accepted. Environment packages must expose a verifiers v1 taskset and configure it through the environment's taskset block.

Validation

  • Ruff lint passed on the 13 changed Python files
  • Ruff format check passed on the 13 changed Python files
  • Self-contained config and orchestrator metrics tests: 55 passed, 64 deselected
  • The full isolated config-file scan was also attempted: its remaining 27 cases could not import optional taskset packages such as alphabet-sort-v1, wordle-v1, and wiki-search-v1; no merge-related assertion failures were observed

Note

High Risk
Breaking config API and a wide refactor of rollout scheduling, permits, and group finalization on the critical train/eval path.

Overview
Removes classic verifiers v0 ([legacy]) environments so train/eval and env-server only accept v1 tasksets under env.taskset.

Config models drop legacy, is_legacy, and v0/v1 mixing validation; env_id comes from the v1 env and startup fails without a taskset id. The RL launcher no longer writes legacy into per-source env-server TOMLs, and env-server always calls serve_env with config_data instead of the legacy bridge (env_id, extra_env_kwargs).

Orchestration is unified on Env.run + task_data: client-side taskset loading only, no run_group / requires_group_scoring, and the dispatcher treats one in-flight permit per run (not group_size upfront). GroupState.task is required; train/eval sources no longer use variable permit costs or legacy index rows. Train sink drops group-scoring “partial group” discard; buffering/cancellation/metrics assume the v1 episode model. Unit config fixtures switch from legacy.id to reverse-text-v1 tasksets.

Reviewed by Cursor Bugbot for commit 7f33e97. Bugbot is set up for automated code reviews on this repo. Configure here.

…v0-envs

# Conflicts:
#	src/prime_rl/entrypoints/rl.py
#	tests/unit/train/rl/test_packer.py
#	tests/unit/train/test_runs.py
@hallerite
hallerite marked this pull request as ready for review August 10, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant