concurrency-probe: TTFT + prefill columns (already-measured, never reported) - #808
Open
noonghunna wants to merge 1 commit into
Open
concurrency-probe: TTFT + prefill columns (already-measured, never reported)#808noonghunna wants to merge 1 commit into
noonghunna wants to merge 1 commit into
Conversation
…in verdict/RESULT Per-request TTFT was already measured (streaming t_first) but never reported. Adds ttft_ms + pf_t/s columns to round rows, a concurrent-prefill line to the verdict, and ttft_ms/pf_tps fields to the machine-readable RESULT line. Field-validated: exposed the N=4 mixed-regime split (60.2 steady vs 16.3 during simultaneous 4x10K loads, TTFT 33.6s). Groundwork for #805. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4LeYTBtjVXgpDgXSNUSgr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The probe measures per-request TTFT internally (streaming first-token time) but never surfaced it. Adds:
ttft_ms+pf_t/sper-round columns (prefill derived asPROMPT_TOKENS/ttft, bench.sh convention), aconcurrent prefill:verdict line, and additivettft_ms=/pf_tps=fields on the RESULT line (SWEEP-parse compatible).Field-validated on the Laguna concurrency campaign: the deep-prompt rung exposed the mixed-regime split (60.2 tok/s aggregate steady decode vs 16.3 with 33.6 s TTFT during simultaneous 4×10K context loads) that pure-decode reporting hides. Groundwork for the #805 rebench-full concurrency step.
🤖 Generated with Claude Code
https://claude.ai/code/session_01B4LeYTBtjVXgpDgXSNUSgr