[Fix] run_pseudo_labelling: KeyError on dotted split name like train.360 (#96) - #172
Open
yushuosun wants to merge 1 commit into
Open
[Fix] run_pseudo_labelling: KeyError on dotted split name like train.360 (#96)#172yushuosun wants to merge 1 commit into
yushuosun wants to merge 1 commit into
Conversation
…ing)
eval_step_with_save(split) is called with the original split name (e.g.
"train.360"), and raw_datasets is keyed by that name. Line 886 rebound
`split = split.replace(".", "-").split("/")[-1]` ("train-360"), so the later
non-streaming raw_datasets[split] access raised KeyError. Use a separate
pretty_split for the output filename and keep `split` as the original key.
There was a problem hiding this comment.
Pull request overview
Fixes a crash in run_pseudo_labelling when dataset split names contain dots (e.g. train.360) in the non-streaming path by ensuring the dataset-key split name is not mutated while still producing a filesystem-friendly output filename.
Changes:
- Stop rebinding
split(dataset key) and instead introducepretty_splitfor CSV output naming. - Ensure downstream non-streaming post-processing continues to index
raw_datasets[...]with the original split key.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Pseudo-labelling a dataset whose split name contains a dot (e.g.
librispeech_asrtrain.360) crashes withKeyError: 'train-360'in the non-streaming path (#96).Root cause
In
training/run_pseudo_labelling.py,eval_step_with_save(split)receives the original split name (train.360), andraw_datasets/vectorized_datasets/file_ids_datasetare all keyed by that original name. But line 886 rebinds the variable:so the later non-streaming post-processing (
raw_datasets[split].add_column(...)) looks uptrain-360, which doesn't exist →KeyError.Modifications
training/run_pseudo_labelling.py: introduce a separatepretty_splitfor the output CSV filename and leavesplitas the original dataset key, so allraw_datasets[split]accesses resolve correctly.Duplicate-check
pulls?state=open&per_page=100 --paginate; none touch the split-key path ineval_step_with_save.train-360when not streaming. #96 has no connected/cross-referenced PR and no in-progress claim.