Skip to content

Improve Unicode regex full-fold parity - #911

Merged
fglock merged 1 commit into
masterfrom
fix/regex-fold-test-parity
Aug 10, 2026
Merged

Improve Unicode regex full-fold parity#911
fglock merged 1 commit into
masterfrom
fix/regex-fold-test-parity

Conversation

@fglock

@fglock fglock commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • derive the complete multi-character case-fold table from the ICU4J runtime dependency
  • recognize full folds in escaped literals, reverse-fold sequences, and simple character classes
  • preserve Perl's distinct /a and /aa compiler flags and avoid rewriting across scoped (?aa:...) groups
  • add system-Perl-validated regression coverage for sharp-s, long-s, ligatures, classes, and /a versus /aa

Fold results

Test Before After Change
re/reg_fold.t 6663/7111 6925/7111 +262 passing
re/fold_grind_u.t 66/68842 66/24309 44,533 fewer emitted failures
re/fold_grind_d.t 22/11469 22/7591 3,878 fewer emitted failures
re/fold_grind_a.t 21/8609 26/2147 +5 passing, 6,462 fewer emitted failures
re/fold_grind_aa.t 18/25283 75/22811 +57 passing, 2,472 fewer emitted failures

The grind files summarize successful internal subtests but emit individual failures, so a lower denominator means additional internal cases now pass silently. op/ref.t remains unchanged at 421/481 and is outside this fold-focused change.

Validation

  • make — passed, all unit shards (5m12s)
  • prove src/test/resources/unit/regex_full_casefold.t with system Perl — 8/8 passed
  • six-file perl_test_runner.pl comparison — no timeouts; fold counts above

Derive complete multi-character folds from ICU4J, recognize escaped fold
components and character classes, and preserve the distinct /a and /aa
semantics during regex compilation.

Generated with [Codex](https://developers.openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
@fglock
fglock merged commit e9dfbaa into master Aug 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant