Skip to content

pp_match: count matches rather than emitting for =()= $str =~ /.../g#24558

Open
richardleach wants to merge 3 commits into
Perl:bleadfrom
richardleach:count_match_idiom
Open

pp_match: count matches rather than emitting for =()= $str =~ /.../g#24558
richardleach wants to merge 3 commits into
Perl:bleadfrom
richardleach:count_match_idiom

Conversation

@richardleach

Copy link
Copy Markdown
Contributor

A common idiom for counting the number of matches in a string is:

my $count =()= $str =~ /$pat/g;

This wasn't previously special-cased, so pp_match would emit all
matches (or captures, depending upon $pat) by copying them into
new mortal SVs, only for pp_aassign to just calculate the number
of SVs on the stack, with the new mortals freed by the next FREETMPS.

With this commit, the emitted OP tree looks more like this:

my $count = $str =~ /$pat/g;

with pp_match emitting only a single SV containing a final count
of how many SVs there previously would have been.

my $str = "Perl" x 50_000_000; my $count = () = $str =~ /Per/g; print $count, "\n";
was used to measure the difference this makes.

/usr/bin/time -v ... showed:

  • 5.43.11
	Maximum resident set size (kbytes): 4520208
	Minor (reclaiming a frame) page faults: 904658
  • Now
	Maximum resident set size (kbytes): 200460
	Minor (reclaiming a frame) page faults: 504

perf stat ... showed:

  • 5.43.11
	          4,953.15 msec task-clock                       #    0.962 CPUs utilized             
                56      context-switches                 #   11.306 /sec                      
                 0      cpu-migrations                   #    0.000 /sec                      
           905,221      page-faults                      #  182.757 K/sec                     
    23,099,997,992      cycles                           #    4.664 GHz                       
     2,260,714,999      stalled-cycles-frontend          #    9.79% frontend cycles idle      
    63,686,791,017      instructions                     #    2.76  insn per cycle            
    13,292,368,507      branches                         #    2.684 G/sec                     
       124,086,655      branch-misses                    #    0.93% of all branches 
  • Now
          1,090.87 msec task-clock                       #    0.999 CPUs utilized             
                 9      context-switches                 #    8.250 /sec                      
                 0      cpu-migrations                   #    0.000 /sec                      
               483      page-faults                      #  442.764 /sec                      
     4,937,382,721      cycles                           #    4.526 GHz                       
         9,767,979      stalled-cycles-frontend          #    0.20% frontend cycles idle      
    21,584,781,272      instructions                     #    4.37  insn per cycle            
     4,154,191,923      branches                         #    3.808 G/sec                     
           381,747      branch-misses                    #    0.01% of all branches

  • This set of changes requires a perldelta entry, and I'll add one once blead is open.

@richardleach richardleach added the defer-next-dev This PR should not be merged yet, but await the next development cycle label Jul 13, 2026
@leonerd

leonerd commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Tests are currently failing on threads (and therefore refusing to run on CI at all).

op.c:9245:41: warning: passing argument 1 of ‘S_maybe_match_count’ from incompatible pointer type [-Wincompatible-pointer-types]
 9245 |             right = S_maybe_match_count(right);

I suspect line 9245 wants to read:

    right = S_maybe_match_count(aTHX_ right);

and that should get things moving again.

@leonerd

leonerd commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Tests are currently failing on threads

As a general note on threads: If you make a habit of compiling test perls with threads, you generally discover these things as you go, whereas if you build without threads, there are a lot of macros that become empty text (most notably pTHX_ and aTHX_), and therefore it's easy to forget to add one somewhere and since the result looks identical to the C compiler, it won't notice in that setup.

I make sure to always keep that in, with:

./Configure -Dusethreads ...

@guest20

guest20 commented Jul 13, 2026

Copy link
Copy Markdown

Does this mean we need an entry in L<perlop/Operator Precedence and Associativity> for =()=†?

__
†. and perhaps a less old-internet name for the operator?

@richardleach

Copy link
Copy Markdown
Contributor Author

Tests are currently failing on threads

Thanks. Sorry about that, thought I'd already tested threaded but must have been a different PR branch.

@richardleach

Copy link
Copy Markdown
Contributor Author

Does this mean we need an entry in L<perlop/Operator Precedence and Associativity> for =()=†?

I don't think there has been any change as a result of this commit, but happy to be shown otherwise.

@richardleach
richardleach force-pushed the count_match_idiom branch 2 times, most recently from 9ae0a71 to 96e4269 Compare July 13, 2026 21:09
Comment thread pp_hot.c Outdated
@guest20

guest20 commented Jul 14, 2026

Copy link
Copy Markdown

I don't think there has been any change as a result of this commit, but happy to be shown otherwise.

I'm not suggesting that you've changed the language. I'm suggesting improving the docs.

As far as I know =()= wasn't in the interpreter until now so it was fine that it was only in a bunch of "secret operator" blog posts and talks. Now that it's reified and comes with a spiffy performance boost it might be nice if the perlop explained that doing a =()= is a better idea than shoving the matches in an array⸸ and then tossing them away

__
⸸. or does this also punch up my $count = my @array = $thing =~ /(Perl)/g too?

@jkeenan jkeenan removed the defer-next-dev This PR should not be merged yet, but await the next development cycle label Jul 15, 2026
@richardleach

Copy link
Copy Markdown
Contributor Author

As far as I know =()= wasn't in the interpreter until now so it was fine that it was only in a bunch of "secret operator" blog posts and talks. Now that it's reified and comes with a spiffy performance boost it might be nice if the perlop explained that doing a =()= is a better idea than shoving the matches in an array⸸ and then tossing them away

perlfaq4 mentions this method. Adding a note there seems worthwhile, so I'll add a commit for that.

__ ⸸. or does this also punch up my $count = my @array = $thing =~ /(Perl)/g too?

No, effectively the compiler is incapable of knowing whether or not @array (or elements of it) will be used subsequently. (At least, at the present time.) Generation of all the list elements cannot be optimized away for that case.

A new flag - OPpMATCH_JUST_COUNT - is set when the following
pattern of OPs, used to obtain a count of matches, is observed:

    $count = () = $str =~ /.../g

This is a common idiom for counting the number of times a regex
matches a target string. However, it pushes all matches to the
stack - creating a new SV to hold them - only for OP_AASSIGN
to just see how many SV*s there are and then discard them all.

With this new flag, subsequent commits can reduce - or eliminate -
the amount of mortal SV churn.
@richardleach

Copy link
Copy Markdown
Contributor Author

As far as I know =()= wasn't in the interpreter until now so it was fine that it was only in a bunch of "secret operator" blog posts and talks. Now that it's reified and comes with a spiffy performance boost it might be nice if the perlop explained that doing a =()= is a better idea than shoving the matches in an array⸸ and then tossing them away

perlfaq4 mentions this method. Adding a note there seems worthwhile, so I'll add a commit for that.

Oh, that's maintained on CPAN. I'll submit a PR upstream if/once this one gets merged.

A common idiom for counting the number of matches in a string is:

    my $count =()= $str =~ /$pat/g;

This wasn't previously special-cased, so `pp_match` would emit all
matches (or captures, depending upon `$pat`) by copying them into
new mortal SVs, only for `pp_aassign` to just calculate the number
of SVs on the stack, with the new mortals freed by the next `FREETMPS`.

With this commit, the emitted OP tree looks more like this:

    my $count = $str =~ /$pat/g;

with `pp_match` emitting only a single SV containing a final count
of how many SVs there previously _would_ have been.

`my $str = "Perl" x 50_000_000; my $count = () = $str =~ /Per/g; print $count, "\n";`
was used to measure the difference this makes.

`/usr/bin/time -v ...` showed:
* 5.43.11
```
	Maximum resident set size (kbytes): 4520208
	Minor (reclaiming a frame) page faults: 904658
```
* Now
```
	Maximum resident set size (kbytes): 200460
	Minor (reclaiming a frame) page faults: 504
```

`perf stat ...` showed:
* 5.43.11
```
	          4,953.15 msec task-clock                       #    0.962 CPUs utilized
                56      context-switches                 #   11.306 /sec
                 0      cpu-migrations                   #    0.000 /sec
           905,221      page-faults                      #  182.757 K/sec
    23,099,997,992      cycles                           #    4.664 GHz
     2,260,714,999      stalled-cycles-frontend          #    9.79% frontend cycles idle
    63,686,791,017      instructions                     #    2.76  insn per cycle
    13,292,368,507      branches                         #    2.684 G/sec
       124,086,655      branch-misses                    #    0.93% of all branches
```
* Now
```
          1,090.87 msec task-clock                       #    0.999 CPUs utilized
                 9      context-switches                 #    8.250 /sec
                 0      cpu-migrations                   #    0.000 /sec
               483      page-faults                      #  442.764 /sec
     4,937,382,721      cycles                           #    4.526 GHz
         9,767,979      stalled-cycles-frontend          #    0.20% frontend cycles idle
    21,584,781,272      instructions                     #    4.37  insn per cycle
     4,154,191,923      branches                         #    3.808 G/sec
           381,747      branch-misses                    #    0.01% of all branches
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants