Fix parallel Testbench bootstrap and test profiling - #513
Merged
Conversation
Build the default parallel test application with Testbench's configured extra attributes so package discovery exclusions and other bootstrap metadata match the normal Testbench lifecycle. Keep the bootstrap confined to the default resolver and preserve an already initialized runtime. Ensure the parallel testing provider is present exactly once without disturbing configured providers. Add subprocess coverage for custom resolvers, bootstrapper configuration, provider discovery exclusions, callback ordering and counts, and runtime reuse across the worker lifecycle.
Replace the PHPUnit slow-test extension, whose warnings are lost under ParaTest, with a raw ParaTest profiler that owns a temporary JUnit report and lists every test above a documented lifecycle-duration threshold. Preserve the child runner's output and exit status, distinguish runner failures from profiler failures, clean reports on every path, and match terminal behavior. Measure package --profile durations from PHPUnit preparation so setup and teardown costs are included consistently. Add regressions for ordering, thresholds, setup timing, successful and failing suites, missing reports, report cleanup, and real multi-worker Testbench execution. Include the executable profiler in formatting and static analysis, remove the obsolete extension and subscribers, wire composer test:profile to the new script, and remove stale plan references to the deleted mechanism.
Move the ParaTest profiler into hypervel/testing as a Composer binary so package repositories can use one maintained implementation. Keep the Components script timeout disabled and preserve project-root configuration resolution when the binary is invoked through Composer's generated proxy. Preserve failing runner exit codes when reports are absent or malformed, reject unusable reports from successful runs, and provide an actionable error when the optional ParaTest dependency is missing. Keep the extensionless executable covered by formatting and static analysis while excluding only the repository's downloaded top-level binaries. Document the reserved JUnit option and root-relative paths, and cover the complete report-status matrix, missing dependency behavior, cleanup, forwarded arguments, and full test lifecycle timing.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes two problems in the parallel test tooling and makes the profiler reusable by packages that run ParaTest directly.
Testbench parallel bootstrap
Parallel package workers created their application without Testbench's configured extra attributes. Settings such as
dont-discover, configured providers, and configured bootstrappers could therefore behave differently from a normal Testbench application.The default parallel application resolver now:
The regression coverage exercises the real default resolver in subprocesses. It covers configured providers and bootstrappers, discovery exclusions, process callbacks, runtime reuse, and custom resolvers.
Accurate parallel profiling
The previous slow-test extension ran inside PHPUnit workers. ParaTest did not reliably surface those worker warnings in the parent process, so a run could finish without showing the slow tests it found.
composer test:profilenow runs ParaTest with an owned temporary JUnit report, then prints every test that meets the displayed threshold, sorted from slowest to fastest. The JUnit duration includes setup, execution, and teardown. The package-level--profiletimer now starts at PHPUnit preparation as well, so both profiling paths measure the same lifecycle.The profiler streams ParaTest output while the suite runs and preserves ParaTest's exit status for test and runner failures. A failed runner remains the primary failure even if its report is absent or incomplete. A successful run that fails to produce a readable report is rejected, and the temporary report is removed on every exit path.
Shared package binary
The profiler now ships as the
hypervel-test-profileComposer binary inhypervel/testing. Components uses the same executable as installed package consumers, avoiding separate implementations that can drift.The binary:
--log-junitfor its owned report;brianium/paratestwhen the optional parallel-test dependency is missing; andThe obsolete slow-test extension and subscribers are removed from the default PHPUnit configuration. Documentation now explains the direct package command, its root-relative paths, and its reserved report option. The extensionless executable remains covered by formatting and static analysis, while downloaded top-level binaries remain excluded.
Validation
The changes were checked with the full Components verification workflow, the Testbench package-mode and dogfood suites, focused profiler and subprocess coverage, static analysis, and formatting. The shared executable was also run through generated Composer proxies in package repositories with forwarded filters, paths, and process counts.