Version: 0.2.18
In the browser runner, afterEach hooks run inside the try block, after await t.fn():
await t.fn(); // test body
for (const suite of [...chain].reverse()) {
for (const hook of suite.afterEach) await hook(); // inside the try
}
...
} catch (e) { ...failed++ }
So when a test throws, its afterEach hooks are skipped entirely. Any per-test cleanup — restoring stubs, clearing DOM — silently doesn't happen, and the damage compounds into later tests.
Real case (crank #375, webkit): exactly one genuine failure existed, but 26 of the suite's files stub console.error in beforeEach and restore it in afterEach. The failing test skipped its restore, so every subsequent test that stubbed got Sinon's "Attempted to wrap error which is already wrapped", and the run reported 23 failures for 1 bug. The 22 phantom messages also buried the real failure's actual assertion text.
node:test and bun:test both run afterEach hooks even when the test fails (and report hook errors separately). The browser runner should match: run afterEach in a finally, keeping the test's own error as the reported failure if both throw.
(beforeEach failing → skipping the test body is fine and matches the other runtimes; it's only after-hooks that need the guarantee.)
Found while diagnosing the webkit failure now fixed on the crank side (bikeshaving/crank#379); related to the firefox work headed for 0.2.19 — would be good company in that release.
Version: 0.2.18
In the browser runner,
afterEachhooks run inside thetryblock, afterawait t.fn():So when a test throws, its
afterEachhooks are skipped entirely. Any per-test cleanup — restoring stubs, clearing DOM — silently doesn't happen, and the damage compounds into later tests.Real case (crank #375, webkit): exactly one genuine failure existed, but 26 of the suite's files stub
console.errorinbeforeEachand restore it inafterEach. The failing test skipped its restore, so every subsequent test that stubbed got Sinon's "Attempted to wrap error which is already wrapped", and the run reported 23 failures for 1 bug. The 22 phantom messages also buried the real failure's actual assertion text.node:test and bun:test both run afterEach hooks even when the test fails (and report hook errors separately). The browser runner should match: run
afterEachin afinally, keeping the test's own error as the reported failure if both throw.(
beforeEachfailing → skipping the test body is fine and matches the other runtimes; it's only after-hooks that need the guarantee.)Found while diagnosing the webkit failure now fixed on the crank side (bikeshaving/crank#379); related to the firefox work headed for 0.2.19 — would be good company in that release.