Repository navigation
Running tests with --test-coverage-branches (same for lines|functions) doesn't emit test:fail when threshold isn't met #54812
Description
Activity
I'm able to reproduce. I'll have a look at this later today.
FWIW the process does exit with code 1, however all the tests do pass.
CC @nodejs/test_runner
Reacted by Roberto BianchiA
test:failevent should not be emitted for coverage issues. There is atest:coverageevent that would be more appropriate.Reacted by Aviv Keller, Benjamin Gruenbaum and Moshe AtlowA
test:failevent should not be emitted for coverage issues. There is atest:coverageevent that would be more appropriate.Mmm, well, the CHANGELOG (not the docs, since they're not updated yet) says:
If the code coverage fails to meet the specified thresholds for any category, the process will exit with code 1.
Considering that, it feels more like a
test:fail. Anyway, if there is a clear indication of coverage check failures somewhere, it'd be enough 👍🏼test:failmeans a test failed. It's already possible to end up with a failing exit code independent of coverage. For example, if a test passes, but creates asetTimeout()or other async activity that generates an error. The test has already finished and reported itself as being successful. So the error gets surfaced through a diagnostic and the exit code is set to 1. That's basically what is happening in the code coverage case as well. I agree that it should be signaled in the coverage event though.Reacted by Roberto Bianchi, Benjamin Gruenbaum and Chemi AtlowSee #54813
Reacted by Roberto Bianchi- added a commit that references this issue
on Sep 21, 2024 - added a commit that references this issue
on Oct 4, 2024
Version
22.8.0
Platform
Subsystem
No response
What steps will reproduce the bug?
I'm running Node with the following options:
and calling an executable that calls the run method as by doc.
If a test fails, a
test:failevent is properly emitted.If the coverage threshold is not met, I get a
messagefrom thetest:diagnosticevent likeError: 82.35% function coverage does not meet a threshold of 95%.(I get the same on thestdout). But notest:failevent is emitted, which makes it harder to detect it.How often does it reproduce? Is there a required condition?
Always
What is the expected behavior? Why is that the expected behavior?
test:failevent should be emitted once coverage is not meetWhat do you see instead?
no
test:failevent emittedAdditional information
same thing apply when using
--test-coverage-functionsor--test-coverage-linesoptions