PHPUnit should fail if there are warnings

[marinaglancy]

When there are warnings in the phpunit results the exit code is 0.

The typical warning we receive is incorrect "@Covers" tag. The pipeline does not fail but the output shows that there are warnings:

$ . check moodle-plugin-ci phpunit --coverage-text
 RUN  PHPUnit tests for tool_tenant
Moodle 4.0 (Build: 20220419), b10a543752f72d95987b8672ecf0a1348cdc9bac
Php: 7.3.33, pgsql: 10.20 (Debian 10.20-1.pgdg90+1), OS: Linux 4.4.0-1052-aws x86_64
PHPUnit 9.5.13 by Sebastian Bergmann and contributors.
..........S.S.......................................WWWWW......  63 / 191 ( 32%)
........................................SSSSSSS................ 126 / 191 ( 65%)
.....................................................WWWWWWWWWW 189 / 191 ( 98%)
WW                                                              191 / 191 (100%)
Time: 00:33.950, Memory: 319.00 MB
There were 17 warnings:
1) tool_tenant\external_dashboard_test::test_reset_linked_dashboard
"@covers \tool_tenant\external\dashboard\" is invalid
phpvfscomposer:///builds/workplace/moodle/vendor/phpunit/phpunit/phpunit:97
2) tool_tenant\external_dashboard_test::test_reset_tenant_dashboard
"@covers \tool_tenant\external\dashboard\" is invalid
phpvfscomposer:///builds/workplace/moodle/vendor/phpunit/phpunit/phpunit:97
...
17) tool_tenant\users_report_test::test_subtenant_users_own_sub_tenant_showall
"@covers \tool_tenant\users_report" is invalid
phpvfscomposer:///builds/workplace/moodle/vendor/phpunit/phpunit/phpunit:97
WARNINGS!
Tests: 191, Assertions: 1202, Warnings: 17, Skipped: 9.
Code Coverage Report:        
  2022-04-21 20:36:07        

To test the patch I created test plugin that has warnings in @Covers directives, here is the GHA file: https://github.com/marinaglancy/moodle-tool_testpluginci/blob/master/.github/workflows/test.yml
and here is the result of the GHA: https://github.com/marinaglancy/moodle-tool_testpluginci/runs/6117617797?check_suite_focus=true

[stronk7]

Hi @marinaglancy,

would you mind applying this patch to your PR? That should solve travis problems, if I'm not wrong. Alternatively I think that I can push a commit to your branch (never have done that previously), if that's ok for you.

Ciao :-)

Here it's the patch, it's making self tests to pass here:

diff --git a/tests/Fixture/moodle-local_ci/tests/lib_test.php b/tests/Fixture/moodle-local_ci/tests/lib_test.php
index 287aa77..8143487 100644
--- a/tests/Fixture/moodle-local_ci/tests/lib_test.php
+++ b/tests/Fixture/moodle-local_ci/tests/lib_test.php
@@ -52,7 +52,7 @@ class lib_test extends \basic_testcase {
     /**
      * Test subtraction.
      *
-     * @covers ::local_ci_substract
+     * @covers ::local_ci_subtract
      */
     public function test_local_ci_subtract() {
         $this->assertEquals(0, local_ci_subtract(2, 2));
@@ -63,7 +63,7 @@ class lib_test extends \basic_testcase {
     /**
      * Test math class.
      *
-     * @covers \local_ci\math\add
+     * @covers \local_ci_math::add
      */
     public function test_local_ci_math() {
         $math = new \local_ci_math();
@@ -75,7 +75,7 @@ class lib_test extends \basic_testcase {
     /**
      * Test math class.
      *
-     * @covers \local_ci\math\add
+     * @covers \local_ci\math::add
      */
     public function test_local_ci_math_class() {
         $math = new math();

[stronk7]

Thanks Marina, I think it now looks perfect!