Inherit groups from parent classes

[stivni]

It would be very useful to me to set the group of a parent TestCase class.

class TestThatAreAlwaysGroupX extends TestCase {}
class SomeSpecificTestCase extends TestThatAreAlwaysGroupX { /* ... */ }
class AnotherSpecificTestCase extends TestThatAreAlwaysGroupX { /* ... */ }

I would love phpunit --group X to run all tests in SomeSpecificTestCase and AnotherSpecificTestCase without having to annotate them.

I see this achievable by annotating TestThatAreAlwaysGroupX and/or by doing this programmatically in the getGroups() of the TestCase. Both ways currently don't work.

Would anyone else have this need?
Could someone point me in the right direction for implementing this?

Thanks!

[epdenouden]

Hi @stivni !

This sounds useful to me and I think this already works for annotations from the parent class of a test.

It's possible yet not trivial. There was some recent work on an annotation cache done during the recent code sprint, I will check this weekend and report back.

[spawnia]

This is a feature that i would be interested in, too. Filtering test execution based on the parent class of test classes is a natural way of grouping tests together.

I am not sure if piggybacking on groups is optimal. It could be better to add a new flag to the CLI that allows filtering tests based on their parent class.

For context, a similar question was asked on StackOverflow: https://stackoverflow.com/questions/56198699/run-phpunit-tests-by-testcase-class-that-they-extend

[mrtus]

There was some recent work on an annotation cache done during the recent code sprint, I will check this weekend and report back.

@epdenouden Any feedback on this? I'm curious since this would help us out big time.

[mrtus]

@sebastianbergmann Maybe you know if there is any feedback on this issue?

[andy-educake]

We just came across a use case for this feature. Is it something that would be desirable? I would be willing to try and create a PR for it if needed.

[EdmondDantes]

I just ran into this challenge today!
the right idea!

[sebastianbergmann]

@sebastianbergmann Maybe you know if there is any feedback on this issue?

I am sorry to say this, but me this does not sound useful.

[andy-educake]

So our use case is this: alongside our "pure" unit tests we have some tests that use the database plus some tests that test integrations with 3rd parties. In both these cases we have a specific BaseTestCase which does the common setup, which we then inherit from. It would be really handy to be able to annotate those BaseTestCases with a particular group so that all tests inheriting from it would be part of that group (database/integration/whatever). Otherwise we have to rely on those tests being annotated manually and being subject to human error.

[EdmondDantes]

So our use case is this: alongside our "pure" unit tests we have some tests that use the database plus some

Quite right.
now I have to either:
1.define test groups in XML (I don't believe lazy programmers will do it manually)
2. use annotations for each method. And this is very inconvenient when the entire module should automatically fall into the required test group.

[spawnia]

Good point @andy-educake.

In short, the selection of which tests to execute naturally coincides with a commonality in their setup, for which inheritance of test classes is the most commmonly used implementation.

[Riimu]

I was just looking into feature like this myself. Basically in our project, we have roughly two kinds of tests (with couple subtypes) in our project:

• Unit tests
• Integration tests

The main difference between these two tests are that unit tests cannot use database/redis/queues and must mock all external connections. Integration tests, however, have may have external services available.

Currently, in our test setup I have enforced this by overriding the getName() method on the parent classes to provide either [UnitTest] or [Integration] suffix to the test name. Then on github actions, I use --filter="[UnitTest]" to only run unit tests in a step that does not provide the external resources (and integration tests in other step that does provide the external service containers).

Our code base is pretty much plagued by all kinds of static calls (partly thanks to active record like DB pattern), so the purpose of these tests to better ensure that each test only does what it's supposed to. In other words, limiting the available external services makes it somewhat easier to limit the scope of some of the tests.

The current solution "kind of" works, but it's hardly ideal. It also breaks some IDE integrations due to changes in test names. Being able to define groups in parent classes or somehow trivially separate the tests into separate test suites based on parent class would be much better solution.

[Riimu]

After posting the previous comment, thinking a bit more about a better solution and looking into PHPUnit source code, I noticed that the code calls method suite() on test classes by default to get the test suite, which allows way to override the test suite class and inject groups via that way. Essentially, my (non-ideal) solution for now is to have the parent test case implement the suite method like:

  public static function suite(string $class): Test {
    return new CustomTestSuite($class);
  }

And then the test suite class is implemented like:

class CustomTestSuite extends TestSuite {
  public function addTest(Test $test, $groups = []): void {
    if ($test instanceof BaseTestCase) {
      $groups[] = $test instanceof IntegrationTest
        ? 'integration-test'
        : 'unit-test';
    }

    parent::addTest($test, $groups);
  }
}

Now I can, for example, run only the unit tests with ./vendor/bin/phpunit --group unit-test.

Edit: Note, however, that this seems to break running indivual tests from classes in IntelliJ due to bug: https://youtrack.jetbrains.com/issue/WI-58064

[sebastianbergmann]

I can see how this would be useful, but adding support for metadata inheritance could lead to backward compatibility issues. Therefore I have decided to not implement this.

To solve the use cases for the requested feature mentioned above, I suggest the approaches outlined below.

Using the XML configuration file, you can create separate test suites. See https://github.com/sebastianbergmann/raytracer/blob/9b24095dd535e40401e613d44ddb0ee82c755b66/phpunit.xml#L17-L23 for an example.

Also using the XML configuration file, you can specify a comma-separated list of groups the tests found in a directory should be added to:

tests
├── a
│   └── OneTest.php
└── b
    └── TwoTest.php

phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
>
    <testsuites>
        <testsuite name="default">
            <directory groups="foo,baz">tests/a</directory>
            <directory groups="bar,barbara">tests/b</directory>
        </testsuite>
    </testsuites>
</phpunit>

tests/a/OneTest.php

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class OneTest extends TestCase
{
    public function testOne(): void
    {
        $this->assertTrue(true);
    }
}

tests/b/TwoTest.php

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class TwoTest extends TestCase
{
    public function testTwo(): void
    {
        $this->assertTrue(true);
    }
}

❯ ./vendor/bin/phpunit --list-groups
PHPUnit 11.3.3 by Sebastian Bergmann and contributors.

Available test group(s):
 - bar
 - barbara
 - baz
 - default
 - foo

I consider the fact that tests that are added to groups via the XML configuration file are also added to the default group a bug.