Version 7.1.0, @BeforeGroups and @AfterGroups Annotations are not working #2229

MahmoudElSharkawy · 2020-01-21T20:02:05Z

I believe that this issue is a regression as they are working normally on version 7.0.0

TestNG Version

7.1.0

Expected behavior

The @BeforeGroups and @AfterGroups should be working properly when triggering the execution from a TestNG class, xml file or surefire plugin.

Actual behavior

The @BeforeGroups and @AfterGroups are not working when executing the the run from a TestNG class or from the surefire plugin, only works when the test is triggered from a testng.xml file when I include the group name in the run parameter.

Is the issue reproductible on runner?

I'm running the test class directly via the IDE [Eclipse] and via the cmd using surefire plugin .

When running the test class directly via the IDE >> the issue happens and the Groups annotations doesn't work.
When running from the xml file while including the groups names in the run parameter >> the Groups annotations works fine.

Same happens with maven surefire plugin.

Sample code to reproduce:

package testNG;

import org.testng.annotations.AfterGroups;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.Test;

public class TestGroupsAnnotations {

	@BeforeGroups(groups = "testGroup1")
	public void beforeGroups1() {
		System.out.println("Test Before Groups Annotation");
	}
	
	@AfterGroups(groups = "testGroup1")
	public void afterGroups1() {
		System.out.println("Test After Groups Annotation");
	}


	@Test(groups = "testGroup1")
	public void test1() {
		System.out.println("Test #1");
	}
	@Test(groups = "testGroup2")
	public void test2() {
		System.out.println("Test #2");
	}
	@Test(groups = "testGroup1")
	public void test3() {
		System.out.println("Test #3");
	}
	@Test(groups = "testGroup2")
	public void test4() {
		System.out.println("Test #4");
	}
	
	
	@BeforeGroups(groups = "testGroup2")
	public void beforeGroups2() {
		System.out.println("Test Before Groups Annotation #2");
	}
	
	@AfterGroups(groups = "testGroup2")
	public void afterGroups2() {
		System.out.println("Test After Groups Annotation #2");
	}
	
}

Version 7.1.0 behavior

Version 7.0.0 behavior

MohabMohie · 2020-01-21T20:08:39Z

I can reproduce the same issue on my environment as well.
Eclipse (12-2019) + JDK 13 + Maven

Working on TestNG 7.0.0
Not working on TestNG 7.1.0

krmahadevan · 2020-01-27T09:51:11Z

@MahmoudElSharkawy - Can you please help with the following:

Please provide a sample that can be used to recreate this issue.
Please share information on how are you running the test. Is it via the IDE or is it via maven surefire plugin that uses a testng suite xml file.

MahmoudElSharkawy · 2020-01-27T10:25:08Z

@MahmoudElSharkawy - Can you please help with the following:

1. Please provide a sample that can be used to recreate this issue.

2. Please share information on how are you running the test. Is it via the IDE or is it via maven surefire plugin that uses a testng suite xml file.

@krmahadevan
I'm running the test class directly via the IDE [Eclipse] (Right click >> Run As TesNG Test) .

When running the test class directly via the IDE >> the issue happens and the Groups annotations doesn't work.
When running from the xml file while including the groups names >> the Groups annotations works fine.

Same happens with maven surefire plugin.

Sample code:

package testNG;

import org.testng.annotations.AfterGroups;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.Test;

public class TestGroupsAnnotations {

	@BeforeGroups(groups = "testGroup1")
	public void beforeGroups1() {
		System.out.println("Test Before Groups Annotation");
	}
	
	@AfterGroups(groups = "testGroup1")
	public void afterGroups1() {
		System.out.println("Test After Groups Annotation");
	}


	@Test(groups = "testGroup1")
	public void test1() {
		System.out.println("Test #1");
	}
	@Test(groups = "testGroup2")
	public void test2() {
		System.out.println("Test #2");
	}
	@Test(groups = "testGroup1")
	public void test3() {
		System.out.println("Test #3");
	}
	@Test(groups = "testGroup2")
	public void test4() {
		System.out.println("Test #4");
	}
	
	
	@BeforeGroups(groups = "testGroup2")
	public void beforeGroups2() {
		System.out.println("Test Before Groups Annotation #2");
	}
	
	@AfterGroups(groups = "testGroup2")
	public void afterGroups2() {
		System.out.println("Test After Groups Annotation #2");
	}
	
}

krmahadevan · 2020-01-27T10:35:20Z

@MahmoudElSharkawy - Looks like this is a problem with the TestNG eclipse plugin.

You might want to log this issue in https://github.com/cbeust/testng-eclipse/issues

Same happens with maven surefire plugin.

I didn't quite understand this part. Does this mean that you are using mvn clean test -Dtest= TestGroupsAnnotations ? If yes, then this wont work like that because you aren't specifying what groups to be run. In that case, the group mode won't be even selected.

MahmoudElSharkawy · 2020-01-27T10:52:27Z

@MahmoudElSharkawy - Looks like this is a problem with the TestNG eclipse plugin.

You might want to log this issue in https://github.com/cbeust/testng-eclipse/issues

Same happens with maven surefire plugin.

I didn't quite understand this part. Does this mean that you are using mvn clean test -Dtest= TestGroupsAnnotations ? If yes, then this wont work like that because you aren't specifying what groups to be run. In that case, the group mode won't be even selected.

@krmahadevan
I mean the behavior is the same when I run from the IDE and the test class directly and when I run from the surefire plugin.
If I uderstand this right, using the @BeforeGroups & @AfterGroups annotations should be working when I run the test class from the IDE or with surefire plugin with the same behavior.. It will however work as you said if I'm not using those annotations and only specifying the groups names for the @test test cases only; Here I need to specify what groups to run via the xml file.

Please find below two screenshots from both TestNG version from the surefire plugin execution (Both and from the same sample provided in the previous comment)

7.1.0

7.0.0

Nikasan · 2020-03-02T10:20:54Z

Reproduced the same issue: TestNG (7.1.0) +Gradle
sample code:

public class classForExperiments {
@BeforeGroups(groups = "IT Department")
    public void before_it()
    {
        System.out.println("This method will be executed before the execution of IT Department group");
    }
    @Test
    public void testcase1()
    {
        System.out.println("HR");
    }
    @Test(groups= {"IT Department"})
    public void testcase2()
    {
        System.out.println("Software Developer");
    }
    @Test(groups= {"IT Department"})
    public void testcase3()
    {
        System.out.println("QA Analyst");
    }
}

Expected behavior:
HR
This method will be executed before the execution of IT Department group
Software Developer
QA Analyst

Actual:
HR
Software Developer
QA Analyst

Run from IDEA (last version)

MikeLaptev · 2020-04-23T17:17:50Z

Looks like I've faced a similar issue after upgrading to 7.1.0

EyalSooliman · 2020-06-03T15:26:38Z

I get the same issue after upgrading to 7.1 on maven. org.testng fail to load the dependency files.

krmahadevan · 2020-06-18T08:43:12Z

Folks,
The behavior of TestNG was changed as part of 7.1.0 via this PR #2167

By default group selection is not applicable until and unless its explicitly chosen via the suite xml file.

Prior to TestNG 7.1.0 if there was a @BeforeGroups/@AfterGroups annotation, that method was getting executed irrespective of whether group selection was applied or not.

This behavior has been changed and now @BeforeGroups/@AfterGroups annotated methods would get executed only when group filtering is turned on via the suite xml file.

So if this behavior is needed to be enabled, please create a suite xml file, resort to selection of groups adding

<groups>
      <run>
        <include name="group-one"/>
      </run>
</groups>

in suite xml file and then using it.

Closing this issue.

MohabMohie · 2020-06-18T23:47:50Z

@krmahadevan is there another way for those of us who don't use a suite xml?
My solution is built on top of using the surefire plugin, and passing the tests parameter to create a suite in the runtime... so is there a way to do something by code to get the old behavior? like a property that I can set or something?

krmahadevan · 2020-06-19T03:42:29Z

@MohabMohie - This wasn't a regression to begin with. This was a bug that always existed in TestNG which was fixed. So there's no way of doing this, unless and until, you know what the groups are. If that knowledge is there, then you can very well use an implementation of IAlterSuiteListener wherein you can weave in the group information programmatically into the suite and then work with it. There's no property that can be used to toggle this.

On a side note, I would be curious to know as to what exists within these @BeforeGroups and @AfterGroups ? It sounds like they perhaps are in the wrong place to begin with. If its a setup/tear down that you would like to be executed all the time, you can perhaps put them within an implementation of IInvokedMethodListener which you can wire in via a SPI mechanism ?

If you can add more details (please feel free to be as elaborate as possible) I can suggest some alternatives.

martinaldrin · 2021-10-20T16:28:17Z

Hi,

I think we also have phased this issue now when we Testing TestNg 7.5, we are using 6.14 at the moment and are working on uplifting to latest TestNg, but it seems not possible due to this change. We have over 500 Projects and thousands of test cases in different organizations and thousands developers. It will never be possible to adapt all of the Test and as I see it is not even a possible alternative to this change.

In many Test Cases we using groups to run different Before/After methods/class and we have test within the same test class with different groups and without groups. As I understand now we must specify the groups in the suite, but that will not work for us.

I don't think this was a bug correction, I think that removing a very useful feature from TestNg that simplifies test cases allot.

MahmoudElSharkawy · 2021-10-20T16:35:54Z

Hi,

I think we also have phased this issue now when we Testing TestNg 7.5, we are using 6.14 at the moment and are working on uplifting to latest TestNg, but it seems not possible due to this change. We have over 500 Projects and thousands of test cases in different organizations and thousands developers. It will never be possible to adapt all of the Test and as I see it is not even a possible alternative to this change.

In many Test Cases we using groups to run different Before/After methods/class and we have test within the same test class with different groups and without groups. As I understand now we must specify the groups in the suite, but that will not work for us.

I don't think this was a bug correction, I think that removing a very useful feature from TestNg that simplifies test cases allot.

Thank you for stating this. I totally agree; It was a very useful feature and it's removed without introducing any useful alternatives. For what I know and understand, Annotations are meant to run in the class and no need for suits to be configured.

@krmahadevan

martinaldrin · 2021-10-20T16:53:14Z

We have been working hard for long time to get correction on master and PR merged and then I suddenly met this issue and adapt to other Non backward compatible changes. I'm surprised that a very nice feature is treated as a bug and removed. this makes it extremely hard to do an uplift now. We using groups in plenty places without specify them in a suite, and for regular unit tests we don't even have suites but still uses groups.

In the project I'm working in we are never allowed to introduce an Non Backward Compatible change, since different projects fetched our project automatically, and if the test don't pass it will not be uplifted to any other project and get verdict -1. The projects are complied with our Framework after all tests are passed, so even biniary Non Backward compatible changes could be extremely hard to solve. So the only way to handle such change is if we could adapt all 500 projects in advanced and make them compatible with both TestNg 6.14 and 7.5 otherwise any change will not be allowed.

Today we do forking on 6.14 and 7.5 to adapt to new TestNg 7.5 to allow a smoother migration, but this is really a blocker for us.

cedricbeust-okta · 2021-10-20T22:04:02Z

@martinaldrin @MahmoudElSharkawy @MohabMohie

Really sorry for the inconvenience.

@krmahadevan Could the old behavior be implemented with listeners?

krmahadevan · 2021-10-21T11:48:55Z

@cedricbeust-okta - Sure I will take a look at this and find out what can be done in terms of this.

juherr · 2021-10-21T11:52:15Z

Could someone summarize the expected behavior?

If I understand well, do you want a way to run @BeforeGroups/@AfterGroups event if there is no group filtered?

@krmahadevan Why not consider that all groups are requested when none is specified? It is already the behavior for @Test methods, right?

krmahadevan · 2021-10-21T11:57:27Z

@juherr

If I understand well, do you want a way to run @BeforeGroups/@AfterGroups event if there is no group filtered?

This is basically what I am trying to understand as well. The BeforeGroups and AfterGroups is basically intended to be setup/tear downs that are to be done at the level of groups. When there is no group filtering enabled then there's no reason why these configurations need to be executed.

But there's one caveat. These two annotations do have a alwaysRun attribute which when enabled should make sure that these configurations run all the time (atleast that's my understanding and expectation) and that doesn't seem to be happening properly which I think we need to fix. I am trying to figure that part which if we address should solve all the issues because that attribute should be toggleable via an annotation transformer.

Why not consider that all groups are requested when none is specified?
I am not sure if that would be intuitive. Instead we can perhaps just have the alwaysRun attribute honoured properly and get that to work fine which would be more intuitive (by default this attribute is disabled)

It is already the behavior for @test methods, right?

Not sure if thats true because if you specify a group then that filtering gets applied, if you dont specify groups then no filtering is applied. But these two config annotations have a special need which is to setup and tear down only at group level. If they need to be run for everything, then ideally one should be using the BeforeSuite|AfterSuite or BeforeTest|AfterTest.

juherr · 2021-10-21T12:18:33Z

These two annotations do have a alwaysRun attribute which when enabled should make sure that these configurations run all the time (atleast that's my understanding and expectation) and that doesn't seem to be happening properly which I think we need to fix. I am trying to figure that part which if we address should solve all the issues because that attribute should be toggleable via an annotation transformer.

It sounds good 👍

martinaldrin · 2021-10-21T13:53:00Z

Hi,

Here is an simple test example on a unit test without a suite, there might be many more cases.
Today Before and after groups are ignored, but the test methods are executed which feels very wrong.

The benefit of using Before/AfterGroups instead of Before/AfterMethod is that they are only executed once per group and not before every method. This enable that we can do time costly operations only once per group, e.g. setup a connection that is needed only for that group.

public class Demo {

	boolean valueA = false;
	boolean valueB = false;
	
	@BeforeGroups(groups = "groupA")
	public void beforeGroupA() {
		valueA = true;
	}
	
	@BeforeGroups(groups = "groupB")
	public void beforeGroupB() {
		valueB = true;
	}
	
    @Test
    public void testA() {
        System.out.println("TestA");
    }
    
    @Test
    public void testB() {
        System.out.println("TestB");
    }
    
    @Test
    public void testC() {
        System.out.println("TestC");
    }
    
      @Test(groups = "groupA")
    public void testGroupA1() {
       Assert.assertTrue(valueA, "BeforeGroupA was not executed");
    }
    
    @Test(groups = "groupA")
    public void testGroupA2() {
       Assert.assertTrue(valueA, "BeforeGroupA was not executed");
    }
    
    @Test(groups = "groupA")
    public void testGroupA3() {
       Assert.assertTrue(valueA, "BeforeGroupA was not executed");
    }
    
    @Test(groups = "groupB")
    public void testGroupB() {
    	Assert.assertTrue(valueB, "BeforeGroupB was not executed");
    }
    
    @AfterGroups(groups = "groupA")
	public void afterGroupA() {
		valueA = false;
	}
	
    @AfterGroups(groups = "groupB")
	public void afterGroupB() {
		valueB = false;
	}
    

    @AfterClass
    public void afterClass() {
    	Assert.assertFalse(valueA, "AfterGroupsA was not executed");
    	Assert.assertFalse(valueB, "AfterGroupsB was not executed");
    }
    
}

Test result: 6.14.x

Test result: 7.5

martinaldrin · 2021-10-21T17:09:38Z

Regarding the alwaysRun, that will not solve it for us, even if you fix it. Since we should only run the before/afterGroup if test methods exists.

Here is one more example, why before/afterGroup is very useful. If I add a groupC and not have a test method that depends on that group, BeforeGroup C will not run.

The group in the suite file we use to enable/disable different groups, but if not added in suite file all groups with test methods will be executed. If we only want to run group we specify that group in the suite, or exclude one group.
The group filtering in the suite work both with and without before/afterGroup. So we don't have that connection between these configuration methods as we have between Before/AfterTest, which is tied to the suite.

public class Demo {

	boolean valueA = false;
	boolean valueB = false;
	
	@BeforeGroups(groups = "groupA")
	public void beforeGroupA() {
		System.out.println("beforeGroupA");
		valueA = true;
	}
	
	@BeforeGroups(groups = "groupB")
	public void beforeGroupB() {
		valueB = true;
		System.out.println("beforeGroupB");
	}
	
	@BeforeGroups(groups = "groupC")
	public void beforeGroupC() {
		System.out.println("beforeGroupC No Test exist, should not run.");
	}
	
    @Test
    public void testA() {
        System.out.println("TestA");
    }
    
    @Test
    public void testB() {
        System.out.println("TestB");
    }
    
    @Test
    public void testC() {
        System.out.println("TestC");
    }
    
    @Test(groups = "groupA")
    public void testGroupA1() {
    	 System.out.println("testGroupA1");
       Assert.assertTrue(valueA, "BeforeGroupA was not executed");
    }
    
    @Test(groups = "groupA")
    public void testGroupA2() {
    	System.out.println("testGroupA2");
       Assert.assertTrue(valueA, "BeforeGroupA was not executed");
    }
    
    @Test(groups = "groupA")
    public void testGroupA3() {
    	System.out.println("testGroupA3");
       Assert.assertTrue(valueA, "BeforeGroupA was not executed");
    }
    
    @Test(groups = "groupB")
    public void testGroupB() {
    	System.out.println("testGroupB");
    	Assert.assertTrue(valueB, "BeforeGroupB was not executed");
    }
    
    @AfterGroups(groups = "groupA")
	public void afterGroupA() {
    	System.out.println("afterGroupA");
		valueA = false;
	}
	
    @AfterGroups(groups = "groupB")
	public void afterGroupB() {
    	System.out.println("afterGroupB");
		valueB = false;
	}
    

    @AfterClass
    public void afterClass() {
    	Assert.assertFalse(valueA, "AfterGroupsA was not executed");
    	Assert.assertFalse(valueB, "AfterGroupsB was not executed");
    }
    
}

The execution order, show how great this feature is in TestNg 6.14.x, beforeGroupC is excluded since not test exists and the before methods are executed just before the tests

TestA
TestB
TestC
beforeGroupA
testGroupA1
testGroupA2
testGroupA3
afterGroupA
beforeGroupB
testGroupB
afterGroupB

juherr · 2021-10-21T20:14:51Z

As @krmahadevan explained, groups are used to select test methods.
As I understand, the previous behavior was a side-effect of an issue.

I agree the feature you want (@Before/AfterGroups methods activation depending on the tests in the suite) is great but has never existed in TestNG before.

juherr · 2021-10-22T07:29:25Z

Today we do forking on 6.14 and 7.5 to adapt to new TestNg 7.5 to allow a smoother migration, but this is really a blocker for us.

@martinaldrin Instead, why not move the forking effort into the implementation of the feature you want?

martinaldrin · 2021-10-22T07:37:26Z

Hi @juherr, forking we only do to adapt to master and waiting for PR to be merged, we don't want to maintain forks. We want to use official releases. And I also understand that many other want to get back the original behavior, since it breaks many test cases and makes it hard to uplift. And the Before/AfterGroups is a very useful feature that we don't want to lose.

martinaldrin · 2021-10-22T09:54:03Z

I agree the feature you want (@Before/AfterGroups methods activation depending on the tests in the suite) is great but has never existed in TestNG before.

It have exists since we use it, and many other uses it. And in the documentation there is no information that Before/AfterGroups should only work if a group is defined in the suite. I hope we can get this functionality back in a similar way. Since that will help me and many other to easier migrate to latest TestNg.

Elisedlund-ericsson · 2021-10-22T10:00:53Z

I also have issue with this missing on the latest testng (and its forsure working in previous TestNG versions!)

I find it very stupid and strange that it will run the test case in with group:

@Test(groups = "testGroup1")
public void test1() {
	System.out.println("Test #1");
}

but not run the before/after methods:

@BeforeGroups(groups = "testGroup1")
public void beforeGroups1() {
	System.out.println("Test Before Groups Annotation");
}

@AfterGroups(groups = "testGroup1")
public void afterGroups1() {
	System.out.println("Test After Groups Annotation");
}

if it did not run the "@test(groups = **"testGroup1")" because that group was not selected I would be more understanding,
but as it is now it runs that test case and then I expect that it would run the @BeforeGroups/@AfterGroups(groups = "testGroup1") which I consider the test case have dependencies towards,

as the "@test(groups = "testGroup1")" is running, to me that is the same thing as the group being active and therefor all the groups dependencies should also run

martinaldrin · 2021-10-22T10:05:18Z

I totally agree with @Elisedlund-ericsson. it is very strange that only Before/AfterGroups are excluded if not defined in suite xml but for all other methods they are executed. But the group filter in the suite xml applies on all methods that have the group, not only Before/AfterGroups.

krmahadevan · 2021-10-22T10:12:37Z

@Elisedlund-ericsson

@Test(groups = "testGroup1")
public void test1() {
	System.out.println("Test #1");
}

If you run this within the IDE, you are not selecting any groups. TestNG by default is built to run all test methods in a class when there are no groups.

But the same is not applicable for @BeforeGroups and @AfterGroups because by their definition these are group level setups and teardowns that are to be executed only when the groups to which they are part of is selected for execution.

These two annotations are supposed to be used ONLY when you are bringing in test filtering using groups and TestNG will execute these methods when the groups to which they are supposed to be part of, get executed.

Not sure what the confusion is. But like I mentioned before, this annotation does have a alwaysRun which if turned on can be configured to run this group configuration irrespective of which group is being chosen for execution. But for that TestNG should know if the user is running with group level filtering enabled or with it being disabled.

There is no counter intuitiveness in this behaviour.

martinaldrin · 2021-10-22T10:37:52Z

I don't agree, For Before/AfterTest is very clear that they depends on the suite.
But the group definition in the suite is used for filtering which groups we want to include/exclude.
The include/exclude of groups should in my "opinion" work in similar way as include/exclude methods and that was how it was working in 6.14.

For test is pretty obvious, here we give the test a name, but that is not the case for groups or methods.

Regarding the alwaysRun I don't think that will solve this issue, and it would also require many tests cases to be updated.

krmahadevan · 2021-10-22T11:42:12Z

@martinaldrin

I don't agree, For Before/AfterTest is very clear that they depends on the suite.

Please be advised that irrespective of whether you create a suite xml or not, TestNG always creates one, when you run an individual test class from within an IDE. Thats how TestNG is built.

The include/exclude of groups should in my "opinion" work in similar way as include/exclude methods and that was how it was working in 6.14.

Am pretty sure that you can pass in the list of groups to which your @BeforeGroups and @AfterGroups can belong to. That is provided for this exact reason so that a group level configuration can be tied to a specific group or groups in question.
The bug that was fixed was when one doesnt provide any list of groups to which these configuration methods dont belong to and when a user has not specified that it needs to always run, then they shouldn't be executed because these are group level configurations. Why run them when no group level filtering is even being opted for by the user.

Regarding the alwaysRun I don't think that will solve this issue, and it would also require many tests cases to be updated.

Not sure what you mean that it would solve this issue. You dont need to update anything. You can do this via an annotation transformer which exists for this very same reason.

Let me first do the needed changes to support what I am talking about and then we can circle back to the same discussion. I have noticed that currently the alwaysRun attribute is not being honoured and so it would need to be fixed.

martinaldrin · 2021-10-22T11:51:37Z

Am pretty sure that you can pass in the list of groups to which your @BeforeGroups and @AfterGroups can belong to. That is provided for this exact reason so that a group level configuration can be tied to a specific group or groups in question. The bug that was fixed was when one doesnt provide any list of groups to which these configuration methods dont belong to and when a user has not specified that it needs to always run, then they shouldn't be executed because these are group level configurations. Why run them when no group level filtering is even being opted for by the user.

If Before/AfterGroups can only run if we do include in the suite xml, what is then the purpose of exclude groups in the suite, since they will never run. that means exclude on group will now only exclude other configuration methods and test methods.
In my opinion, Before/AfterGroups should run by default if user have not specified any include/exclude in the suite. If we do include, only these groups should be execute, Similar to how include of methods works, all methods are included by default and if you add the filtering with the include tag only those methods will run.

I don't think that was a bug that you refer to, if users don't want to run a certain group, they should exclude the groups that shouldn't run. That is how we have done this for very long time. Why should group works in a different way compare to methods?

Let me first do the needed changes to support what I am talking about and then we can circle back to the same discussion. I have noticed that currently the alwaysRun attribute is not being honoured and so it would need to be fixed.

Regarding alwaysRun, there is no documentation at all for this argument on Before/AfterGroup

Also, Before/AfterGroup was introduced in the Initial commit 2006, So I guess there is plenty of users using this feature in a similar way as we do. After so long time, I don't think a certain behavior can be treated as a bug.

krmahadevan · 2021-10-22T12:04:09Z

@cedricbeust-okta @cbeust

I think we need your help to clarify what is supposed to be the exact behaviour of this configuration annotations since you have more insights into this than me.
Please let me know how you want this to be taken forward.

cbeust · 2021-10-22T14:16:32Z

Krishnan,

At this point, I think what the original intent is or was is irrelevant (assuming there was even one :-)).

Quite a few users seem to rely on the old behavior so we should find a way to restore it.

And if we can't make this consistent with the current design, I think it would be fine to have a special flag just to enable this behavior, so that new users don't rely on it accidentally but old users can still use it.

martinaldrin · 2021-10-22T15:11:48Z

Hi,

For me a property would be fine, because that allow us to keep using a very nice feature.

I did a quick test, just by adding false in this method, my example test that I provided pass and running as expected.

diff --git a/testng-core-api/src/main/java/org/testng/xml/XmlTest.java b/testng-core-api/src/main/java/org/testng/xml/XmlTest.java
index 8595da8e..c49c1fef 100644
--- a/testng-core-api/src/main/java/org/testng/xml/XmlTest.java
+++ b/testng-core-api/src/main/java/org/testng/xml/XmlTest.java
@@ -115,7 +115,7 @@ public class XmlTest implements Cloneable {
   }
 
   public boolean isGroupFilteringDisabled() {
-    return getIncludedGroups().isEmpty() && getExcludedGroups().isEmpty();
+    return false;
   }
 
   /** @return Returns the classes. */

krmahadevan · 2021-10-23T01:55:52Z

Hi,

For me a property would be fine, because that allow us to keep using a very nice feature.

I did a quick test, just by adding false in this method, my example test that I provided pass and running as expected.

diff --git a/testng-core-api/src/main/java/org/testng/xml/XmlTest.java b/testng-core-api/src/main/java/org/testng/xml/XmlTest.java
index 8595da8e..c49c1fef 100644
--- a/testng-core-api/src/main/java/org/testng/xml/XmlTest.java
+++ b/testng-core-api/src/main/java/org/testng/xml/XmlTest.java
@@ -115,7 +115,7 @@ public class XmlTest implements Cloneable {
   }
 
   public boolean isGroupFilteringDisabled() {
-    return getIncludedGroups().isEmpty() && getExcludedGroups().isEmpty();
+    return false;
   }
 
   /** @return Returns the classes. */

Why not submit a pull request with these changes @martinaldrin since you already have the fix

martinaldrin · 2021-10-23T07:13:34Z

Hi,

I just know how to revert it to the original behavior. I don't yet know how properties are used within TestNg if properties should be used. I just did this change to test my example and to test some of the failing test I have found so far that relates to this issue. And it looks promising, So I will run more test, but during my test I found new Non Back ward compatible changes that I need to adapt to in our framework before I can test more. And as soon I have tested more i can Push a PR.
I depend on another TestNg fork at the moment, so I can not easily push a PullRequest. But I will try to handle that somehow.

Closes testng-team#2229 Ensure that BeforeGroups and AfterGroups config Methods always get invoked irrespective of whether Group level filtering being done by users.

Closes #2229 Ensure that BeforeGroups and AfterGroups config Methods always get invoked irrespective of whether Group level filtering being done by users.

krmahadevan added the Need: sample label Jan 27, 2020

krmahadevan added this to the 7.2 milestone Jan 27, 2020

MahmoudElSharkawy changed the title ~~Version 7.1.0, @BeforeGroups and @AfterGroups are not working when running from a TestNG class~~ Version 7.1.0, @BeforeGroups and @AfterGroups are not working Jan 28, 2020

MahmoudElSharkawy changed the title ~~Version 7.1.0, @BeforeGroups and @AfterGroups are not working~~ Version 7.1.0, @BeforeGroups and @AfterGroups Annotations are not working Jan 28, 2020

krmahadevan closed this as completed Jun 18, 2020

juherr reopened this Oct 21, 2021

krmahadevan mentioned this issue Oct 24, 2021

Restore BeforeGroups|AfterGroups functionality back #2660

Merged

2 tasks

juherr linked a pull request Oct 24, 2021 that will close this issue

[GITHUB-2135] @BeforeClass and @AfterClass in superclass ignore group… #2661

Closed

krmahadevan closed this as completed in #2660 Oct 25, 2021

krmahadevan added a commit that referenced this issue Oct 25, 2021

Restore BeforeGroups|AfterGroups functionality back

49dbdc9

Closes #2229 Ensure that BeforeGroups and AfterGroups config Methods always get invoked irrespective of whether Group level filtering being done by users.

krmahadevan modified the milestones: 7.2, 7.5.0 Oct 25, 2021

krmahadevan removed the Need: sample label Oct 25, 2021

Version 7.1.0, @BeforeGroups and @AfterGroups Annotations are not working #2229

Version 7.1.0, @BeforeGroups and @AfterGroups Annotations are not working #2229

Comments

MahmoudElSharkawy commented Jan 21, 2020 • edited Loading

TestNG Version

Expected behavior

Actual behavior

Is the issue reproductible on runner?

Sample code to reproduce:

MohabMohie commented Jan 21, 2020

krmahadevan commented Jan 27, 2020

MahmoudElSharkawy commented Jan 27, 2020 • edited Loading

Sample code:

krmahadevan commented Jan 27, 2020

MahmoudElSharkawy commented Jan 27, 2020 • edited Loading

Nikasan commented Mar 2, 2020 • edited by krmahadevan Loading

MikeLaptev commented Apr 23, 2020

EyalSooliman commented Jun 3, 2020

krmahadevan commented Jun 18, 2020

MohabMohie commented Jun 18, 2020

krmahadevan commented Jun 19, 2020

martinaldrin commented Oct 20, 2021

MahmoudElSharkawy commented Oct 20, 2021 • edited Loading

martinaldrin commented Oct 20, 2021 • edited Loading

cedricbeust-okta commented Oct 20, 2021

krmahadevan commented Oct 21, 2021

juherr commented Oct 21, 2021

krmahadevan commented Oct 21, 2021

juherr commented Oct 21, 2021

martinaldrin commented Oct 21, 2021 • edited Loading

martinaldrin commented Oct 21, 2021 • edited Loading

juherr commented Oct 21, 2021

juherr commented Oct 22, 2021

martinaldrin commented Oct 22, 2021 • edited Loading

martinaldrin commented Oct 22, 2021

Elisedlund-ericsson commented Oct 22, 2021 • edited Loading

martinaldrin commented Oct 22, 2021

krmahadevan commented Oct 22, 2021

martinaldrin commented Oct 22, 2021

krmahadevan commented Oct 22, 2021

martinaldrin commented Oct 22, 2021 • edited Loading

krmahadevan commented Oct 22, 2021

cbeust commented Oct 22, 2021

martinaldrin commented Oct 22, 2021 • edited Loading

krmahadevan commented Oct 23, 2021

martinaldrin commented Oct 23, 2021

MahmoudElSharkawy commented Jan 21, 2020 •

edited

Loading

MahmoudElSharkawy commented Jan 27, 2020 •

edited

Loading

MahmoudElSharkawy commented Jan 27, 2020 •

edited

Loading

Nikasan commented Mar 2, 2020 •

edited by krmahadevan

Loading

MahmoudElSharkawy commented Oct 20, 2021 •

edited

Loading

martinaldrin commented Oct 20, 2021 •

edited

Loading

martinaldrin commented Oct 21, 2021 •

edited

Loading

martinaldrin commented Oct 21, 2021 •

edited

Loading

martinaldrin commented Oct 22, 2021 •

edited

Loading

Elisedlund-ericsson commented Oct 22, 2021 •

edited

Loading

martinaldrin commented Oct 22, 2021 •

edited

Loading

martinaldrin commented Oct 22, 2021 •

edited

Loading