If there is Background and 2 or more tests, if 1 test failed the others are also marked as failed.

[vitaliirymar]

Describe the bug
In a scenario outline with examples, if 1 example passed and another one failed, the report shows 2 failed tests

Steps to Reproduce
Steps to reproduce the behavior:

1. Run Scenario Outline with at least 2 Examples where one of them fails.
   E.g.
   Scenario Outline: Verify math
   Given def mathResult = < vara > + < varb >
   Then assert mathResult == < result >

   Examples:
   | vara | varb| result |
   | 2 | 2 | 4 |
   | 1 | 2 | 5 |

2. Check ReportPortal results

Expected behavior
ReportPortal displays 1 passed test and 1 failed test

Actual behavior
ReportPortal displays 2 failed tests
The passed test is marked as a failed, but without failed steps and without error log inside the steps.

Dependency versions
agent-java-karate 5.0.0

[HardNorth]

@vitaliirymar Thanks for reporting this, but unfortunately I cannot reproduce your case. I created a separate test just for this, but it passes normally:

Could you please isolate and share your example?

[vitaliirymar]

@HardNorth I managed to reproduce it with Background. Will rename the issue title.

[HardNorth]

@vitaliirymar, Please provide any example. Since I also have tests covering Background functionality.

[vitaliirymar]

@HardNorth try this one:

Feature: Karate Scenario Examples

  Background:
    * def util = Java.type('helpers')

  Scenario Outline: Verify math
    Given def mathResult = < vara > + < varb >
    Then assert mathResult == < result >

    Examples:
      | vara | varb| result |
      | 2 | 2 | 4 |
      | 1 | 2 | 5 |

[vitaliirymar]

@HardNorth one more request - could you please run the same test (no matter with/without Background) with 2 Examples wit the ReportPortalHook() runner:

 @Test
    public void TestRunner() {
        Results results = Runner
                .path("classpath:testExamples")
                .hook(new ReportPortalHook())
                .outputCucumberJson(true)//.path("target/cucumber-reports")
                .tags("~@ignore")
                .parallel(5);
    }

I see the following - one scenario example is outside the Feature

[vitaliirymar]

Update: The issue is also reproducible with 2 or more regular tests when 1 of them fails, all the other tests are marked as failed, if there is Background. E.g.:

Feature: Karate Scenario Examples

  Background: Set varb
    Given def varb = 2

  Scenario: Verify math 1
    Given def mathResult = 4 + varb
    Then assert mathResult == 6

  Scenario: Verify math 2
    Given def mathResult = 2 + 3
    Then assert mathResult == 4

[HardNorth]

Should be fixed in: https://github.com/reportportal/agent-java-karate/releases/tag/5.0.2

[vitaliirymar]

@HardNorth , the issue is still reproducible. It is related to Background.
Check these scenarios:
#23 (comment)
#23 (comment)

[HardNorth]

@vitaliirymar Should be fixed in: https://github.com/reportportal/agent-java-karate/releases/tag/5.0.3

[vrymar]

@HardNorth thanks! It's fixed now!