UnitTest: Make the console cmd Dev::UnitTest::Run a frontend for TAP::Harness

[bschmalhofer]

As prove -I . -I Kernel\cpan-lib -r scripts/testis now working, the console command command can be converted to doing the same thing as prove. The important features should be preserved. Unused features should be removed.

Internally Kernel::System::UnitTest::Drivercould changed to be a proper Test2 testing tool. See https://metacpan.org/pod/Test2::Manual::Tooling . This should be safe for all the encoding issues, because the question whether ùse utf8`is active is then solely determined by the test script itself.

Switching to App::Provewould also allow to remove the funny 1; in the last line of the test scripts.

[svenoe]

First we have to discuss the pros and cons of this!! ;)

[bschmalhofer]

I'll change the label to 'question'.

[bschmalhofer]

This change would also allow the use of Test2::V0 and Test::Builder.

[bschmalhofer]

During the September meeting of the OTOBO Team it was decided that the continous integration features of Dev::UnitTest::Run are no longer supported. These features can be removed. Concretely the following options can be removed:

• --submit-url
• --submit-auth
• --submit-result-as-exit-code
• --job-id
• --scenario
• --attachment-path
• --pre-submit-script

The reason is that the infrastructure for managing these submissions do not exist for OTOBO. New infrastructure might be based on Github actions or on https://metacpan.org/release/Tapper. But that will be a different issue.

[bschmalhofer]

The relevant module is actually TAP::Harness, not App::Prove. App::Prove uses TAP::Harness.

[bschmalhofer]

The option --test-runs can also be added again when it is found to be useful.

The same applies to the option --data-diff-type. Supporting this option is hard because the driver instance is now created by the script itself. So, for now, let's stick with the default diff type table.

[bschmalhofer]

The Selenium screenshot feature will be disabled for now as well.

[bschmalhofer]

The migration seems to work mostly. One failure is UnitTest/Blacklist.t, It looks like STDOUT from

$ExitCode = $CommandObject->Execute( '--test', $Test->{Test} );
is not captured in Blacklist.t. Strange.

[bschmalhofer]

Let's create a branch based on rel-10_0 that backports these adaptions to the test suite. Based on the result of test runs we can decide whether these adaption can be included in 10.0.4. The actual backport will be done in #447 .

[bschmalhofer]

Tests look fine:

161021 Test Summary Report
161022 -------------------
161023 /opt/otobo/scripts/test/UnitTest/Blacklist.t                                                              (Wstat: 0 Tests: 3 Failed: 1)
161024   Failed test:  2
161025   Parse errors: More than one plan found in TAP output
161026                 Tests out of sequence.  Found (1) but expected (2)
161027                 Tests out of sequence.  Found (2) but expected (3)
161028                 Bad plan.  You planned 2 tests but ran 3.
161029 Files=949, Tests=56381, 1775 wallclock secs ( 7.87 usr  2.00 sys + 1106.61 cusr 122.07 csys = 1238.55 CPU)
161030 Result: FAIL
161031 ran tests for product OTOBO 10.1.x on yourhost.example.com

The issue with UnitTest/Blacklist.t is handled in #472 . Closing this issue. Further problems should be handled in dedicated issues.