MQE-1331: MFTF Run:Test Will Run All Tests That Are Already Generated

[soumyau]

• MC-1331
  updated code to run only tests specified in run command

Description

Fixed Issues (if relevant)

1. magento/magento2-functional-testing-framework#<issue_number>: Issue title
2. ...

Contribution checklist

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/verification tests (if applicable)
• All automated tests passed successfully (all builds on Travis CI are green)
• Changes to Framework doesn't have backward incompatible changes for tests or have related Pull Request with fixes to tests

[coveralls]

Coverage remained the same at 52.176% when pulling 9006361 on MQE-1331 into 8f54cbb on develop.

[KevinBKozan]

Small code structure suggestion, but otherwise this change works well 👍

[KevinBKozan]

What do you think about restructuring this a little bit:

$returnCode = 0;
        $codeceptionCommand = realpath(PROJECT_ROOT . '/vendor/bin/codecept') . ' run functional ';
        $testsDirectory = TESTS_MODULE_PATH . DIRECTORY_SEPARATOR . TestGenerator::GENERATED_DIR . DIRECTORY_SEPARATOR;
        //execute only tests specified as arguments in run command
        foreach ($tests as $test) {
            $testGroup = TestGenerator::DEFAULT_DIR . DIRECTORY_SEPARATOR;
            $testName = $test . 'Cest.php';
            $fullCommand = $codeceptionCommand . $testsDirectory . $testGroup . $testName . '--verbose --steps';

This sets up for easier refactor when we implement running a test in the suite context, since whoever touches it only needs to redefine $testGroup

[soumyau]

Restructured it this way. I like the part where we are moving static portions of the command outside of foreach.

[KevinBKozan]

How do you feel about adding in some explicit error checking:

if (!realpath($testsDirectory . $testGroup . $testName))

Right now if I try to reference a test that isn't generated I get a massive stack trace and an exception in Loader.php

[soumyau]

Fixed, looks like this is a pre-existing error. Good catch.