diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 000000000..ef21d7bde --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,174 @@ +# Contribution Guidelines for the Magento Functional Testing Framework + +- [Contribution requirements](#contribution-requirements) +- [Fork a repository](#fork-a-repository) + - [Update the fork with the latest changes](#update-the-fork-with-the-latest-changes) +- [Create a pull request](#create-a-pull-request) +- [Report an issue](#report-an-issue) +- [Read labels](#read-labels) + - [Pull request status](#pull-request-status) + - [Issue resolution status](#issue-resolution-status) + - [Domains impacted](#domains-impacted) + - [Type](#type) + +Use the [fork] & [pull] model to contribute to the Magento Functional Testing Framework (MFTF) code base. +This contribution model has contributors maintaining their own copy of the forked code base (which can be easily synced with the main copy). +The forked repository is then used to submit a request to the base repository to pull a set of changes (pull request). + +Contributions can take the form of new components or features, changes to existing features, tests, documentation (such as developer guides, user guides, examples, or specifications), bug fixes, optimizations, or just good suggestions. + +The MFTF development team reviews all issues and contributions submitted by the community of developers in a "first-in, first-out" basis. +During the review we might require clarifications from the contributor. +If there is no response from the contributor for two weeks, the issue is closed. + +Often when the MFTF team works on reviewing the suggested changes, we will add a label to the issue to indicate to our internal team certain information, like status or who is working the issue. +If you’re ever curious what the different labels mean, see the [table][labels] below for an explanation of each one. + +Refer to [Magento Contributor Agreement] for detailed information about the License Agreement. +All contributors are required to submit a click-through form to agree to the terms. + +## Contribution requirements + +1. Contributions must adhere to [Magento coding standards]. +2. Refer to the Magento development team’s [Definition of Done]. + We use these guidelines internally to ensure that we deliver well-tested, well-documented, solid code, and we encourage you to as well! +3. Pull requests (PRs) must be accompanied by a meaningful description of their purpose. + Comprehensive descriptions increase the chances that a pull request is merged quickly and without additional clarification requests. +4. Commits must be accompanied by meaningful commit messages. +5. PRs that include bug fixing must be accompanied by a step-by-step description of how to reproduce the bug. +6. PRs that include new logic or new features must be submitted along with: + + - Unit/integration test coverage + - Proposed documentation update. + For the documentation contribution guidelines, see [DevDocs Contributing][devdocs]. +7. For large features or changes, [open an issue][issue] to discuss first. + This may prevent duplicate or unnecessary effort, and it may gain you some additional contributors. +8. To report a bug, [open an issue][issue], and follow [guidelines about bugfix issues][issue reporting]. +9. All automated tests must pass successfully (all builds on [Travis CI] must be green). + +## Fork a repository + +To fork a repository on Github, do the following: + +1. Navigate to the [MFTF repository]. +2. Click **Fork** at the top right. +3. Clone the repo into your development environment and start playing. + +Learn more in the [Fork a repo][github fork] GitHub article. + +### Update the fork with the latest changes + +As community and Magento writers’ changes are merged to the repository, your fork becomes outdated and pull requests might result in conflicts. +To see if your fork is outdated, open the fork page in GitHub and if at the top displays the following message: + +__This branch is NUMBER commits behind magento:develop.__ + +It means your fork must be updated. + +There are two ways to update your fork. +The typical way is discussed in the [Syncing a fork][github sync fork] GitHub article. +Make sure to update from the correct branch! + +The other way is to create a reverse pull request from the original repository. +Though this method has the downside of inserting unnecessary information into fork commit history. + +1. In your fork, click **New pull request**. +2. Click the "switching the base" link and then click **Create pull request**. +3. Provide a descriptive name for your pull request in the provided field. +4. Scroll to the bottom of the page and click **Merge pull request**, then click **Confirm Merge**. + +## Create a pull request + +First, check the [existing PRs] and make sure you are not duplicating others’ work! + +To create a pull request do the following: + +1. Create a feature branch for your changes and push those changes to the copy of your repository on GitHub. + This is the best way to organize and even update your PR. +2. In your repository, click **Pull requests**, and then click **New pull request**. +3. Ensure that you are creating a PR to the **magento/magento2-functional-testing-framework: develop** branch. + We accept PRs to this branch only. +4. Review the changes, then click **Create pull request**. + Fill out the form, and click **Create pull request** again to submit the PR—that’s it! + +Learn more in the [Creating a pull request][create pr] GitHub article. + +After submitting your PR, you can head over to the repository’s [Pull Requests panel][existing PRs] to see your PR along with the others. +Your PR undergoes automated testing, and if it passes, the core team considers it for inclusion in the Magento Functional Testing Framework codebase. +If some tests fail, make the corresponding corrections in your code. + +## Report an issue + +If you find a bug in Magento Functional Testing Framework code, you can report it by creating an issue in the Magento Functional Testing Framework repository. + +Before creating an issue, do the following: + +1. Read the [issue reporting guidelines][issue reporting] to learn how to create an issue that can be processed in a timely manner. +2. Check the documentation to make sure the behavior you are reporting is really a bug, not a feature. +3. Check the [existing issues] to make sure you are not duplicating somebody’s work. + +To add an issue: + +1. [Open a new issue][open new issue] +2. Fill in the **Title** and issue description +3. Click **Submit new issue** + +Learn more in the [Creating an issue][create issue] GitHub article. + +## Read labels + +Refer to the tables with descriptions of each label below. +The labels reflect the status, impact, or which team is working on it. + +### Pull request status + +Label| Description +---|--- +**accept**| The pull request has been accepted to be merged into mainline code. +**reject**| The pull request has been rejected. The most common cases are when the issue has already been fixed in another code contribution, or there is an issue with the code contribution. +**needsUpdate**| We need more information from the PR author to properly prioritize and process the pull request. + +### Issue resolution status + +Label| Description +---|--- +**acknowledged**| We validated the issue and created an internal ticket. +**needsUpdate**| We need more information from the PR author to properly prioritize and process the issue. +**cannot reproduce**| We do not have enough details from the issue description to reproduce the issue. +**non-issue**| We don't think that this is an issue according to the provided information. + +### Domains impacted + +Label| Description +---|--- +**PROD**| Affects the Product team (mostly feature requests or business logic change). +**DOC**| Affects Documentation domain. +**TECH**| Affects Architect Group (mostly to make decisions around technology changes). + +### Type + +Label| Description +---|--- +**bugfix**| The issue or pull request is about fixing a bug. +**enhancement**| The issue or pull request that makes the MFTF even more awesome (for example new features, optimization, refactoring, etc). + +[fork]: #fork-a-repository +[issue]: #report-an-issue +[labels]: #read-labels +[pull]: #create-a-pull-request + +[create issue]: https://help.github.com/articles/creating-an-issue/ +[create pr]: https://help.github.com/articles/creating-a-pull-request/ +[Definition of Done]: https://devdocs.magento.com/guides/v2.2/contributor-guide/contributing_dod.html +[devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md +[existing issues]: https://github.com/magento/magento2-functional-testing-framework/issues?q=is%3Aopen+is%3Aissue +[existing PRs]: https://github.com/magento/magento2-functional-testing-framework/pulls?q=is%3Aopen+is%3Apr +[GitHub documentation]: https://help.github.com/articles/syncing-a-fork +[github fork]: https://help.github.com/articles/fork-a-repo/ +[github sync fork]: https://help.github.com/articles/syncing-a-fork/ +[issue reporting]: https://github.com/magento/magento2-functional-testing-framework/wiki/Issue-reporting-guidelines +[Magento coding standards]: https://devdocs.magento.com/guides/v2.2/coding-standards/bk-coding-standards.html +[Magento Contributor Agreement]: http://www.magento.com/legaldocuments/mca +[MFTF repository]: https://github.com/magento/magento2-functional-testing-framework +[open new issue]: https://github.com/magento/magento2-functional-testing-framework/issues/new +[Travis CI]: https://travis-ci.com/magento/magento2-functional-testing-framework/pull_requests \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index c8a7efa13..2b157e571 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,29 @@ Magento Functional Testing Framework Changelog ================================================ +2.3.7 +----- +### Enhancements +* Traceability + * Test generation errors output xml filename where they were encountered, as well as xml parent nodes where applicable. + * Duplicate element detection now outputs parent element where duplicate was found. +* Maintainability + * Standalone MFTF can now be pointed at a Magento installation folder to generate and execute tests. + * See DevDocs for more information. + * MFTF now checks for `test` and `actionGroup` elements that have the same `name` in the same file. +* Customizability + * Updated prefered syntax for `actionGroup` `argument`s that use `xml.data` (old syntax is still supported) + * Old: `xml.data` + * New: `{{xml.data}}` +* Modularity + * `ModuleResolver` now utilizes each Magento module's `registration.php` to map MFTF test material directories. +### Fixes +* The `waitForPageLoad` action now correctly uses the given `timeout` attribute for all of its checks. +* Firefox compatibility issues in javascript error logging were fixed. +* Fixed an issue where arguments containing `-` would not properly resolve parameterized selectors. +* Fixed an issue where actions using `parameterArray` would not resolve `$persisted.data$` references. +* Fixed an issue where composer installations of Magento would fail to parse MFTF materials under a path `vendor/magento/module-/` + 2.3.6 ----- ### Enhancements diff --git a/README.md b/README.md index 9687c3b47..487902d96 100755 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Magento Functional Testing Framework +# Magento Functional Testing Framework (MFTF) [![Build Status](https://travis-ci.org/magento/magento2-functional-testing-framework.svg?branch=develop)](https://travis-ci.org/magento/magento2-functional-testing-framework) @@ -6,69 +6,66 @@ ---- -## System Requirements -[Magento Functional Testing Framework system requirements](http://devdocs.magento.com/guides/v2.2/magento-functional-testing-framework/getting-started.html#prepare-environment) - ## Installation -To install the Magento Functional Testing Framework, see [Getting Started](http://devdocs.magento.com/guides/v2.2/magento-functional-testing-framework/getting-started.html) - -## Contributing -Contributions can take the form of new components or features, changes to existing features, tests, documentation (such as developer guides, user guides, examples, or specifications), bug fixes, optimizations, or just good suggestions. -To learn about how to make a contribution, click [here][1]. +For the installation guidelines and system requirements, read [Getting Started](https://devdocs.magento.com/mftf/2.3/getting-started.html). -To open an issue, click [here][2]. +## Contributing -To suggest documentation improvements, click [here][3]. +We would appreciate your contributions to new components or new features, changes to the existing features, tests, documentation, specifications, bug fixes, optimizations, or just good suggestions. +Report about an issue or request features opening a GitHub issue. +Learn more about contributing in our [Contribution Guidelines](https://devdocs.magento.com/mftf/2.3/contribution-guidelines.html). -[1]: -[2]: -[3]: +If you want to participate in the documentation work, see [DevDocs Contributing](https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md). ### Labels applied by the MFTF team -Refer to the tables with descriptions of each label below. These labels are applied by the MFTF development team to community contributed issues and pull requests, to communicate status, impact, or which team is working on it. +Refer to the tables with descriptions of each label below. +These labels are applied by the MFTF development team to community contributed issues and pull requests, to communicate status, impact, or which team is working on it. ### Pull Request Status -Label| Description ----|--- -**accept**| The pull request has been accepted and will be merged into mainline code. -**reject**| The pull request has been rejected and will not be merged into mainline code. Possible reasons can include but are not limited to: issue has already been fixed in another code contribution, or there is an issue with the code contribution. -**needsUpdate**| The Magento Team needs additional information from the reporter to properly prioritize and process the pull request. +| Label | Description | +| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **accept** | The pull request has been accepted and will be merged into mainline code. | +| **reject** | The pull request has been rejected and will not be merged into mainline code. Possible reasons can include but are not limited to: issue has already been fixed in another code contribution, or there is an issue with the code contribution. | +| **needsUpdate** | The Magento Team needs additional information from the reporter to properly prioritize and process the pull request. | ### Issue Resolution Status -Label| Description ----|--- -**acknowledged**| The Magento Team has validated the issue and an internal ticket has been created. -**needsUpdate**| The Magento Team needs additional information from the reporter to properly prioritize and process the issue or pull request. -**cannot reproduce**| The Magento Team has not confirmed that this issue contains the minimum required information to reproduce. -**non-issue**| The Magento Team has not recognised any issue according to provided information. +| Label | Description | +| -------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| **acknowledged** | The Magento Team has validated the issue and an internal ticket has been created. | +| **needsUpdate** | The Magento Team needs additional information from the reporter to properly prioritize and process the issue or pull request. | +| **cannot reproduce** | The Magento Team has not confirmed that this issue contains the minimum required information to reproduce. | +| **non-issue** | The Magento Team has not recognized any issue according to provided information. | ### Domains Impacted -Label| Description ----|--- -**PROD**| Affects the Product team (mostly feature requests or business logic change). -**DOC**| Affects Documentation domain. -**TECH**| Affects Architect Group (mostly to make decisions around technology changes). +| Label | Description | +| -------- | ----------------------------------------------------------------------------- | +| **PROD** | Affects the Product team (mostly feature requests or business logic change). | +| **DOC** | Affects Documentation domain. | +| **TECH** | Affects Architect Group (mostly to make decisions around technology changes). | ### Type -Label| Description ----|--- -**bugfix**| The issue or pull request relates to bug fixing. -**enhancement**| The issue or pull request that raises the MFTF to a higher degree (for example new features, optimization, refactoring, etc). +| Label | Description | +| --------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| **bugfix** | The issue or pull request relates to bug fixing. | +| **enhancement** | The issue or pull request that raises the MFTF to a higher degree (for example new features, optimization, refactoring, etc). | ## Reporting security issues -To report security vulnerabilities in Magento software or web sites, please e-mail security@magento.com. Please do not report security issues using GitHub. Be sure to encrypt your e-mail with our encryption key if it includes sensitive information. Learn more about reporting security issues here. +To report security vulnerabilities and other security issues in the Magento software or web sites, send an email with the report at [security@magento.com](mailto:security@magento.com). +Do not report security issues using GitHub. +Be sure to encrypt your e-mail with our [encryption key](https://info2.magento.com/rs/magentoenterprise/images/security_at_magento.asc) if it includes sensitive information. +Learn more about reporting security issues [here](https://magento.com/security/reporting-magento-security-issue). -Stay up-to-date on the latest security news and patches for Magento by signing up for Security Alert Notifications. +Stay up-to-date on the latest security news and patches for Magento by signing up for [Security Alert Notifications](https://magento.com/security/sign-up). ## License -Each Magento source file included in this distribution is licensed under AGPL 3.0 +Each Magento source file included in this distribution is licensed under AGPL 3.0. -Please see LICENSE_AGPL3.txt for the full text of the AGPL 3.0 license or contact license@magentocommerce.com for a copy. +See the license [here](LICENSE_AGPL3.txt) or contact [license@magentocommerce.com](mailto:license@magentocommerce.com) for a copy. diff --git a/bin/mftf b/bin/mftf index ef96ecca0..e82a0009a 100755 --- a/bin/mftf +++ b/bin/mftf @@ -12,7 +12,7 @@ if (PHP_SAPI !== 'cli') { } $autoloadPath = realpath(__DIR__ . '/../../../autoload.php'); -$testBootstrapPath = realpath(__DIR__ . '/../dev/tests/functional/_bootstrap.php'); +$testBootstrapPath = realpath(__DIR__ . '/../dev/tests/functional/standalone_bootstrap.php'); try { if (file_exists($autoloadPath)) { @@ -29,7 +29,7 @@ try { try { $application = new Symfony\Component\Console\Application(); $application->setName('Magento Functional Testing Framework CLI'); - $application->setVersion('2.3.6'); + $application->setVersion('2.3.7'); /** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */ $commandList = new \Magento\FunctionalTestingFramework\Console\CommandList; foreach ($commandList->getCommands() as $command) { diff --git a/composer.json b/composer.json index 60d6f2653..b4cf1e81e 100755 --- a/composer.json +++ b/composer.json @@ -2,7 +2,7 @@ "name": "magento/magento2-functional-testing-framework", "description": "Magento2 Functional Testing Framework", "type": "library", - "version": "2.3.6", + "version": "2.3.7", "license": "AGPL-3.0", "keywords": ["magento", "automation", "functional", "testing"], "config": { diff --git a/composer.lock b/composer.lock index 8ec534fda..6f29bf92d 100644 --- a/composer.lock +++ b/composer.lock @@ -4,7 +4,7 @@ "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies", "This file is @generated automatically" ], - "content-hash": "4dc0a2b0fc82647238bf0892f3b65d0f", + "content-hash": "b0a708feb45b7cbaa369da36a14cd7df", "packages": [ { "name": "allure-framework/allure-codeception", @@ -297,16 +297,16 @@ }, { "name": "consolidation/annotated-command", - "version": "2.8.5", + "version": "2.9.1", "source": { "type": "git", "url": "https://github.com/consolidation/annotated-command.git", - "reference": "1e8ff512072422b850b44aa721b5b303e4a5ebb3" + "reference": "4bdbb8fa149e1cc1511bd77b0bc4729fd66bccac" }, "dist": { "type": "zip", - "url": "https://api.github.com/repos/consolidation/annotated-command/zipball/1e8ff512072422b850b44aa721b5b303e4a5ebb3", - "reference": "1e8ff512072422b850b44aa721b5b303e4a5ebb3", + "url": "https://api.github.com/repos/consolidation/annotated-command/zipball/4bdbb8fa149e1cc1511bd77b0bc4729fd66bccac", + "reference": "4bdbb8fa149e1cc1511bd77b0bc4729fd66bccac", "shasum": "" }, "require": { @@ -345,7 +345,7 @@ } ], "description": "Initialize Symfony Console commands from annotated command class methods.", - "time": "2018-08-18T23:51:49+00:00" + "time": "2018-09-19T17:47:18+00:00" }, { "name": "consolidation/config", @@ -2639,16 +2639,16 @@ }, { "name": "phpunit/phpunit", - "version": "6.5.12", + "version": "6.5.13", "source": { "type": "git", "url": "https://github.com/sebastianbergmann/phpunit.git", - "reference": "24da433d7384824d65ea93fbb462e2f31bbb494e" + "reference": "0973426fb012359b2f18d3bd1e90ef1172839693" }, "dist": { "type": "zip", - "url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/24da433d7384824d65ea93fbb462e2f31bbb494e", - "reference": "24da433d7384824d65ea93fbb462e2f31bbb494e", + "url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/0973426fb012359b2f18d3bd1e90ef1172839693", + "reference": "0973426fb012359b2f18d3bd1e90ef1172839693", "shasum": "" }, "require": { @@ -2719,7 +2719,7 @@ "testing", "xunit" ], - "time": "2018-08-22T06:32:48+00:00" + "time": "2018-09-08T15:10:43+00:00" }, { "name": "phpunit/phpunit-mock-objects", @@ -5390,16 +5390,16 @@ }, { "name": "squizlabs/php_codesniffer", - "version": "3.3.1", + "version": "3.3.2", "source": { "type": "git", "url": "https://github.com/squizlabs/PHP_CodeSniffer.git", - "reference": "628a481780561150481a9ec74709092b9759b3ec" + "reference": "6ad28354c04b364c3c71a34e4a18b629cc3b231e" }, "dist": { "type": "zip", - "url": "https://api.github.com/repos/squizlabs/PHP_CodeSniffer/zipball/628a481780561150481a9ec74709092b9759b3ec", - "reference": "628a481780561150481a9ec74709092b9759b3ec", + "url": "https://api.github.com/repos/squizlabs/PHP_CodeSniffer/zipball/6ad28354c04b364c3c71a34e4a18b629cc3b231e", + "reference": "6ad28354c04b364c3c71a34e4a18b629cc3b231e", "shasum": "" }, "require": { @@ -5437,7 +5437,7 @@ "phpcs", "standards" ], - "time": "2018-07-26T23:47:18+00:00" + "time": "2018-09-23T23:08:17+00:00" }, { "name": "symfony/config", diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php index 4364b7f9a..b55f12c2b 100644 --- a/dev/tests/_bootstrap.php +++ b/dev/tests/_bootstrap.php @@ -40,7 +40,8 @@ 'MAGENTO_BASE_URL' => 'http://baseurl:8080', 'MAGENTO_BACKEND_NAME' => 'admin', 'MAGENTO_ADMIN_USERNAME' => 'admin', - 'MAGENTO_ADMIN_PASSWORD' => 'admin123' + 'MAGENTO_ADMIN_PASSWORD' => 'admin123', + 'DEFAULT_TIMEZONE' => 'America/Los_Angeles' ]; foreach ($TEST_ENVS as $key => $value) { diff --git a/dev/tests/functional/_bootstrap.php b/dev/tests/functional/standalone_bootstrap.php similarity index 60% rename from dev/tests/functional/_bootstrap.php rename to dev/tests/functional/standalone_bootstrap.php index 219a2400f..763062d04 100755 --- a/dev/tests/functional/_bootstrap.php +++ b/dev/tests/functional/standalone_bootstrap.php @@ -12,31 +12,30 @@ } defined('PROJECT_ROOT') || define('PROJECT_ROOT', dirname(dirname(dirname(__DIR__)))); + require_once realpath(PROJECT_ROOT . '/vendor/autoload.php'); //Load constants from .env file -defined('FW_BP') || define('FW_BP', PROJECT_ROOT); - -// add the debug flag here -$debug_mode = $_ENV['MFTF_DEBUG'] ?? false; -if (!(bool)$debug_mode && extension_loaded('xdebug')) { - xdebug_disable(); -} - -$RELATIVE_TESTS_MODULE_PATH = '/tests/functional/tests/MFTF'; - -defined('MAGENTO_BP') || define('MAGENTO_BP', PROJECT_ROOT); -defined('TESTS_BP') || define('TESTS_BP', dirname(dirname(__DIR__))); -defined('TESTS_MODULE_PATH') || define('TESTS_MODULE_PATH', realpath(TESTS_BP . $RELATIVE_TESTS_MODULE_PATH)); - -if (file_exists(TESTS_BP . DIRECTORY_SEPARATOR . '.env')) { - $env = new \Dotenv\Loader(TESTS_BP . DIRECTORY_SEPARATOR . '.env'); +$envFilePath = dirname(dirname(__DIR__)); +if (file_exists($envFilePath . DIRECTORY_SEPARATOR . '.env')) { + $env = new \Dotenv\Loader($envFilePath . DIRECTORY_SEPARATOR . '.env'); $env->load(); foreach ($_ENV as $key => $var) { defined($key) || define($key, $var); } + if (array_key_exists('MAGENTO_BP', $_ENV)) { + // TODO REMOVE THIS CODE ONCE WE HAVE STOPPED SUPPORTING dev/tests/acceptance PATH + // define TEST_PATH and TEST_MODULE_PATH + defined('TESTS_BP') || define('TESTS_BP', dirname(dirname(__DIR__))); + $RELATIVE_TESTS_MODULE_PATH = '/tests/functional/tests/MFTF'; + defined('TESTS_MODULE_PATH') || define( + 'TESTS_MODULE_PATH', + realpath(TESTS_BP . $RELATIVE_TESTS_MODULE_PATH) + ); + } + defined('MAGENTO_CLI_COMMAND_PATH') || define( 'MAGENTO_CLI_COMMAND_PATH', 'dev/tests/acceptance/utils/command.php' @@ -45,4 +44,28 @@ defined('MAGENTO_CLI_COMMAND_PARAMETER') || define('MAGENTO_CLI_COMMAND_PARAMETER', 'command'); $env->setEnvironmentVariable('MAGENTO_CLI_COMMAND_PARAMETER', MAGENTO_CLI_COMMAND_PARAMETER); + + defined('DEFAULT_TIMEZONE') || define('DEFAULT_TIMEZONE', 'America/Los_Angeles'); + $env->setEnvironmentVariable('DEFAULT_TIMEZONE', DEFAULT_TIMEZONE); + + try { + new DateTimeZone(DEFAULT_TIMEZONE); + } catch (\Exception $e) { + throw new \Exception("Invalid DEFAULT_TIMEZONE in .env: " . DEFAULT_TIMEZONE . PHP_EOL); + } + +} + +defined('FW_BP') || define('FW_BP', PROJECT_ROOT); +defined('MAGENTO_BP') || define('MAGENTO_BP', PROJECT_ROOT); +defined('TESTS_BP') || define('TESTS_BP', dirname(dirname(__DIR__))); + +$RELATIVE_TESTS_MODULE_PATH = '/tests/functional/tests/MFTF'; +defined('TESTS_MODULE_PATH') || define('TESTS_MODULE_PATH', realpath(TESTS_BP . $RELATIVE_TESTS_MODULE_PATH)); + + +// add the debug flag here +$debug_mode = $_ENV['MFTF_DEBUG'] ?? false; +if (!(bool)$debug_mode && extension_loaded('xdebug')) { + xdebug_disable(); } diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/ActionGroupDomTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/ActionGroupDomTest.php index 5d1dd7878..6a0c926be 100644 --- a/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/ActionGroupDomTest.php +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/ActionGroupDomTest.php @@ -25,7 +25,7 @@ public function testActionGroupDomStepKeyValidation() "; $exceptionCollector = new ExceptionCollector(); - $actionDom = new ActionGroupDom($sampleXml, 'dupeStepKeyActionGroup.xml', $exceptionCollector); + new ActionGroupDom($sampleXml, 'dupeStepKeyActionGroup.xml', $exceptionCollector); $this->expectException(\Exception::class); $exceptionCollector->throwException(); @@ -47,4 +47,25 @@ public function testActionGroupDomInvalidXmlValidation() $this->expectExceptionMessage("XML Parse Error: invalid.xml\n"); new ActionGroupDom($sampleXml, 'invalid.xml', $exceptionCollector); } + + /** + * Test detection of two ActionGroups with the same Name in the same file. + */ + public function testActionGroupDomDuplicateActionGroupsValidation() + { + $sampleXml = ' + + + + + + + '; + + $exceptionCollector = new ExceptionCollector(); + new ActionGroupDom($sampleXml, 'dupeNameActionGroup.xml', $exceptionCollector); + $this->expectException(\Exception::class); + $this->expectExceptionMessageRegExp("/name: actionGroupName is used more than once./"); + $exceptionCollector->throwException(); + } } diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/DomTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/DomTest.php new file mode 100644 index 000000000..e00926978 --- /dev/null +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Test/Config/DomTest.php @@ -0,0 +1,55 @@ + + + + + + '; + + $exceptionCollector = new ExceptionCollector(); + new ActionGroupDom($sampleXml, 'dupeStepKeyTest.xml', $exceptionCollector); + + $this->expectException(\Exception::class); + $this->expectExceptionMessageRegExp("/stepKey: key1 is used more than once. \(Parent: testName\)/"); + $exceptionCollector->throwException(); + } + + /** + * Test detection of two Tests with the same Name in the same file. + */ + public function testTestNameDuplicateValidation() + { + $sampleXml = ' + + + + + + + '; + + $exceptionCollector = new ExceptionCollector(); + new ActionGroupDom($sampleXml, 'dupeTestsTest.xml', $exceptionCollector); + $this->expectException(\Exception::class); + $this->expectExceptionMessageRegExp("/name: testName is used more than once./"); + $exceptionCollector->throwException(); + } +} diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Test/Objects/ActionObjectTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Test/Objects/ActionObjectTest.php index 6163b4d38..511c9748f 100644 --- a/dev/tests/unit/Magento/FunctionalTestFramework/Test/Objects/ActionObjectTest.php +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Test/Objects/ActionObjectTest.php @@ -370,6 +370,21 @@ public function testTooManyArgumentException() $actionObject->resolveReferences(); } + /** + * Action object should throw an exception if the timezone provided is not valid. + */ + public function testInvalidTimezoneException() + { + $this->expectException(TestReferenceException::class); + + $actionObject = new ActionObject('key123', 'generateDate', [ + 'timezone' => "INVALID_TIMEZONE" + ]); + + // Call the method under test + $actionObject->resolveReferences(); + } + private function mockSectionHandlerWithElement($elementObject) { $sectionObject = new SectionObject('SectionObject', ['elementObject' => $elementObject]); diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Util/ActionMergeUtilTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Util/ActionMergeUtilTest.php new file mode 100644 index 000000000..913b106a7 --- /dev/null +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Util/ActionMergeUtilTest.php @@ -0,0 +1,34 @@ + '{{someEntity.entity}}' + ]); + + $this->expectExceptionMessage("Could not resolve entity reference \"{{someEntity.entity}}\" " . + "in Action with stepKey \"fakeAction\".\n" . + "Exception occurred parsing action at StepKey \"fakeAction\""); + + $testActionMergeUtil->resolveActionSteps(["merge123" => $actionObject]); + } +} diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Util/ModuleResolverTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Util/ModuleResolverTest.php index 15aa4ad9d..6a407eded 100644 --- a/dev/tests/unit/Magento/FunctionalTestFramework/Util/ModuleResolverTest.php +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Util/ModuleResolverTest.php @@ -96,27 +96,22 @@ public function testGetModulePathsLocations() ); // Define the Module paths from app/code - $appCodePath = MAGENTO_BP - . DIRECTORY_SEPARATOR - . 'app' . DIRECTORY_SEPARATOR - . 'code' . DIRECTORY_SEPARATOR; + $magentoBaseCodePath = MAGENTO_BP; // Define the Module paths from default TESTS_MODULE_PATH $modulePath = defined('TESTS_MODULE_PATH') ? TESTS_MODULE_PATH : TESTS_BP; // Define the Module paths from vendor modules - $vendorCodePath = PROJECT_ROOT - . DIRECTORY_SEPARATOR - . 'vendor' . DIRECTORY_SEPARATOR; + $projectRootCodePath = PROJECT_ROOT; $mockResolver->verifyInvoked('globRelevantPaths', [$modulePath, '']); $mockResolver->verifyInvoked( 'globRelevantPaths', - [$appCodePath, DIRECTORY_SEPARATOR . 'Test' . DIRECTORY_SEPARATOR .'Mftf'] + [$magentoBaseCodePath, 'Test' . DIRECTORY_SEPARATOR .'Mftf'] ); $mockResolver->verifyInvoked( 'globRelevantPaths', - [$vendorCodePath, DIRECTORY_SEPARATOR . 'Test' . DIRECTORY_SEPARATOR .'Mftf'] + [$projectRootCodePath, 'Test' . DIRECTORY_SEPARATOR .'Mftf'] ); } @@ -151,8 +146,6 @@ public function testGetModulePathsBlacklist() function ($arg1, $arg2) { if ($arg2 === "") { $mockValue = ["somePath" => "somePath"]; - } elseif (strpos($arg1, "app")) { - $mockValue = ["otherPath" => "otherPath"]; } else { $mockValue = ["lastPath" => "lastPath"]; } @@ -161,7 +154,7 @@ function ($arg1, $arg2) { ); $resolver = ModuleResolver::getInstance(); $this->setMockResolverProperties($resolver, null, null, ["somePath"]); - $this->assertEquals(["otherPath", "lastPath"], $resolver->getModulesPath()); + $this->assertEquals(["lastPath", "lastPath"], $resolver->getModulesPath()); TestLoggingUtil::getInstance()->validateMockLogStatement( 'info', 'excluding module', diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Util/TestGeneratorTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Util/TestGeneratorTest.php new file mode 100644 index 000000000..097150888 --- /dev/null +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Util/TestGeneratorTest.php @@ -0,0 +1,41 @@ + '{{someEntity.entity}}' + ]); + + $testObject = new TestObject("sampleTest", ["merge123" => $actionObject], [], [], "filename"); + + $testGeneratorObject = TestGenerator::getInstance("", ["sampleTest" => $testObject]); + + AspectMock::double(TestGenerator::class, ['loadAllTestObjects' => ["sampleTest" => $testObject]]); + + $this->expectExceptionMessage("Could not resolve entity reference \"{{someEntity.entity}}\" " . + "in Action with stepKey \"fakeAction\".\n" . + "Exception occurred parsing action at StepKey \"fakeAction\" in Test \"sampleTest\""); + + $testGeneratorObject->createAllTestFiles(null, []); + } +} diff --git a/dev/tests/unit/Magento/FunctionalTestFramework/Util/Validation/DuplicateNodeValidationUtilTest.php b/dev/tests/unit/Magento/FunctionalTestFramework/Util/Validation/DuplicateNodeValidationUtilTest.php index ad47b73ba..5dba1cb8b 100644 --- a/dev/tests/unit/Magento/FunctionalTestFramework/Util/Validation/DuplicateNodeValidationUtilTest.php +++ b/dev/tests/unit/Magento/FunctionalTestFramework/Util/Validation/DuplicateNodeValidationUtilTest.php @@ -29,6 +29,7 @@ public function testTestActionValidation() '; $uniqueIdentifier = "stepKey"; $filename = "file"; + $testName = "test"; // Perform Test $dom = new \DOMDocument(); @@ -40,8 +41,7 @@ public function testTestActionValidation() $validator->validateChildUniqueness( $testNode, $filename, - $uniqueIdentifier, - $exceptionCollector + $testName ); $this->expectException(\Exception::class); $exceptionCollector->throwException(); diff --git a/dev/tests/verification/Resources/ActionGroupWithParameterizedElementWithHyphen.txt b/dev/tests/verification/Resources/ActionGroupWithParameterizedElementWithHyphen.txt new file mode 100644 index 000000000..64a350505 --- /dev/null +++ b/dev/tests/verification/Resources/ActionGroupWithParameterizedElementWithHyphen.txt @@ -0,0 +1,32 @@ +executeJS("#element .full-width"); + } +} diff --git a/dev/tests/verification/Resources/ActionGroupWithSectionAndDataAsArguments.txt b/dev/tests/verification/Resources/ActionGroupWithSectionAndDataAsArguments.txt new file mode 100644 index 000000000..18881c269 --- /dev/null +++ b/dev/tests/verification/Resources/ActionGroupWithSectionAndDataAsArguments.txt @@ -0,0 +1,32 @@ +waitForElementVisible("#element .John"); + } +} diff --git a/dev/tests/verification/Resources/BasicFunctionalTest.txt b/dev/tests/verification/Resources/BasicFunctionalTest.txt index 197a645cc..493d3c2c0 100644 --- a/dev/tests/verification/Resources/BasicFunctionalTest.txt +++ b/dev/tests/verification/Resources/BasicFunctionalTest.txt @@ -107,6 +107,10 @@ class BasicFunctionalTestCest $date->setTimestamp(strtotime("Now")); $date->setTimezone(new \DateTimeZone("America/Los_Angeles")); $generateDateKey = $date->format("H:i:s"); + $date = new \DateTime(); + $date->setTimestamp(strtotime("Now")); + $date->setTimezone(new \DateTimeZone("UTC")); + $generateDateKey2 = $date->format("H:i:s"); $grabAttributeFromKey1 = $I->grabAttributeFrom(".functionalTestSelector", "someInput"); $grabCookieKey1 = $I->grabCookie("grabCookieInput", ['domain' => 'www.google.com']); $grabFromCurrentUrlKey1 = $I->grabFromCurrentUrl("/grabCurrentUrl"); @@ -124,9 +128,9 @@ class BasicFunctionalTestCest $I->pauseExecution(); $I->performOn("#selector", function(\WebDriverElement $el) {return $el->isDisplayed();}); $I->pressKey("#page", "a"); - $I->pressKey("#page", ['ctrl','a'],'new'); - $I->pressKey("#page", ['shift','111'],'1','x'); - $I->pressKey("#page", ['ctrl', 'a'],\Facebook\WebDriver\WebDriverKeys::DELETE); + $I->pressKey("#page", ['ctrl', 'a'],'new'); + $I->pressKey("#page", ['shift', '111'],'1','x'); + $I->pressKey("#page", ['ctrl', 'a'],\Facebook\WebDriver\WebDriverKeys::DELETE); $I->reloadPage(); $I->resetCookie("cookieInput"); $I->resizeWindow(0, 0); diff --git a/dev/tests/verification/Resources/ParameterArrayTest.txt b/dev/tests/verification/Resources/ParameterArrayTest.txt index 5564bddac..76e1bed32 100644 --- a/dev/tests/verification/Resources/ParameterArrayTest.txt +++ b/dev/tests/verification/Resources/ParameterArrayTest.txt @@ -51,5 +51,11 @@ class ParameterArrayTestCest $I->unselectOption("#selector", ["postname" . msq("simpleParamData")]); $I->unselectOption("#selector", [PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test')]); $I->unselectOption("#selector", ["name", PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test')]); + $I->pressKey("#selector", PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test'), ['ctrl', 'a'],\Facebook\WebDriver\WebDriverKeys::DELETE,PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test')); + $I->pressKey("#selector", ['ctrl', 'a'], 10, 20,\Facebook\WebDriver\WebDriverKeys::DELETE,PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test')); + $I->pressKey("#selector", ['ctrl', 'a'],'new', 10, 20,\Facebook\WebDriver\WebDriverKeys::DELETE,PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test')); + $I->pressKey("#selector", ['ctrl', 'a'],'new', 1, ['ctrl'], ['shift', 'ctrl', 'del'], [PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test'), 'a', "name"]); + $I->pressKey("#selector", ['ctrl', 'a'],'new', 1, ['ctrl'], ['shift', 'ctrl', 'del'], 0, [PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test'), PersistedObjectHandler::getInstance()->retrieveEntityField('simpleDataKey', 'name', 'test')]); + $I->pressKey("#selector", ['ctrl', 'a'],'new', 1, ['ctrl'], ['shift', 'ctrl', 'del'], [msq("simpleParamData") . "prename", "postname" . msq("simpleParamData")]); } } diff --git a/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml b/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml index 75ac71ef2..245447f7d 100644 --- a/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml +++ b/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml @@ -118,4 +118,12 @@ + + + + + + + + diff --git a/dev/tests/verification/TestModule/ActionGroup/FunctionalActionGroup.xml b/dev/tests/verification/TestModule/ActionGroup/FunctionalActionGroup.xml index 728b606b2..e6a7bad00 100644 --- a/dev/tests/verification/TestModule/ActionGroup/FunctionalActionGroup.xml +++ b/dev/tests/verification/TestModule/ActionGroup/FunctionalActionGroup.xml @@ -85,4 +85,10 @@ + + + + + + diff --git a/dev/tests/verification/TestModule/Test/ActionGroupTest.xml b/dev/tests/verification/TestModule/Test/ActionGroupTest.xml index 848fe8bcc..56be24441 100644 --- a/dev/tests/verification/TestModule/Test/ActionGroupTest.xml +++ b/dev/tests/verification/TestModule/Test/ActionGroupTest.xml @@ -158,4 +158,17 @@ + + + + + + + + + + + + + diff --git a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml index 322c5ac74..d120e5c31 100644 --- a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml +++ b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml @@ -62,6 +62,7 @@ + diff --git a/dev/tests/verification/TestModule/Test/ParameterArrayTest.xml b/dev/tests/verification/TestModule/Test/ParameterArrayTest.xml index 2d91671cb..e9bef4244 100644 --- a/dev/tests/verification/TestModule/Test/ParameterArrayTest.xml +++ b/dev/tests/verification/TestModule/Test/ParameterArrayTest.xml @@ -27,5 +27,12 @@ + + + + + + + diff --git a/dev/tests/verification/Tests/ActionGroupGenerationTest.php b/dev/tests/verification/Tests/ActionGroupGenerationTest.php index 65d65d04d..dec8c6d14 100644 --- a/dev/tests/verification/Tests/ActionGroupGenerationTest.php +++ b/dev/tests/verification/Tests/ActionGroupGenerationTest.php @@ -195,4 +195,26 @@ public function testActionGroupWithSkipReadiness() { $this->generateAndCompareTest('ActionGroupSkipReadiness'); } + + /** + * Test an action group with an arg containing stepKey text + * + * @throws \Exception + * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException + */ + public function testActionGroupWithSectionAndDataArguments() + { + $this->generateAndCompareTest('ActionGroupWithSectionAndDataAsArguments'); + } + + /** + * Test an action group with an arg that resolves into section.element with a hyphen in the parameter + * + * @throws \Exception + * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException + */ + public function testActionGroupWithHyphen() + { + $this->generateAndCompareTest('ActionGroupWithParameterizedElementWithHyphen'); + } } diff --git a/etc/config/.env.example b/etc/config/.env.example index c04afaab8..17ea384ca 100644 --- a/etc/config/.env.example +++ b/etc/config/.env.example @@ -31,6 +31,9 @@ BROWSER=chrome #FW_BP= #TESTS_MODULE_PATH= +#*** Uncomment this property to change the default timezone MFTF will use for the generateDate action ***# +#DEFAULT_TIMEZONE=America/Los_Angeles + #*** These properties impact the modules loaded into MFTF, you can point to your own full path, or a custom set of modules located with the core set MODULE_WHITELIST=Magento_Framework,Magento_ConfigurableProductWishlist,Magento_ConfigurableProductCatalogSearch #CUSTOM_MODULE_PATHS= diff --git a/etc/config/codeception.dist.yml b/etc/config/codeception.dist.yml index 273ede0b0..a4ea0dcb5 100755 --- a/etc/config/codeception.dist.yml +++ b/etc/config/codeception.dist.yml @@ -23,5 +23,7 @@ extensions: - env - zephyrId - useCaseId + Magento\FunctionalTestingFramework\Extension\TestContextExtension: + driver: \Magento\FunctionalTestingFramework\Module\MagentoWebDriver params: - .env \ No newline at end of file diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfDom.php b/src/Magento/FunctionalTestingFramework/Config/MftfDom.php index 2a4c5f8b6..a5fc31d4a 100644 --- a/src/Magento/FunctionalTestingFramework/Config/MftfDom.php +++ b/src/Magento/FunctionalTestingFramework/Config/MftfDom.php @@ -59,4 +59,18 @@ public function merge($xml, $filename = null, $exceptionCollector = null) $dom = $this->initDom($xml, $filename, $exceptionCollector); $this->mergeNode($dom->documentElement, ''); } + + /** + * Checks if the filename given ends with the correct suffix. + * @param string $filename + * @param string $suffix + * @return boolean + */ + public function checkFilenameSuffix($filename, $suffix) + { + if (substr_compare($filename, $suffix, -strlen($suffix)) === 0) { + return true; + } + return false; + } } diff --git a/src/Magento/FunctionalTestingFramework/DataGenerator/Config/Dom.php b/src/Magento/FunctionalTestingFramework/DataGenerator/Config/Dom.php index abef7390a..d4b8f7c8f 100644 --- a/src/Magento/FunctionalTestingFramework/DataGenerator/Config/Dom.php +++ b/src/Magento/FunctionalTestingFramework/DataGenerator/Config/Dom.php @@ -18,6 +18,7 @@ class Dom extends \Magento\FunctionalTestingFramework\Config\MftfDom { const DATA_FILE_NAME_ENDING = "Data"; const DATA_META_FILENAME_ATTRIBUTE = "filename"; + const DATA_META_NAME_ATTRIBUTE = "name"; /** * NodeValidationUtil @@ -74,7 +75,8 @@ public function initDom($xml, $filename = null) $entityNode->setAttribute(self::DATA_META_FILENAME_ATTRIBUTE, $filename); $this->validationUtil->validateChildUniqueness( $entityNode, - $filename + $filename, + $entityNode->getAttribute(self::DATA_META_NAME_ATTRIBUTE) ); } } diff --git a/src/Magento/FunctionalTestingFramework/DataGenerator/Config/OperationDom.php b/src/Magento/FunctionalTestingFramework/DataGenerator/Config/OperationDom.php index 98694ea47..e76289919 100644 --- a/src/Magento/FunctionalTestingFramework/DataGenerator/Config/OperationDom.php +++ b/src/Magento/FunctionalTestingFramework/DataGenerator/Config/OperationDom.php @@ -18,6 +18,7 @@ class OperationDom extends \Magento\FunctionalTestingFramework\Config\MftfDom { const METADATA_FILE_NAME_ENDING = "meta"; const METADATA_META_FILENAME_ATTRIBUTE = "filename"; + const METADATA_META_NAME_ATTRIBUTE = "name"; /** * NodeValidationUtil @@ -74,7 +75,8 @@ public function initDom($xml, $filename = null) $operationNode->setAttribute(self::METADATA_META_FILENAME_ATTRIBUTE, $filename); $this->validateOperationElements( $operationNode, - $filename + $filename, + $operationNode->getAttribute(self::METADATA_META_NAME_ATTRIBUTE) ); } } @@ -86,13 +88,15 @@ public function initDom($xml, $filename = null) * Recurse through child elements and validate uniqueKeys * @param \DOMElement $parentNode * @param string $filename + * @param string $topParent * @return void */ - public function validateOperationElements(\DOMElement $parentNode, $filename) + public function validateOperationElements(\DOMElement $parentNode, $filename, $topParent) { $this->validationUtil->validateChildUniqueness( $parentNode, - $filename + $filename, + $topParent ); $childNodes = $parentNode->childNodes; @@ -103,7 +107,8 @@ public function validateOperationElements(\DOMElement $parentNode, $filename) } $this->validateOperationElements( $currentNode, - $filename + $filename, + $topParent ); } } diff --git a/src/Magento/FunctionalTestingFramework/Extension/BaseExtension.php b/src/Magento/FunctionalTestingFramework/Extension/BaseExtension.php new file mode 100644 index 000000000..29f263ed2 --- /dev/null +++ b/src/Magento/FunctionalTestingFramework/Extension/BaseExtension.php @@ -0,0 +1,107 @@ + 'beforeTest', + Events::STEP_BEFORE => 'beforeStep' + ]; + + /** + * The current URI of the active page + * + * @var string + */ + private $uri; + + /** + * Codeception event listener function - initialize uri before test + * + * @param \Codeception\Event\TestEvent $e + * @return void + * @throws \Exception + * + * @SuppressWarnings(PHPMD.UnusedFormalParameter) + */ + public function beforeTest(\Codeception\Event\TestEvent $e) + { + $this->uri = null; + } + + /** + * Codeception event listener function - check for page uri change before step + * + * @param \Codeception\Event\StepEvent $e + * @return void + * @throws \Exception + * + * @SuppressWarnings(PHPMD.UnusedFormalParameter) + */ + public function beforeStep(\Codeception\Event\StepEvent $e) + { + $this->pageChanged(); + } + + /** + * WebDriver instance for execution + * + * @return WebDriver + * @throws ModuleRequireException + */ + public function getDriver() + { + return $this->getModule($this->config['driver']); + } + + /** + * Gets the active page URI from the start of the most recent step + * + * @return string + */ + public function getUri() + { + return $this->uri; + } + + /** + * Check if page uri has changed + * + * @return boolean + */ + protected function pageChanged() + { + try { + if ($this->getDriver() === null) { + return false; + } + $currentUri = $this->getDriver()->_getCurrentUri(); + + if ($this->uri !== $currentUri) { + $this->uri = $currentUri; + return true; + } + } catch (\Exception $e) { + // just fall through and return false + } + return false; + } +} diff --git a/src/Magento/FunctionalTestingFramework/Extension/ErrorLogger.php b/src/Magento/FunctionalTestingFramework/Extension/ErrorLogger.php index bc29fcd64..b0621df1b 100644 --- a/src/Magento/FunctionalTestingFramework/Extension/ErrorLogger.php +++ b/src/Magento/FunctionalTestingFramework/Extension/ErrorLogger.php @@ -41,17 +41,22 @@ private function __construct() /** * Loops through stepEvent for browser log entries - * @param \Facebook\WebDriver\Remote\RemoteWebDriver $webDriver - * @param \Codeception\Event\StepEvent $stepEvent + * + * @param \Magento\FunctionalTestingFramework\Module\MagentoWebDriver $module + * @param \Codeception\Event\StepEvent $stepEvent * @return void */ - public function logErrors($webDriver, $stepEvent) + public function logErrors($module, $stepEvent) { //Types available should be "server", "browser", "driver". Only care about browser at the moment. - $browserLogEntries = $webDriver->manage()->getLog("browser"); - foreach ($browserLogEntries as $entry) { - if (array_key_exists("source", $entry) && $entry["source"] === "javascript") { - $this->logError("javascript", $stepEvent, $entry); + if (in_array("browser", $module->webDriver->manage()->getAvailableLogTypes())) { + $browserLogEntries = $module->webDriver->manage()->getLog("browser"); + foreach ($browserLogEntries as $entry) { + if (array_key_exists("source", $entry) && $entry["source"] === "javascript") { + $this->logError("javascript", $stepEvent, $entry); + //Set javascript error in MagentoWebDriver internal array + $module->setJsError("ERROR({$entry["level"]}) - " . $entry["message"]); + } } } } diff --git a/src/Magento/FunctionalTestingFramework/Extension/PageReadinessExtension.php b/src/Magento/FunctionalTestingFramework/Extension/PageReadinessExtension.php index 709647096..8d49ad86c 100644 --- a/src/Magento/FunctionalTestingFramework/Extension/PageReadinessExtension.php +++ b/src/Magento/FunctionalTestingFramework/Extension/PageReadinessExtension.php @@ -8,10 +8,6 @@ use Codeception\Event\StepEvent; use Codeception\Event\TestEvent; -use Codeception\Events; -use Codeception\Exception\ModuleRequireException; -use Codeception\Extension; -use Codeception\Module\WebDriver; use Codeception\Step; use Facebook\WebDriver\Exception\UnexpectedAlertOpenException; use Magento\FunctionalTestingFramework\Extension\ReadinessMetrics\AbstractMetricCheck; @@ -23,18 +19,8 @@ /** * Class PageReadinessExtension */ -class PageReadinessExtension extends Extension +class PageReadinessExtension extends BaseExtension { - /** - * Codeception Events Mapping to methods - * - * @var array - */ - public static $events = [ - Events::TEST_BEFORE => 'beforeTest', - Events::STEP_BEFORE => 'beforeStep' - ]; - /** * List of action types that should bypass metric checks * shouldSkipCheck() also checks for the 'Comment' step type, which doesn't follow the $step->getAction() pattern @@ -73,13 +59,6 @@ class PageReadinessExtension extends Extension */ private $testName; - /** - * The current URI of the active page - * - * @var string - */ - private $uri; - /** * Initialize local vars * @@ -90,27 +69,19 @@ public function _initialize() { $this->logger = LoggingUtil::getInstance()->getLogger(get_class($this)); $this->verbose = MftfApplicationConfig::getConfig()->verboseEnabled(); - } - - /** - * WebDriver instance to use to execute readiness metric checks - * - * @return WebDriver - * @throws ModuleRequireException - */ - public function getDriver() - { - return $this->getModule($this->config['driver']); + parent::_initialize(); } /** * Initialize the readiness metrics for the test * - * @param \Codeception\Event\TestEvent $e + * @param TestEvent $e * @return void + * @throws \Exception */ public function beforeTest(TestEvent $e) { + parent::beforeTest($e); if (isset($this->config['resetFailureThreshold'])) { $failThreshold = intval($this->config['resetFailureThreshold']); } else { @@ -118,7 +89,6 @@ public function beforeTest(TestEvent $e) } $this->testName = $e->getTest()->getMetadata()->getName(); - $this->uri = null; $this->getDriver()->_setConfig(['skipReadiness' => false]); @@ -136,6 +106,8 @@ public function beforeTest(TestEvent $e) * @param StepEvent $e * @return void * @throws \Exception + * + * @SuppressWarnings(PHPMD.CyclomaticComplexity) */ public function beforeStep(StepEvent $e) { @@ -145,7 +117,20 @@ public function beforeStep(StepEvent $e) return; } - $this->checkForNewPage($step); + // Check if page has changed and reset metric tracking if so + if ($this->pageChanged($step)) { + $this->logDebug( + 'Page URI changed; resetting readiness metric failure tracking', + [ + 'step' => $step->__toString(), + 'newUri' => $this->getUri() + ] + ); + /** @var AbstractMetricCheck $metric */ + foreach ($this->readinessMetrics as $metric) { + $metric->resetTracker(); + } + } // todo: Implement step parameter to override global timeout configuration if (isset($this->config['timeout'])) { @@ -182,48 +167,6 @@ function () use ($metrics) { } } - /** - * Check if the URI has changed and reset metric tracking if so - * - * @param Step $step - * @return void - */ - private function checkForNewPage($step) - { - try { - $currentUri = $this->getDriver()->_getCurrentUri(); - - if ($this->uri !== $currentUri) { - $this->logDebug( - 'Page URI changed; resetting readiness metric failure tracking', - [ - 'step' => $step->__toString(), - 'newUri' => $currentUri - ] - ); - - /** @var AbstractMetricCheck $metric */ - foreach ($this->readinessMetrics as $metric) { - $metric->resetTracker(); - } - - $this->uri = $currentUri; - } - } catch (\Exception $e) { - $this->logDebug('Could not retrieve current URI', ['step' => $step->__toString()]); - } - } - - /** - * Gets the active page URI from the start of the most recent step - * - * @return string - */ - public function getUri() - { - return $this->uri; - } - /** * Gets the name of the active test * @@ -263,7 +206,7 @@ private function logDebug($message, $context = []) if ($this->verbose) { $logContext = [ 'test' => $this->testName, - 'uri' => $this->uri + 'uri' => $this->getUri() ]; foreach ($this->readinessMetrics as $metric) { $logContext[$metric->getName()] = $metric->getStoredValue(); diff --git a/src/Magento/FunctionalTestingFramework/Extension/ReadinessMetrics/AbstractMetricCheck.php b/src/Magento/FunctionalTestingFramework/Extension/ReadinessMetrics/AbstractMetricCheck.php index 417ae336f..8b611283d 100644 --- a/src/Magento/FunctionalTestingFramework/Extension/ReadinessMetrics/AbstractMetricCheck.php +++ b/src/Magento/FunctionalTestingFramework/Extension/ReadinessMetrics/AbstractMetricCheck.php @@ -214,7 +214,6 @@ protected function getDriver() * @param string $script * @param array $arguments * @return mixed - * @throws UnexpectedAlertOpenException * @throws ModuleRequireException */ protected function executeJs($script, $arguments = []) diff --git a/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php b/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php index 07f177dc3..3895d9efd 100644 --- a/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php +++ b/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php @@ -6,32 +6,40 @@ namespace Magento\FunctionalTestingFramework\Extension; -use \Codeception\Events; +use Codeception\Events; use Magento\FunctionalTestingFramework\DataGenerator\Handlers\PersistedObjectHandler; -use Magento\FunctionalTestingFramework\Extension\ErrorLogger; -use Magento\FunctionalTestingFramework\Module\MagentoWebDriver; /** * Class TestContextExtension * @SuppressWarnings(PHPMD.UnusedPrivateField) */ -class TestContextExtension extends \Codeception\Extension +class TestContextExtension extends BaseExtension { const TEST_PHASE_AFTER = "_after"; - // @codingStandardsIgnoreStart - const MAGENTO_WEB_DRIVER_CLASS = "\Magento\FunctionalTestingFramework\Module\MagentoWebDriver"; - // @codingStandardsIgnoreEnd /** * Codeception Events Mapping to methods * @var array */ - public static $events = [ - Events::TEST_START => 'testStart', - Events::TEST_FAIL => 'testFail', - Events::STEP_AFTER => 'afterStep', - Events::TEST_END => 'testEnd' - ]; + public static $events; + + /** + * Initialize local vars + * + * @return void + * @throws \Exception + */ + public function _initialize() + { + $events = [ + Events::TEST_START => 'testStart', + Events::TEST_FAIL => 'testFail', + Events::STEP_AFTER => 'afterStep', + Events::TEST_END => 'testEnd' + ]; + self::$events = array_merge(parent::$events, $events); + parent::_initialize(); + } /** * Codeception event listener function, triggered on test start. @@ -64,6 +72,7 @@ public function testFail(\Codeception\Event\FailEvent $e) * Codeception event listener function, triggered on test ending (naturally or by error). * @param \Codeception\Event\TestEvent $e * @return void + * @throws \Exception */ public function testEnd(\Codeception\Event\TestEvent $e) { @@ -92,13 +101,13 @@ function () use ($cest) { } } // Reset Session and Cookies after all Test Runs, workaround due to functional.suite.yml restart: true - $this->getModule(self::MAGENTO_WEB_DRIVER_CLASS)->_runAfter($e->getTest()); + $this->getDriver()->_runAfter($e->getTest()); } /** * Runs cest's after block, if necessary. - * @param Symfony\Component\EventDispatcher\Event $e - * @param \Codeception\TestInterface $cest + * @param \Symfony\Component\EventDispatcher\Event $e + * @param \Codeception\TestInterface $cest * @return void */ private function runAfterBlock($e, $cest) @@ -135,17 +144,30 @@ public function extractContext($trace, $class) return null; } + /** + * Codeception event listener function, triggered before step. + * Check if it's a new page. + * + * @param \Codeception\Event\StepEvent $e + * @return void + * @throws \Exception + */ + public function beforeStep(\Codeception\Event\StepEvent $e) + { + if ($this->pageChanged($e->getStep())) { + $this->getDriver()->cleanJsError(); + } + } + /** * Codeception event listener function, triggered after step. * Calls ErrorLogger to log JS errors encountered. * @param \Codeception\Event\StepEvent $e * @return void + * @throws \Exception */ public function afterStep(\Codeception\Event\StepEvent $e) { - // @codingStandardsIgnoreStart - $webDriver = $this->getModule("\Magento\FunctionalTestingFramework\Module\MagentoWebDriver")->webDriver; - // @codingStandardsIgnoreEnd - ErrorLogger::getInstance()->logErrors($webDriver, $e); + ErrorLogger::getInstance()->logErrors($this->getDriver(), $e); } } diff --git a/src/Magento/FunctionalTestingFramework/Module/MagentoWebDriver.php b/src/Magento/FunctionalTestingFramework/Module/MagentoWebDriver.php index cab4bb7e3..c64aa3721 100644 --- a/src/Magento/FunctionalTestingFramework/Module/MagentoWebDriver.php +++ b/src/Magento/FunctionalTestingFramework/Module/MagentoWebDriver.php @@ -105,6 +105,13 @@ class MagentoWebDriver extends WebDriver */ private $htmlReport; + /** + * Array to store Javascript errors + * + * @var string[] + */ + private $jsErrors = []; + /** * Sanitizes config, then initializes using parent. * @return void @@ -113,6 +120,7 @@ public function _initialize() { $this->config = ConfigSanitizerUtil::sanitizeWebDriverConfig($this->config); parent::_initialize(); + $this->cleanJsError(); } /** @@ -124,6 +132,7 @@ public function _resetConfig() { parent::_resetConfig(); $this->config = ConfigSanitizerUtil::sanitizeWebDriverConfig($this->config); + $this->cleanJsError(); } /** @@ -367,16 +376,17 @@ public function waitForPageLoad($timeout = null) $this->waitForJS('return document.readyState == "complete"', $timeout); $this->waitForAjaxLoad($timeout); - $this->waitForLoadingMaskToDisappear(); + $this->waitForLoadingMaskToDisappear($timeout); } /** * Wait for all visible loading masks to disappear. Gets all elements by mask selector, then loops over them. * + * @param integer $timeout * @throws \Exception * @return void */ - public function waitForLoadingMaskToDisappear() + public function waitForLoadingMaskToDisappear($timeout = null) { foreach (self::$loadingMasksLocators as $maskLocator) { // Get count of elements found for looping. @@ -385,23 +395,7 @@ public function waitForLoadingMaskToDisappear() for ($i = 1; $i <= count($loadingMaskElements); $i++) { // Formatting and looping on i as we can't interact elements returned above // eg. (//div[@data-role="spinner"])[1] - $this->waitForElementNotVisible("({$maskLocator})[{$i}]", 30); - } - } - } - - /** - * Verify that there are no JavaScript errors in the console. - * - * @throws ModuleException - * @return void - */ - public function dontSeeJsError() - { - $logs = $this->webDriver->manage()->getLog('browser'); - foreach ($logs as $log) { - if ($log['level'] == 'SEVERE') { - throw new ModuleException($this, 'Errors in JavaScript: ' . json_encode($log)); + $this->waitForElementNotVisible("({$maskLocator})[{$i}]", $timeout); } } } @@ -708,4 +702,53 @@ public function skipReadinessCheck($check) { $this->config['skipReadiness'] = $check; } + + /** + * Clean Javascript errors in internal array + * + * @return void + */ + public function cleanJsError() + { + $this->jsErrors = []; + } + + /** + * Save Javascript error message to internal array + * + * @param string $errMsg + * @return void + */ + public function setJsError($errMsg) + { + $this->jsErrors[] = $errMsg; + } + + /** + * Get all Javascript errors + * + * @return string + */ + private function getJsErrors() + { + $errors = ''; + + if (!empty($this->jsErrors)) { + $errors = 'Errors in JavaScript:'; + foreach ($this->jsErrors as $jsError) { + $errors .= "\n" . $jsError; + } + } + return $errors; + } + + /** + * Verify that there is no JavaScript error in browser logs + * + * @return void + */ + public function dontSeeJsError() + { + $this->assertEmpty($this->jsErrors, $this->getJsErrors()); + } } diff --git a/src/Magento/FunctionalTestingFramework/Page/Config/Dom.php b/src/Magento/FunctionalTestingFramework/Page/Config/Dom.php index 0d8739636..bb6d08f58 100644 --- a/src/Magento/FunctionalTestingFramework/Page/Config/Dom.php +++ b/src/Magento/FunctionalTestingFramework/Page/Config/Dom.php @@ -20,6 +20,7 @@ class Dom extends \Magento\FunctionalTestingFramework\Config\MftfDom { const PAGE_META_FILENAME_ATTRIBUTE = "filename"; + const PAGE_META_NAME_ATTRIBUTE = "name"; /** * Module Path extractor @@ -79,7 +80,11 @@ public function initDom($xml, $filename = null) $dom = parent::initDom($xml, $filename); $pagesNode = $dom->getElementsByTagName('pages')->item(0); - $this->validationUtil->validateChildUniqueness($pagesNode, $filename); + $this->validationUtil->validateChildUniqueness( + $pagesNode, + $filename, + $pagesNode->getAttribute(self::PAGE_META_NAME_ATTRIBUTE) + ); $pageNodes = $dom->getElementsByTagName('page'); $currentModule = $this->modulePathExtractor->extractModuleName($filename) . diff --git a/src/Magento/FunctionalTestingFramework/Page/Config/SectionDom.php b/src/Magento/FunctionalTestingFramework/Page/Config/SectionDom.php index c3476e7dd..b7f5d5db9 100644 --- a/src/Magento/FunctionalTestingFramework/Page/Config/SectionDom.php +++ b/src/Magento/FunctionalTestingFramework/Page/Config/SectionDom.php @@ -20,6 +20,7 @@ class SectionDom extends \Magento\FunctionalTestingFramework\Config\MftfDom { const SECTION_META_FILENAME_ATTRIBUTE = "filename"; + const SECTION_META_NAME_ATTRIBUTE = "name"; /** * NodeValidationUtil @@ -71,7 +72,11 @@ public function initDom($xml, $filename = null) $sectionNodes = $dom->getElementsByTagName('section'); foreach ($sectionNodes as $sectionNode) { $sectionNode->setAttribute(self::SECTION_META_FILENAME_ATTRIBUTE, $filename); - $this->validationUtil->validateChildUniqueness($sectionNode, $filename); + $this->validationUtil->validateChildUniqueness( + $sectionNode, + $filename, + $sectionNode->getAttribute(self::SECTION_META_NAME_ATTRIBUTE) + ); } return $dom; } diff --git a/src/Magento/FunctionalTestingFramework/Test/Config/ActionGroupDom.php b/src/Magento/FunctionalTestingFramework/Test/Config/ActionGroupDom.php index 36f7a4384..7eec3286d 100644 --- a/src/Magento/FunctionalTestingFramework/Test/Config/ActionGroupDom.php +++ b/src/Magento/FunctionalTestingFramework/Test/Config/ActionGroupDom.php @@ -15,6 +15,7 @@ class ActionGroupDom extends Dom { const ACTION_GROUP_FILE_NAME_ENDING = "ActionGroup.xml"; + const ACTION_GROUP_META_NAME_ATTRIBUTE = "name"; /** * Takes a dom element from xml and appends the filename based on location while also validating the action group @@ -28,29 +29,23 @@ public function initDom($xml, $filename = null) { $dom = parent::initDom($xml, $filename); - if (strpos($filename, self::ACTION_GROUP_FILE_NAME_ENDING)) { + if ($this->checkFilenameSuffix($filename, self::ACTION_GROUP_FILE_NAME_ENDING)) { + $actionGroupsNode = $dom->getElementsByTagName('actionGroups')[0]; $actionGroupNodes = $dom->getElementsByTagName('actionGroup'); + + $this->testsValidationUtil->validateChildUniqueness( + $actionGroupsNode, + $filename, + null + ); foreach ($actionGroupNodes as $actionGroupNode) { /** @var \DOMElement $actionGroupNode */ $actionGroupNode->setAttribute(self::TEST_META_FILENAME_ATTRIBUTE, $filename); - $this->validationUtil->validateChildUniqueness( + $this->actionsValidationUtil->validateChildUniqueness( $actionGroupNode, - $filename + $filename, + $actionGroupNode->getAttribute(self::ACTION_GROUP_META_NAME_ATTRIBUTE) ); - $beforeNode = $actionGroupNode->getElementsByTagName('before')->item(0); - $afterNode = $actionGroupNode->getElementsByTagName('after')->item(0); - if (isset($beforeNode)) { - $this->validationUtil->validateChildUniqueness( - $beforeNode, - $filename - ); - } - if (isset($afterNode)) { - $this->validationUtil->validateChildUniqueness( - $afterNode, - $filename - ); - } if ($actionGroupNode->getAttribute(self::TEST_MERGE_POINTER_AFTER) !== "") { $this->appendMergePointerToActions( $actionGroupNode, diff --git a/src/Magento/FunctionalTestingFramework/Test/Config/Dom.php b/src/Magento/FunctionalTestingFramework/Test/Config/Dom.php index 828586b3c..c13e8f056 100644 --- a/src/Magento/FunctionalTestingFramework/Test/Config/Dom.php +++ b/src/Magento/FunctionalTestingFramework/Test/Config/Dom.php @@ -19,7 +19,7 @@ */ class Dom extends \Magento\FunctionalTestingFramework\Config\MftfDom { - const TEST_FILE_NAME_ENDING = 'Test'; + const TEST_FILE_NAME_ENDING = 'Test.xml'; const TEST_META_FILENAME_ATTRIBUTE = 'filename'; const TEST_META_NAME_ATTRIBUTE = 'name'; const TEST_HOOK_NAMES = ["after", "before"]; @@ -27,10 +27,16 @@ class Dom extends \Magento\FunctionalTestingFramework\Config\MftfDom const TEST_MERGE_POINTER_AFTER = "insertAfter"; /** - * NodeValidationUtil + * NodeValidationUtil for test actions * @var DuplicateNodeValidationUtil */ - protected $validationUtil; + protected $actionsValidationUtil; + + /** + * NodeValidationUtil for test names + * @var DuplicateNodeValidationUtil + */ + protected $testsValidationUtil; /** * ExceptionCollector @@ -57,7 +63,8 @@ public function __construct( $schemaFile = null, $errorFormat = self::ERROR_FORMAT_DEFAULT ) { - $this->validationUtil = new DuplicateNodeValidationUtil('stepKey', $exceptionCollector); + $this->actionsValidationUtil = new DuplicateNodeValidationUtil('stepKey', $exceptionCollector); + $this->testsValidationUtil = new DuplicateNodeValidationUtil('name', $exceptionCollector); $this->exceptionCollector = $exceptionCollector; parent::__construct( $xml, @@ -81,8 +88,14 @@ public function initDom($xml, $filename = null) { $dom = parent::initDom($xml, $filename); - if (strpos($filename, self::TEST_FILE_NAME_ENDING)) { + if ($this->checkFilenameSuffix($filename, self::TEST_FILE_NAME_ENDING)) { + $testsNode = $dom->getElementsByTagName('tests')[0]; $testNodes = $dom->getElementsByTagName('test'); + $this->testsValidationUtil->validateChildUniqueness( + $testsNode, + $filename, + null + ); foreach ($testNodes as $testNode) { /** @var \DOMElement $testNode */ $testNode->setAttribute(self::TEST_META_FILENAME_ATTRIBUTE, $filename); @@ -102,23 +115,26 @@ public function initDom($xml, $filename = null) ); } - $this->validationUtil->validateChildUniqueness( + $this->actionsValidationUtil->validateChildUniqueness( $testNode, - $filename + $filename, + $testNode->getAttribute(self::TEST_META_NAME_ATTRIBUTE) ); $beforeNode = $testNode->getElementsByTagName('before')->item(0); $afterNode = $testNode->getElementsByTagName('after')->item(0); if (isset($beforeNode)) { - $this->validationUtil->validateChildUniqueness( + $this->actionsValidationUtil->validateChildUniqueness( $beforeNode, - $filename + $filename, + $testNode->getAttribute(self::TEST_META_NAME_ATTRIBUTE) . "/before" ); } if (isset($afterNode)) { - $this->validationUtil->validateChildUniqueness( + $this->actionsValidationUtil->validateChildUniqueness( $afterNode, - $filename + $filename, + $testNode->getAttribute(self::TEST_META_NAME_ATTRIBUTE) . "/after" ); } } diff --git a/src/Magento/FunctionalTestingFramework/Test/Objects/ActionGroupObject.php b/src/Magento/FunctionalTestingFramework/Test/Objects/ActionGroupObject.php index 2b63709fc..67c1ba25c 100644 --- a/src/Magento/FunctionalTestingFramework/Test/Objects/ActionGroupObject.php +++ b/src/Magento/FunctionalTestingFramework/Test/Objects/ActionGroupObject.php @@ -204,7 +204,7 @@ private function resolveAttributesWithArguments($arguments, $attributes) // $regexPattern match on: $matches[0] {{section.element(arg.field)}} // $matches[1] = section.element // $matches[2] = arg.field - $regexPattern = '/{{([\w.\[\]]+)\(*([\w.$\',\s\[\]]+)*\)*}}/'; + $regexPattern = '/{{([^(}]+)\(*([^)}]+)*\)*}}/'; $newActionAttributes = []; foreach ($attributes as $attributeKey => $attributeValue) { diff --git a/src/Magento/FunctionalTestingFramework/Test/Objects/ActionObject.php b/src/Magento/FunctionalTestingFramework/Test/Objects/ActionObject.php index 9f851b130..c91ebaf5c 100644 --- a/src/Magento/FunctionalTestingFramework/Test/Objects/ActionObject.php +++ b/src/Magento/FunctionalTestingFramework/Test/Objects/ActionObject.php @@ -19,6 +19,7 @@ /** * Class ActionObject + * @SuppressWarnings(PHPMD.CouplingBetweenObjects) */ class ActionObject { @@ -62,6 +63,7 @@ class ActionObject const FUNCTION_CLOSURE_ACTIONS = ['waitForElementChange', 'performOn']; const MERGE_ACTION_ORDER_AFTER = 'after'; const MERGE_ACTION_ORDER_BEFORE = 'before'; + const ACTION_ATTRIBUTE_TIMEZONE = 'timezone'; const ACTION_ATTRIBUTE_URL = 'url'; const ACTION_ATTRIBUTE_SELECTOR = 'selector'; const ACTION_ATTRIBUTE_VARIABLE_REGEX_PARAMETER = '/\(.+\)/'; @@ -256,6 +258,7 @@ public function resolveReferences() $this->resolveSelectorReferenceAndTimeout(); $this->resolveUrlReference(); $this->resolveDataInputReferences(); + $this->validateTimezoneAttribute(); if ($this->getType() == "deleteData") { $this->validateMutuallyExclusiveAttributes(self::DELETE_DATA_MUTUAL_EXCLUSIVE_ATTRIBUTES); } @@ -599,6 +602,28 @@ private function validateUrlAreaAgainstActionType($obj) } } + /** + * Validates that the timezone attribute contains a valid value. + * + * @return void + * @throws TestReferenceException + */ + private function validateTimezoneAttribute() + { + $attributes = $this->getCustomActionAttributes(); + if (isset($attributes[self::ACTION_ATTRIBUTE_TIMEZONE])) { + $timezone = $attributes[self::ACTION_ATTRIBUTE_TIMEZONE]; + try { + new \DateTimeZone($timezone); + } catch (\Exception $e) { + throw new TestReferenceException( + "Timezone '{$timezone}' is not a valid timezone", + ["stepKey" => $this->getStepKey(), self::ACTION_ATTRIBUTE_TIMEZONE => $timezone] + ); + } + } + } + /** * Gets the object's dataByName with given $match, differentiating behavior between and nodes. * @param string $obj @@ -633,7 +658,7 @@ private function resolveEntityDataObjectReference($obj, $match) private function resolveParameterization($isParameterized, $replacement, $match, $object) { if ($isParameterized) { - $parameterList = $this->stripAndReturnParameters($match); + $parameterList = $this->stripAndReturnParameters($match) ?: []; $resolvedReplacement = $this->matchParameterReferences($replacement, $parameterList); } else { $resolvedReplacement = $replacement; @@ -657,26 +682,7 @@ private function matchParameterReferences($reference, $parameters) { preg_match_all('/{{[\w.]+}}/', $reference, $varMatches); $varMatches[0] = array_unique($varMatches[0]); - if (count($varMatches[0]) > count($parameters)) { - if (is_array($parameters)) { - $parametersGiven = implode(",", $parameters); - } elseif ($parameters == null) { - $parametersGiven = "NONE"; - } else { - $parametersGiven = $parameters; - } - throw new TestReferenceException( - "Parameter Resolution Failed: Not enough parameters given for reference " . - $reference . ". Parameters Given: " . $parametersGiven, - ["reference" => $reference, "parametersGiven" => $parametersGiven] - ); - } elseif (count($varMatches[0]) < count($parameters)) { - throw new TestReferenceException( - "Parameter Resolution Failed: Too many parameters given for reference " . - $reference . ". Parameters Given: " . implode(", ", $parameters), - ["reference" => $reference, "parametersGiven" => $parameters] - ); - } + $this->checkParameterCount($varMatches[0], $parameters, $reference); //Attempt to Resolve {{data}} references to actual output. Trim parameter for whitespace before processing it. //If regex matched it means that it's either a 'StringLiteral' or $key.data$/$$key.data$$ reference. @@ -705,4 +711,43 @@ private function matchParameterReferences($reference, $parameters) } return $reference; } + + /** + * Checks count of parameters versus matches + * + * @param array $matches + * @param array $parameters + * @param string $reference + * @return void + * @throws \Exception + */ + private function checkParameterCount($matches, $parameters, $reference) + { + if (count($matches) > count($parameters)) { + if (is_array($parameters)) { + $parametersGiven = implode(",", $parameters); + } elseif ($parameters == null) { + $parametersGiven = "NONE"; + } else { + $parametersGiven = $parameters; + } + throw new TestReferenceException( + "Parameter Resolution Failed: Not enough parameters given for reference " . + $reference . ". Parameters Given: " . $parametersGiven, + ["reference" => $reference, "parametersGiven" => $parametersGiven] + ); + } elseif (count($matches) < count($parameters)) { + throw new TestReferenceException( + "Parameter Resolution Failed: Too many parameters given for reference " . + $reference . ". Parameters Given: " . implode(", ", $parameters), + ["reference" => $reference, "parametersGiven" => $parameters] + ); + } elseif (count($matches) == 0) { + throw new TestReferenceException( + "Parameter Resolution Failed: No parameter matches found in parameterized element with selector " . + $reference, + ["reference" => $reference] + ); + } + } } diff --git a/src/Magento/FunctionalTestingFramework/Test/Objects/ArgumentObject.php b/src/Magento/FunctionalTestingFramework/Test/Objects/ArgumentObject.php index c829e66b3..d553993e0 100644 --- a/src/Magento/FunctionalTestingFramework/Test/Objects/ArgumentObject.php +++ b/src/Magento/FunctionalTestingFramework/Test/Objects/ArgumentObject.php @@ -104,8 +104,8 @@ public function getResolvedValue($isInnerArgument) * Takes in boolean to determine if the replacement is being done with an inner argument (as in if it's a parameter) * * Example Type Non Inner Inner - * {{XML.DATA}}: {{XML.DATA}} XML.DATA - * $TEST.DATA$: $TEST.DATA$ $TEST.DATA$ + * {{XML.DATA}} {{XML.DATA}} XML.DATA + * $TEST.DATA$ $TEST.DATA$ $TEST.DATA$ * stringLiteral stringLiteral 'stringLiteral' * * @param boolean $isInnerArgument @@ -114,6 +114,11 @@ public function getResolvedValue($isInnerArgument) private function resolveStringArgument($isInnerArgument) { if ($isInnerArgument) { + if (preg_match('/{{[\w.\[\]]+}}/', $this->value)) { + return ltrim(rtrim($this->value, "}"), "{"); + } elseif (preg_match('/\${1,2}[\w.\[\]]+\${1,2}/', $this->value)) { + return $this->value; + } return "'" . $this->value . "'"; } else { return $this->value; diff --git a/src/Magento/FunctionalTestingFramework/Test/Util/ActionMergeUtil.php b/src/Magento/FunctionalTestingFramework/Test/Util/ActionMergeUtil.php index 0a6bc7957..65fe7c857 100644 --- a/src/Magento/FunctionalTestingFramework/Test/Util/ActionMergeUtil.php +++ b/src/Magento/FunctionalTestingFramework/Test/Util/ActionMergeUtil.php @@ -266,11 +266,19 @@ private function insertReadinessSkips() private function sortActions($parsedSteps) { foreach ($parsedSteps as $parsedStep) { - $parsedStep->resolveReferences(); - if ($parsedStep->getLinkedAction()) { - $this->stepsToMerge[$parsedStep->getStepKey()] = $parsedStep; - } else { - $this->orderedSteps[$parsedStep->getStepKey()] = $parsedStep; + try { + $parsedStep->resolveReferences(); + + if ($parsedStep->getLinkedAction()) { + $this->stepsToMerge[$parsedStep->getStepKey()] = $parsedStep; + } else { + $this->orderedSteps[$parsedStep->getStepKey()] = $parsedStep; + } + } catch (\Exception $e) { + throw new TestReferenceException( + $e->getMessage() . + ".\nException occurred parsing action at StepKey \"" . $parsedStep->getStepKey() . "\"" + ); } } } diff --git a/src/Magento/FunctionalTestingFramework/Util/ModuleResolver.php b/src/Magento/FunctionalTestingFramework/Util/ModuleResolver.php index 0a5974387..b2e1d32a2 100644 --- a/src/Magento/FunctionalTestingFramework/Util/ModuleResolver.php +++ b/src/Magento/FunctionalTestingFramework/Util/ModuleResolver.php @@ -28,6 +28,21 @@ class ModuleResolver */ const CUSTOM_MODULE_PATHS = 'CUSTOM_MODULE_PATHS'; + /** + * List of path types present in Magento Component Registrar + */ + const PATHS = ['module', 'library', 'theme', 'language']; + + /** + * Magento Registrar Class + */ + const REGISTRAR_CLASS = "\Magento\Framework\Component\ComponentRegistrar"; + + /** + * Magento Directory Structure Name Prefix + */ + const MAGENTO_PREFIX = "Magento_"; + /** * Enabled modules. * @@ -218,25 +233,20 @@ private function aggregateTestModulePaths() { $allModulePaths = []; - // Define the Module paths from app/code - $appCodePath = MAGENTO_BP - . DIRECTORY_SEPARATOR - . 'app' . DIRECTORY_SEPARATOR - . 'code' . DIRECTORY_SEPARATOR; + // Define the Module paths from magento bp + $magentoBaseCodePath = MAGENTO_BP; // Define the Module paths from default TESTS_MODULE_PATH $modulePath = defined('TESTS_MODULE_PATH') ? TESTS_MODULE_PATH : TESTS_BP; $modulePath = rtrim($modulePath, DIRECTORY_SEPARATOR); - // Define the Module paths from vendor modules - $vendorCodePath = PROJECT_ROOT - . DIRECTORY_SEPARATOR - . 'vendor' . DIRECTORY_SEPARATOR; + // Define the Module paths from project root + $projectRootCodePath = PROJECT_ROOT; $codePathsToPattern = [ $modulePath => '', - $appCodePath => DIRECTORY_SEPARATOR . 'Test' . DIRECTORY_SEPARATOR . 'Mftf', - $vendorCodePath => DIRECTORY_SEPARATOR . 'Test' . DIRECTORY_SEPARATOR . 'Mftf' + $magentoBaseCodePath => 'Test' . DIRECTORY_SEPARATOR . 'Mftf', + $projectRootCodePath => 'Test' . DIRECTORY_SEPARATOR . 'Mftf' ]; foreach ($codePathsToPattern as $codePath => $pattern) { @@ -264,8 +274,11 @@ private function globRelevantPaths($testPath, $pattern) $relevantPaths = $this->globRelevantWrapper($testPath, $pattern); } + $allComponents = $this->getRegisteredModuleList(); + foreach ($relevantPaths as $codePath) { - $mainModName = basename(str_replace($pattern, '', $codePath)); + $mainModName = array_search($codePath, $allComponents) ?: basename(str_replace($pattern, '', $codePath)); + $mainModName = str_replace(self::MAGENTO_PREFIX, "", $mainModName); $modulePaths[$mainModName][] = $codePath; if (MftfApplicationConfig::getConfig()->verboseEnabled()) { @@ -288,7 +301,15 @@ private function globRelevantPaths($testPath, $pattern) */ private static function globRelevantWrapper($testPath, $pattern) { - return glob($testPath . '*' . DIRECTORY_SEPARATOR . '*' . $pattern); + if ($pattern == "") { + return glob($testPath . '*' . DIRECTORY_SEPARATOR . '*' . $pattern); + } + $subDirectory = "*" . DIRECTORY_SEPARATOR; + $directories = glob($testPath . $subDirectory . $pattern, GLOB_ONLYDIR); + foreach (glob($testPath . $subDirectory, GLOB_ONLYDIR) as $dir) { + $directories = array_merge_recursive($directories, self::globRelevantWrapper($dir, $pattern)); + } + return $directories; } /** @@ -504,4 +525,42 @@ private function getModuleBlacklist() { return $this->moduleBlacklist; } + + /** + * Calls Magento method for determining registered modules. + * + * @return string[] + */ + private function getRegisteredModuleList() + { + if (array_key_exists('MAGENTO_BP', $_ENV)) { + $autoloadPath = realpath(MAGENTO_BP . "/app/autoload.php"); + if ($autoloadPath) { + require_once($autoloadPath); + } else { + throw new TestFrameworkException("Magento app/autoload.php not found with given MAGENTO_BP:" + . MAGENTO_BP); + } + } + + try { + $allComponents = []; + if (!class_exists(self::REGISTRAR_CLASS)) { + throw new TestFrameworkException("Magento Installation not found when loading registered modules.\n"); + } + $components = new \Magento\Framework\Component\ComponentRegistrar(); + foreach (self::PATHS as $componentType) { + $allComponents = array_merge($allComponents, $components->getPaths($componentType)); + } + array_walk($allComponents, function (&$value) { + $value .= DIRECTORY_SEPARATOR . 'Test' . DIRECTORY_SEPARATOR . 'Mftf'; + }); + return $allComponents; + } catch (TestFrameworkException $e) { + LoggingUtil::getInstance()->getLogger(ModuleResolver::class)->warning( + "$e" + ); + } + return []; + } } diff --git a/src/Magento/FunctionalTestingFramework/Util/TestGenerator.php b/src/Magento/FunctionalTestingFramework/Util/TestGenerator.php index eebaccf26..533e09a51 100644 --- a/src/Magento/FunctionalTestingFramework/Util/TestGenerator.php +++ b/src/Magento/FunctionalTestingFramework/Util/TestGenerator.php @@ -37,6 +37,7 @@ class TestGenerator const TEST_SCOPE = 'test'; const HOOK_SCOPE = 'hook'; const SUITE_SCOPE = 'suite'; + const PRESSKEY_ARRAY_ANCHOR_KEY = '987654321098765432109876543210'; /** * Path to the export dir. @@ -960,24 +961,7 @@ public function generateStepsPhp($actionObjects, $generationScope = TestGenerato case "pressKey": $parameterArray = $customActionAttributes['parameterArray'] ?? null; if ($parameterArray) { - // validate the param array is in the correct format - $this->validateParameterArray($parameterArray); - - // trim off the outer braces and add commas for the regex match - $params = "," . substr($parameterArray, 1, strlen($parameterArray) - 2) . ","; - - // we are matching any nested arrays for a simultaneous press, any string literals, and any - // explicit function calls from a class. - preg_match_all('/(\[.*?\])|(\'.*?\')|(\\\\.*?\,)/', $params, $paramInput); - - //clean up the input by trimming any extra commas - $tmpParameterArray = []; - foreach ($paramInput[0] as $params) { - $tmpParameterArray[] = trim($params, ","); - } - - // put the array together as a string to be passed as args - $parameterArray = implode(",", $tmpParameterArray); + $parameterArray = $this->processPressKey($parameterArray); } $testSteps .= $this->wrapFunctionCall( $actor, @@ -1283,7 +1267,7 @@ public function generateStepsPhp($actionObjects, $generationScope = TestGenerato $testSteps .= $argRef; break; case "generateDate": - $timezone = "America/Los_Angeles"; + $timezone = getenv("DEFAULT_TIMEZONE"); if (isset($customActionAttributes['timezone'])) { $timezone = $customActionAttributes['timezone']; } @@ -1593,7 +1577,7 @@ private function generateTestPhp($test) } else { try { $steps = $this->generateStepsPhp($test->getOrderedActions()); - } catch (TestReferenceException $e) { + } catch (\Exception $e) { throw new TestReferenceException($e->getMessage() . " in Test \"" . $test->getName() . "\""); } } @@ -1642,6 +1626,70 @@ private function addUniquenessToParamArray($input) return implode(", ", $result); } + /** + * Process pressKey parameterArray attribute for uniqueness function call and necessary data resolutions + * + * @param string $input + * @return string + */ + private function processPressKey($input) + { + // validate the param array is in the correct format + $input = trim($input); + $this->validateParameterArray($input); + // trim off the outer braces + $input = substr($input, 1, strlen($input) - 2); + + $result = []; + $arrayResult = []; + $count = 0; + + // matches all arrays and replaces them with placeholder to prevent later param manipulation + preg_match_all('/[\[][^\]]*?[\]]/', $input, $paramInput); + if (!empty($paramInput)) { + foreach ($paramInput[0] as $param) { + $arrayResult[self::PRESSKEY_ARRAY_ANCHOR_KEY . $count] = + '[' . trim($this->addUniquenessToParamArray($param)) . ']'; + $input = str_replace($param, self::PRESSKEY_ARRAY_ANCHOR_KEY . $count, $input); + $count++; + } + } + + $paramArray = explode(",", $input); + foreach ($paramArray as $param) { + // matches strings wrapped in ', we assume these are string literals + if (preg_match('/^[\s]*(\'.*?\')[\s]*$/', $param)) { + $result[] = trim($param); + continue; + } + + // matches \ for Facebook WebDriverKeys classes + if (substr(trim($param), 0, 1) === '\\') { + $result[] = trim($param); + continue; + } + + // matches numbers + if (preg_match('/^[\s]*(\d+?)[\s]*$/', $param)) { + $result[] = $param; + continue; + } + + $replacement = $this->addUniquenessFunctionCall(trim($param)); + + $result[] = $replacement; + } + + $result = implode(',', $result); + // reinsert arrays into result + if (!empty($arrayResult)) { + foreach ($arrayResult as $key => $value) { + $result = str_replace($key, $value, $result); + } + } + return $result; + } + /** * Add uniqueness function call to input string based on regex pattern. * diff --git a/src/Magento/FunctionalTestingFramework/Util/Validation/DuplicateNodeValidationUtil.php b/src/Magento/FunctionalTestingFramework/Util/Validation/DuplicateNodeValidationUtil.php index 8508f1abe..0c323a2dd 100644 --- a/src/Magento/FunctionalTestingFramework/Util/Validation/DuplicateNodeValidationUtil.php +++ b/src/Magento/FunctionalTestingFramework/Util/Validation/DuplicateNodeValidationUtil.php @@ -45,7 +45,7 @@ public function __construct($uniqueKey, $exceptionCollector) * @param string $filename * @return void */ - public function validateChildUniqueness(\DOMElement $parentNode, $filename) + public function validateChildUniqueness(\DOMElement $parentNode, $filename, $parentKey) { $childNodes = $parentNode->childNodes; $type = ucfirst($parentNode->tagName); @@ -69,7 +69,11 @@ public function validateChildUniqueness(\DOMElement $parentNode, $filename) $duplicates = array_diff_assoc($keyValues, $withoutDuplicates); $keyError = ""; foreach ($duplicates as $duplicateKey => $duplicateValue) { - $keyError .= "\t{$this->uniqueKey}: {$duplicateValue} is used more than once.\n"; + $keyError .= "\t{$this->uniqueKey}: {$duplicateValue} is used more than once."; + if ($parentKey !== null) { + $keyError .=" (Parent: {$parentKey})"; + } + $keyError .= "\n"; } $errorMsg = "{$type} cannot use {$this->uniqueKey}s more than once.\t\n{$keyError}\tin file: {$filename}"; diff --git a/src/Magento/FunctionalTestingFramework/_bootstrap.php b/src/Magento/FunctionalTestingFramework/_bootstrap.php index 2e769fb8f..34807d2b9 100644 --- a/src/Magento/FunctionalTestingFramework/_bootstrap.php +++ b/src/Magento/FunctionalTestingFramework/_bootstrap.php @@ -7,22 +7,17 @@ // define framework basepath for schema pathing defined('FW_BP') || define('FW_BP', realpath(__DIR__ . '/../../../')); - -// get the root path of the project (we will always be installed under vendor) +// get the root path of the project $projectRootPath = substr(FW_BP, 0, strpos(FW_BP, DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR)); - if (empty($projectRootPath)) { // If ProjectRootPath is empty, we are not under vendor and are executing standalone. - require_once (realpath(FW_BP . "/dev/tests/functional/_bootstrap.php")); + require_once (realpath(FW_BP . "/dev/tests/functional/standalone_bootstrap.php")); return; } +defined('PROJECT_ROOT') || define('PROJECT_ROOT', $projectRootPath); +$envFilepath = realpath($projectRootPath . '/dev/tests/acceptance/'); -// set Magento_BP as Root_Project Path -define('PROJECT_ROOT', $projectRootPath); -defined('MAGENTO_BP') || define('MAGENTO_BP', realpath($projectRootPath)); -// load .env (if it exists) -$envFilepath = realpath(MAGENTO_BP . '/dev/tests/acceptance/'); if (file_exists($envFilepath . DIRECTORY_SEPARATOR . '.env')) { $env = new \Dotenv\Loader($envFilepath . DIRECTORY_SEPARATOR . '.env'); $env->load(); @@ -37,6 +32,10 @@ defined($key) || define($key, $var); } + if (array_key_exists('MAGENTO_BP', $_ENV)) { + defined('TESTS_BP') || define('TESTS_BP', realpath(PROJECT_ROOT . DIRECTORY_SEPARATOR . 'dev/tests/acceptance')); + } + defined('MAGENTO_CLI_COMMAND_PATH') || define( 'MAGENTO_CLI_COMMAND_PATH', 'dev/tests/acceptance/utils/command.php' @@ -45,8 +44,19 @@ defined('MAGENTO_CLI_COMMAND_PARAMETER') || define('MAGENTO_CLI_COMMAND_PARAMETER', 'command'); $env->setEnvironmentVariable('MAGENTO_CLI_COMMAND_PARAMETER', MAGENTO_CLI_COMMAND_PARAMETER); + + defined('DEFAULT_TIMEZONE') || define('DEFAULT_TIMEZONE', 'America/Los_Angeles'); + $env->setEnvironmentVariable('DEFAULT_TIMEZONE', DEFAULT_TIMEZONE); + + try { + new DateTimeZone(DEFAULT_TIMEZONE); + } catch (\Exception $e) { + throw new \Exception("Invalid DEFAULT_TIMEZONE in .env: " . DEFAULT_TIMEZONE . PHP_EOL); + } } + +defined('MAGENTO_BP') || define('MAGENTO_BP', realpath(PROJECT_ROOT)); // TODO REMOVE THIS CODE ONCE WE HAVE STOPPED SUPPORTING dev/tests/acceptance PATH // define TEST_PATH and TEST_MODULE_PATH defined('TESTS_BP') || define('TESTS_BP', realpath(MAGENTO_BP . DIRECTORY_SEPARATOR . 'dev/tests/acceptance/'));