Skip to content

Merge master to develop #480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Oct 10, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
395c910
Lowercased links
dobooth Sep 19, 2019
6675ee5
Merge pull request #463 from magento-devdocs/master
KevinBKozan Sep 19, 2019
cd1bda5
MQE-1579: Outline test isolation best practices
KevinBKozan Sep 27, 2019
521475c
Fixing linting errors
dobooth Oct 1, 2019
090fb6e
MQE-1737: How to write good selectors
tomreece Oct 2, 2019
cca34df
MQE-1740: How to make use of MFTF Suite
soumyau Oct 1, 2019
dbcb122
MQE-1740: How to make use of MFTF Suite
soumyau Oct 1, 2019
be6b819
Editorial review. Grammar and formatting.
dobooth Oct 2, 2019
5d658dc
Finessing headers
dobooth Oct 2, 2019
44195cb
MQE-1737: How to write good selectors
tomreece Oct 3, 2019
ac6bad7
Merged content from selectors.md
dobooth Oct 3, 2019
76a0892
Merge pull request #473 from magento-devdocs/db_mftf_linting
KevinBKozan Oct 3, 2019
e8807e7
MQE-1736: Modularity in MFTF tests
KevinBKozan Oct 3, 2019
18abf17
MQE-1740: How to make use of MFTF Suite
soumyau Oct 3, 2019
bf7f5f3
Merge remote-tracking branch 'origin/MQE-1706' into MQE-1706
soumyau Oct 3, 2019
11a250d
MQE-1579: Outline test isolation best practices
KevinBKozan Oct 7, 2019
ece0f58
Formatting changes
dobooth Oct 7, 2019
5e5fb96
Formatting
dobooth Oct 7, 2019
b124a11
Formatting
dobooth Oct 7, 2019
c44d5d8
MQE-1579: Outline test isolation best practices
KevinBKozan Oct 7, 2019
fbf85a7
MQE-1736: Modularity in MFTF tests
KevinBKozan Oct 7, 2019
9f9a627
Formatting
dobooth Oct 7, 2019
ef8ae84
Merge pull request #475 from magento/MQE-1706
KevinBKozan Oct 8, 2019
e886f1b
MQE-1816: Fix missing MftfApplicationConfig import in GenerateSuiteCo…
tomreece Oct 10, 2019
541448a
MQE-1812: CHANGELOG.MD and Composer version bump
tomreece Oct 10, 2019
7744977
Merge pull request #478 from magento/2.5.1-rc
KevinBKozan Oct 10, 2019
53d8834
Merge branch 'develop' into merge-master-to-develop
tomreece Oct 10, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
Magento Functional Testing Framework Changelog
================================================

2.5.1
-----

### Fixes
* Fixed missing `use` statement in the generate:suite command

### GitHub Issues
* [#471](https://github.com/magento/magento2-functional-testing-framework/issues/471) -- PHP Fatal error: MftfApplicationConfig not found in GenerateSuiteCommand

2.5.0
-----
* Traceability
Expand Down
2 changes: 1 addition & 1 deletion bin/mftf
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ try {
try {
$application = new Symfony\Component\Console\Application();
$application->setName('Magento Functional Testing Framework CLI');
$application->setVersion('2.5.0');
$application->setVersion('2.5.1');
/** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */
$commandList = new \Magento\FunctionalTestingFramework\Console\CommandList;
foreach ($commandList->getCommands() as $command) {
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "magento/magento2-functional-testing-framework",
"description": "Magento2 Functional Testing Framework",
"type": "library",
"version": "2.5.0",
"version": "2.5.1",
"license": "AGPL-3.0",
"keywords": ["magento", "automation", "functional", "testing"],
"config": {
Expand Down
2 changes: 1 addition & 1 deletion composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 10 additions & 10 deletions docs/best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,19 +84,19 @@ Use the _Foo.camelCase_ naming convention, which is similar to _Classes_ and _cl

Use an upper case first letter for:

- File names. Example: _StorefrontCreateCustomerTest.xml_
- Test name attributes. Example: `<test name="TestAllTheThingsTest">`.
- Data entity names. Example: `<entity name="OutOfStockProduct">`.
- Page name. Example: `<page name="AdminLoginPage">`.
- Section name. Example: `<section name="AdminCategorySidebarActionSection">`.
- Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`.
- File names. Example: _StorefrontCreateCustomerTest.xml_
- Test name attributes. Example: `<test name="TestAllTheThingsTest">`.
- Data entity names. Example: `<entity name="OutOfStockProduct">`.
- Page name. Example: `<page name="AdminLoginPage">`.
- Section name. Example: `<section name="AdminCategorySidebarActionSection">`.
- Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`.

#### Lower case

Use a lower case first letter for:

- Data keys. Example: `<data key="firstName">`.
- Element names. Examples: `<element name="confirmDeleteButton"/>`.
- Data keys. Example: `<data key="firstName">`.
- Element names. Examples: `<element name="confirmDeleteButton"/>`.

## Page object

Expand Down Expand Up @@ -134,9 +134,9 @@ Define these three elements and reference them by name in the tests.
1. Keep your tests short and granular for target testing, easier reviews, and easier merge conflict resolution.
It also helps you to identify the cause of test failure.
1. Use comments to keep tests readable and maintainable:
- Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
- Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
It helps to inform the reader of what you are testing and to yield a more descriptive Allure report.
- Explain in comments unclear or tricky test steps.
- Explain in comments unclear or tricky test steps.
1. Refer to [sections] instead of writing selectors.

## Test step merging order
Expand Down
8 changes: 4 additions & 4 deletions docs/credentials.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@ Credentials can be used in actions: [`fillField`][], [`magentoCLI`][], and [`cre

Define the value as a reference to the corresponding key in the credentials file or vault such as `{{_CREDS.my_data_key}}`:

- `_CREDS` is an environment constant pointing to the `.credentials` file
- `my_data_key` is a key in the the `.credentials` file or vault that contains the value to be used in a test step
- `_CREDS` is an environment constant pointing to the `.credentials` file
- `my_data_key` is a key in the the `.credentials` file or vault that contains the value to be used in a test step

For example, reference secret data in the [`fillField`][] action with the `userInput` attribute.

Expand Down Expand Up @@ -180,5 +180,5 @@ The MFTF tests delivered with Magento application do not use credentials and do
[Download Vault]: https://www.hashicorp.com/products/vault/
[Login Vault]: https://www.vaultproject.io/docs/commands/login.html
[Vault KV2]: https://www.vaultproject.io/docs/secrets/kv/kv-v2.html
[`CREDENTIAL_VAULT_ADDRESS`]: configuration.md#CREDENTIAL_VAULT_ADDRESS
[`CREDENTIAL_VAULT_SECRET_BASE_PATH`]: configuration.md#CREDENTIAL_VAULT_SECRET_BASE_PATH
[`CREDENTIAL_VAULT_ADDRESS`]: configuration.md#credential_vault_address
[`CREDENTIAL_VAULT_SECRET_BASE_PATH`]: configuration.md#credential_vault_secret_base_path
40 changes: 20 additions & 20 deletions docs/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ userInput="{{SimpleSubCategory.name}}"

In this example:

* `SimpleSubCategory` is an entity name.
* `name` is a `<data>` key of the entity. The corresponding value will be assigned to `userInput` as a result.
* `SimpleSubCategory` is an entity name.
* `name` is a `<data>` key of the entity. The corresponding value will be assigned to `userInput` as a result.

### Environmental data

Expand All @@ -28,8 +28,8 @@ userInput="{{_ENV.MAGENTO_ADMIN_USERNAME}}"

In this example:

* `_ENV` is a reference to the `dev/tests/acceptance/.env` file, where basic environment variables are set.
* `MAGENTO_ADMIN_USERNAME` is a name of an environment variable.
* `_ENV` is a reference to the `dev/tests/acceptance/.env` file, where basic environment variables are set.
* `MAGENTO_ADMIN_USERNAME` is a name of an environment variable.
The corresponding value will be assigned to `userInput` as a result.

### Sensitive data
Expand All @@ -40,10 +40,10 @@ userInput="{{_CREDS.my_secret_token}}"

In this example:

* `_CREDS` is a constant to reference to the `dev/tests/acceptance/.credentials` file, where sensitive data and secrets are stored for use in a test.
* `MY_SECRET_TOKEN` is the name of a key in the credentials variable.
* `_CREDS` is a constant to reference to the `dev/tests/acceptance/.credentials` file, where sensitive data and secrets are stored for use in a test.
* `MY_SECRET_TOKEN` is the name of a key in the credentials variable.
The corresponding value of the credential will be assigned to `userInput` as a result.
* The decrypted values are only available in the `.credentials` file in which they are stored.
* The decrypted values are only available in the `.credentials` file in which they are stored.

Learn more in [Credentials][].

Expand All @@ -59,8 +59,8 @@ userInput="$createCustomer.email$"

In this example:

* `createCustomer` is a step key of the corresponding test step that creates an entity.
* `email` is a data key of the entity.
* `createCustomer` is a step key of the corresponding test step that creates an entity.
* `email` is a data key of the entity.
The corresponding value will be assigned to `userInput` as a result.

<div class="bs-callout bs-callout-info">
Expand Down Expand Up @@ -118,10 +118,10 @@ The format of `<data>` is:

The following conventions apply to MFTF `<data>`:

* A `<data>` file may contain multiple data entities.
* Camel case is used for `<data>` elements. The name represents the `<data>` type. For example, a file with customer data is `CustomerData.xml`. A file for simple product would be `SimpleProductData.xml`.
* Camel case is used for the entity name.
* The file name must have the suffix `Data.xml`.
* A `<data>` file may contain multiple data entities.
* Camel case is used for `<data>` elements. The name represents the `<data>` type. For example, a file with customer data is `CustomerData.xml`. A file for simple product would be `SimpleProductData.xml`.
* Camel case is used for the entity name.
* The file name must have the suffix `Data.xml`.

## Example

Expand Down Expand Up @@ -152,16 +152,16 @@ All entities that have the same name will be merged during test generation. Both

`_defaultCategory` sets three data fields:

* `name` defines the category name as `simpleCategory` with a unique suffix. Example: `simpleCategory598742365`.
* `name_lwr` defines the category name in lowercase format with a unique suffix. Example: `simplecategory697543215`.
* `is_active` sets the enable category to `true`.
* `name` defines the category name as `simpleCategory` with a unique suffix. Example: `simpleCategory598742365`.
* `name_lwr` defines the category name in lowercase format with a unique suffix. Example: `simplecategory697543215`.
* `is_active` sets the enable category to `true`.

`SimpleSubCategory` sets four data fields:

* `name` that defines the category name with a unique suffix. Example: `SimpleSubCategory458712365`.
* `name_lwr` that defines the category name in lowercase format with a unique suffix. Example: `simplesubcategory753698741`.
* `is_active` sets the enable category to `true`.
* `include_in_menu` that sets the include in the menu to `true`.
* `name` that defines the category name with a unique suffix. Example: `SimpleSubCategory458712365`.
* `name_lwr` that defines the category name in lowercase format with a unique suffix. Example: `simplesubcategory753698741`.
* `is_active` sets the enable category to `true`.
* `include_in_menu` that sets the include in the menu to `true`.

The following is an example of a call in test:

Expand Down
12 changes: 6 additions & 6 deletions docs/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

Debugging within the Magento Functional Testing Framework is helpful in identifying test bugs by allowing you to pause execution so that you may:

- Examine the page.
- Check returned data and other variables being used during run-time.
- Examine the page.
- Check returned data and other variables being used during run-time.

This is straightforward to do once you create a basic Debug Configuration.

## Prerequisites

- [Xdebug][]
- PHPUnit configured for use in [PHPStorm][]
- [Xdebug][]
- PHPUnit configured for use in [PHPStorm][]

## Creating Debug Configuration with PHPStorm

Expand All @@ -27,8 +27,8 @@ If you get a warning `Path to Codeception for local machine is not configured.`:

The easiest method of tagging a test for debugging is the following:

- In your Debug configuration, locate `Test Runner options:` and set `--group testDebug`.
- When you want to debug a test you are working on, simply add `<group value="testDebug"/>` to the annotations. Be sure to remove this after done debugging.
- In your Debug configuration, locate `Test Runner options:` and set `--group testDebug`.
- When you want to debug a test you are working on, simply add `<group value="testDebug"/>` to the annotations. Be sure to remove this after done debugging.

Your Debug Configuration should now be able to run your test and pause execution on any breakpoints you have set in the generated `.php` file under the `_generated` folder.

Expand Down
16 changes: 8 additions & 8 deletions docs/extending.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ For example, only one or two parameters (for example, URL) might vary between te
To avoid copy-pasting and to save some time the Magento Functional Testing Framework (MFTF) enables you to extend test components such as [test], [data], and [action group].
You can create or update any component of the parent body in your new test/action group/entity.

* A test starting with `<test name="SampleTest" extends="ParentTest">` creates a test `SampleTest` that takes body of existing test `ParentTest` and adds to it the body of `SampleTest`.
* An action group starting with `<actionGroup name="SampleActionGroup" extends="ParentActionGroup">` creates an action group based on the `ParentActionGroup`, but with the changes specified in `SampleActionGroup`.
* An entity starting with `<entity name="SampleEntity" extends="ParentEntity">` creates an entity `SampleEntity` that is equivalent to merging the `SampleEntity` with the `ParentEntity`.
* A test starting with `<test name="SampleTest" extends="ParentTest">` creates a test `SampleTest` that takes body of existing test `ParentTest` and adds to it the body of `SampleTest`.
* An action group starting with `<actionGroup name="SampleActionGroup" extends="ParentActionGroup">` creates an action group based on the `ParentActionGroup`, but with the changes specified in `SampleActionGroup`.
* An entity starting with `<entity name="SampleEntity" extends="ParentEntity">` creates an entity `SampleEntity` that is equivalent to merging the `SampleEntity` with the `ParentEntity`.

Specify needed variations for a parent object and produce a copy of the original that incorporates the specified changes (the "delta").

Expand Down Expand Up @@ -71,8 +71,8 @@ __Use case__: Create two similar tests with different `url` (`"{{AdminCategoryPa

__Use case__: Create two similar tests where the second test contains two additional steps:

* `checkOption` before `click` (`stepKey="clickLogin"`)
* `seeInCurrentUrl` after `click` in the `LogInAsAdminTest` test (in the `.../Backend/Test/LogInAsAdminTest.xml` file)
* `checkOption` before `click` (`stepKey="clickLogin"`)
* `seeInCurrentUrl` after `click` in the `LogInAsAdminTest` test (in the `.../Backend/Test/LogInAsAdminTest.xml` file)

> Tests with "extends":

Expand Down Expand Up @@ -119,8 +119,8 @@ __Use case__: Create two similar tests where the second test contains two additi

__Use case__: Create two similar tests where the second one contains two additional actions in the `before` hook:

* `checkOption` before `click` (`stepKey="clickLogin"`)
* `seeInCurrentUrl` after `click` in the `LogInAsAdminTest` test (in the `.../Backend/Test/LogInAsAdminTest.xml` file)
* `checkOption` before `click` (`stepKey="clickLogin"`)
* `seeInCurrentUrl` after `click` in the `LogInAsAdminTest` test (in the `.../Backend/Test/LogInAsAdminTest.xml` file)

> Tests with "extends":

Expand Down Expand Up @@ -366,4 +366,4 @@ __Use case__: Create an entity named `DivPanelGreen`, which is similar to the `D
[test]: ./test.md
[data]: ./data.md
[action group]: ./test/action-groups.md
[actions]: ./test/actions.md
[actions]: ./test/actions.md
20 changes: 10 additions & 10 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ The latest Magento 2.2 release supports MFTF 2.3.8.

Make sure that you have the following software installed and configured on your development environment:

- [PHP version supported by the Magento instance under test][php]
- [Composer 1.3 or later][composer]
- [Java 1.8 or later][java]
- [Selenium Server Standalone 3.1 or later][selenium server] and [ChromeDriver 2.33 or later][chrome driver] or other webdriver in the same directory
- [PHP version supported by the Magento instance under test][php]
- [Composer 1.3 or later][composer]
- [Java 1.8 or later][java]
- [Selenium Server Standalone 3.1 or later][selenium server] and [ChromeDriver 2.33 or later][chrome driver] or other webdriver in the same directory

<div class="bs-callout bs-callout-tip" markdown="1">
[PhpStorm] supports [Codeception test execution][], which is helpful when debugging.
Expand Down Expand Up @@ -147,16 +147,16 @@ vim dev/tests/acceptance/.env

Specify the following parameters, which are required to launch tests:

- `MAGENTO_BASE_URL` must contain a domain name of the Magento instance that will be tested.
- `MAGENTO_BASE_URL` must contain a domain name of the Magento instance that will be tested.
Example: `MAGENTO_BASE_URL=http://magento.test`

- `MAGENTO_BACKEND_NAME` must contain the relative path for the Admin area.
- `MAGENTO_BACKEND_NAME` must contain the relative path for the Admin area.
Example: `MAGENTO_BACKEND_NAME=admin`

- `MAGENTO_ADMIN_USERNAME` must contain the username required for authorization in the Admin area.
- `MAGENTO_ADMIN_USERNAME` must contain the username required for authorization in the Admin area.
Example: `MAGENTO_ADMIN_USERNAME=admin`

- `MAGENTO_ADMIN_PASSWORD` must contain the user password required for authorization in the Admin area.
- `MAGENTO_ADMIN_PASSWORD` must contain the user password required for authorization in the Admin area.
Example: `MAGENTO_ADMIN_PASSWORD=123123q`

<div class="bs-callout bs-callout-info" markdown="1">
Expand Down Expand Up @@ -222,8 +222,8 @@ During testing, the MFTF generates test reports in CLI.
You can generate visual representations of the report data using [Allure Framework][].
To view the reports in GUI:

- [Install Allure][]
- Run the tool to serve the artifacts in `dev/tests/acceptance/tests/_output/allure-results/`:
- [Install Allure][]
- Run the tool to serve the artifacts in `dev/tests/acceptance/tests/_output/allure-results/`:

```bash
allure serve dev/tests/acceptance/tests/_output/allure-results/
Expand Down
Loading