Skip to content

MQE-810: Create a static test to validate references between modules #341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 25, 2019
Merged

MQE-810: Create a static test to validate references between modules #341

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
48a4a56
MQE-810: Create a static test to validate references between modules
KevinBKozan Apr 24, 2019
14face7
Formatting and ordering.
dobooth Apr 24, 2019
f4f2953
Merge branch 'develop' into MQE-810-DOCS
KevinBKozan Apr 24, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 33 additions & 19 deletions docs/commands/mftf.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ This command cleans up the previously generated tests; generates and runs the te
For more details about `failed`, refer to [Reporting][].

### Generate documentation for action groups

```bash
vendor/bin/mftf generate:docs
```
Expand Down Expand Up @@ -115,14 +116,14 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]

#### Options

| Option | Description |
| --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--config=[<default>|<singleRun>|<parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.</br> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.</br> Example: `generate:tests --config=parallel`. |
| `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
| `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15` |
| `--tests` | Defines the test configuration as a JSON string. |
| `--debug` | Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This parameter takes extra processing time. Use it after test generation has failed once. |
| `-r,--remove` | Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only. |
| Option | Description|
| ---| --- |
| `--config=[<default> or <singleRun> or <parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.</br> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.</br> Example: `generate:tests --config=parallel`. |
| `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
| `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
| `--tests` | Defines the test configuration as a JSON string.|
| `--debug` | Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This parameter takes extra processing time. Use it after test generation has failed once. |
| `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|

#### Examples of the JSON configuration

Expand All @@ -131,7 +132,7 @@ The configuration to generate a single test with no suites:
```json
{
"tests":[
"general_test1" //Generate the "general_test1" test.
"general_test1" //Generate the "general_test1" test.
],
"suites": null
}
Expand All @@ -141,10 +142,10 @@ The configuration to generate a single test in the suite:

```json
{
"tests": null, // No tests outside the suite configuration will be generated.
"tests": null, // No tests outside the suite configuration will be generated.
"suites":{
"sample":[ // The suite that contains the test.
"suite_test1" // The test to be generated.
"sample":[ // The suite that contains the test.
"suite_test1" // The test to be generated.
]
}
}
Expand All @@ -159,11 +160,11 @@ Complex configuration to generate a few non-suite tests, a single test in a suit
"general_test2",
"general_test3"
],
"suites":{ //Go to suites.
"sample":[ //Go to the "sample" suite.
"suite_test1" //Generate the "suite_test1" test.
"suites":{ //Go to suites.
"sample":[ //Go to the "sample" suite.
"suite_test1" //Generate the "suite_test1" test.
],
"sample2":[] //Generate all tests in the "sample2" suite.
"sample2":[] //Generate all tests in the "sample2" suite.
}
}
```
Expand All @@ -190,8 +191,8 @@ vendor/bin/mftf generate:suite <suite name> [<suite name>] [--remove]

#### Options

| Option | Description |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Option | Description |
| --- | --- |
| `-r,--remove` | Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `vendor/bin/mftf generate:suite WYSIWYG --remove` cleans up the entire `_generated` directory and generates `WYSIWYG` only. |

#### Example
Expand Down Expand Up @@ -253,7 +254,6 @@ vendor/bin/mftf generate:docs [--clean] [--output=/path/to/alternate/dir]
vendor/bin/mftf generate:docs --clean
```


### `reset`

#### Description
Expand Down Expand Up @@ -389,6 +389,20 @@ vendor/bin/mftf setup:env

The example parameters are taken from the `etc/config/.env.example` file.

### `static:checks`

Runs all MFTF static:checks on the test codebase that MFTF is currently attached to.

#### Existing static checks

* Test Dependency: Checks that test dependencies do not violate Magento module's composer dependencies.

#### Usage

```bash
vendor/bin/mftf static:checks
```

### `upgrade:tests`

Applies all the MFTF major version upgrade scripts to test components in the given path (`test.xml`, `data.xml`, etc).
Expand Down