From 6c9c34d4f00931860a89e49ff859842f03913e20 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Mon, 1 Apr 2019 11:22:28 -0500
Subject: [PATCH 01/63] Updating contribution guides for docs.

---
 .github/CONTRIBUTING.md           |  4 +-
 .github/DOCUMENTATION_TEMPLATE.md | 64 +++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 2 deletions(-)
 create mode 100644 .github/DOCUMENTATION_TEMPLATE.md

diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
index 61805aafe..e5e0cdf3f 100644
--- a/.github/CONTRIBUTING.md
+++ b/.github/CONTRIBUTING.md
@@ -40,7 +40,7 @@ All contributors are required to submit a click-through form to agree to the ter
 
    - Unit/integration test coverage
    - Proposed documentation update.
-     For the documentation contribution guidelines, see [DevDocs Contributing][devdocs].
+     For the documentation contribution guidelines, see [DOCUMENTATION_TEMPLATE][].
 7. For large features or changes, [open an issue][issue] to discuss first.
    This may prevent duplicate or unnecessary effort, and it may gain you some additional contributors.
 8. To report a bug, [open an issue][issue], and follow [guidelines about bugfix issues][issue reporting].
@@ -160,7 +160,7 @@ Label| Description
 [create issue]: https://help.github.com/articles/creating-an-issue/
 [create pr]: https://help.github.com/articles/creating-a-pull-request/
 [Definition of Done]: https://devdocs.magento.com/guides/v2.2/contributor-guide/contributing_dod.html
-[devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.html
+[DOCUMENTATION_TEMPLATE]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.html
 [existing issues]: https://github.com/magento/magento2-functional-testing-framework/issues?q=is%3Aopen+is%3Aissue
 [existing PRs]: https://github.com/magento/magento2-functional-testing-framework/pulls?q=is%3Aopen+is%3Apr
 [GitHub documentation]: https://help.github.com/articles/syncing-a-fork
diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
new file mode 100644
index 000000000..db01a3b3c
--- /dev/null
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -0,0 +1,64 @@
+# How to conttribute to MFTF docs
+
+MFTF documentation is kept within the /docs/ folder in this repo.
+We welcome contributions to the documentation.
+This page describes the process for submitting docs and serves as a template so that docs are written properly.
+
+The contribution workflow for docs is the same as submitting code.
+To contribute to MFTF docs:
+
+1. Make a branch from the [MFTF repo][].
+1. Make edits/additions/deletions as needed.
+1. Submit your PR to the `develop` branch.
+
+Once submitted, it will be reviewed by a member of the documentation team.
+If approved it will be tested and merged.
+If it needs any work, we will inform you.
+
+Any changes to the Table of Contents will need to be made in the regular [devdocs repo][].
+
+## H2 heading - blank line before and after, capitalize first word only
+
+1. Ordered lists all all numbered 1.
+1. The build process will number them correctly.
+1. Single spafce after the number. Blank line before and after list.
+
+### H3
+
+- Unordered lists use dashes.
+- One space after the dash.
+- Blank line before and after list.
+
+## Code Samples
+
+For formatting and code coloring, wrap code samples in the following format.
+Replace the `xml` with the file extension of the code sample type. Use `bash` for command-line text.
+
+```xml
+<xmlSample>
+   ...
+   ...
+</xmlSample>
+```
+
+## Markdown tables
+
+| Header      | Header |
+| ----------- | ----------- |
+| Colume 1 text | Column 2 text|
+| Column 1 text | Column 2 text|
+
+Markdown tables work for simple tables. If you need lists or other complex features within a cell, you may have to use a HTML table.
+
+## Other tips
+
+- Use spaces instead of tabs.
+- One empty line between content. No duplicate empty lines.
+- Read more about how to [Contribute to Magento Devdocs][]
+
+<!-- For readability, we abstract the link URLS to the bottom of the page. The extra set of square brackets denotes it is a link, rather than plain brackets. >
+
+<!-- Link Definitions -->
+[devdocs repo]: https://github.com/magento/devdocs
+[MFTF repo]: https://github.com/magento/magento2-functional-testing-framework
+[Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
\ No newline at end of file

From 0c43a9e9fa7988c4241656c79cc2cb4cc0209517 Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:06:32 -0500
Subject: [PATCH 02/63] Update .github/CONTRIBUTING.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/CONTRIBUTING.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
index e5e0cdf3f..24dc3b467 100644
--- a/.github/CONTRIBUTING.md
+++ b/.github/CONTRIBUTING.md
@@ -160,7 +160,7 @@ Label| Description
 [create issue]: https://help.github.com/articles/creating-an-issue/
 [create pr]: https://help.github.com/articles/creating-a-pull-request/
 [Definition of Done]: https://devdocs.magento.com/guides/v2.2/contributor-guide/contributing_dod.html
-[DOCUMENTATION_TEMPLATE]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.html
+[DOCUMENTATION_TEMPLATE]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
 [existing issues]: https://github.com/magento/magento2-functional-testing-framework/issues?q=is%3Aopen+is%3Aissue
 [existing PRs]: https://github.com/magento/magento2-functional-testing-framework/pulls?q=is%3Aopen+is%3Apr
 [GitHub documentation]: https://help.github.com/articles/syncing-a-fork
@@ -171,4 +171,4 @@ Label| Description
 [Magento Contributor Agreement]: http://www.magento.com/legaldocuments/mca
 [MFTF repository]: https://github.com/magento/magento2-functional-testing-framework
 [open new issue]: https://github.com/magento/magento2-functional-testing-framework/issues/new
-[Travis CI]: https://travis-ci.com/magento/magento2-functional-testing-framework/pull_requests
\ No newline at end of file
+[Travis CI]: https://travis-ci.com/magento/magento2-functional-testing-framework/pull_requests

From 5569b0a587f2f2d46739d79caef15d28b151b574 Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:07:21 -0500
Subject: [PATCH 03/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index db01a3b3c..61e06e213 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -1,6 +1,6 @@
 # How to conttribute to MFTF docs
 
-MFTF documentation is kept within the /docs/ folder in this repo.
+MFTF documentation is kept within the /docs/ folder in this repository.
 We welcome contributions to the documentation.
 This page describes the process for submitting docs and serves as a template so that docs are written properly.
 
@@ -61,4 +61,4 @@ Markdown tables work for simple tables. If you need lists or other complex featu
 <!-- Link Definitions -->
 [devdocs repo]: https://github.com/magento/devdocs
 [MFTF repo]: https://github.com/magento/magento2-functional-testing-framework
-[Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
\ No newline at end of file
+[Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md

From 2cc566698ebd3ad88cd93b699acc0237f98ac17f Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:07:55 -0500
Subject: [PATCH 04/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 61e06e213..a678aaf88 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -2,7 +2,7 @@
 
 MFTF documentation is kept within the /docs/ folder in this repository.
 We welcome contributions to the documentation.
-This page describes the process for submitting docs and serves as a template so that docs are written properly.
+This page describes the process for submitting documentation and serves as a template for a properly written content.
 
 The contribution workflow for docs is the same as submitting code.
 To contribute to MFTF docs:

From 2290e2ecff1e76ae2a7390f1f22f32ffa4789ec4 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 2 Apr 2019 12:25:14 -0500
Subject: [PATCH 05/63] Fixes per review.

---
 .github/DOCUMENTATION_TEMPLATE.md | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index a678aaf88..1171901ac 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -4,35 +4,36 @@ MFTF documentation is kept within the /docs/ folder in this repository.
 We welcome contributions to the documentation.
 This page describes the process for submitting documentation and serves as a template for a properly written content.
 
-The contribution workflow for docs is the same as submitting code.
+The contribution workflow for the Magento functional testing framework (MFTF) documentation is the same as submitting code.
 To contribute to MFTF docs:
 
-1. Make a branch from the [MFTF repo][].
+1. Make a branch from the `develop` branch in the [MFTF repo][].
 1. Make edits/additions/deletions as needed.
 1. Submit your PR to the `develop` branch.
 
-Once submitted, it will be reviewed by a member of the documentation team.
+Once submitted, a member of the documentation team will review and merge it.
 If approved it will be tested and merged.
 If it needs any work, we will inform you.
 
 Any changes to the Table of Contents will need to be made in the regular [devdocs repo][].
 
-## H2 heading - blank line before and after, capitalize first word only
+## H2 Heading - blank line before and after, capitalize first word only
 
-1. Ordered lists all all numbered 1.
+1. Number all ordered list items as `1.`
 1. The build process will number them correctly.
-1. Single spafce after the number. Blank line before and after list.
+1. Single spafce after the number.
+1. Blank line before and after list.
 
-### H3
+## Unordered lists
 
-- Unordered lists use dashes.
-- One space after the dash.
-- Blank line before and after list.
+- Use dashes in unordered lists.
+- Add one space after the dash.
+- Add a blank line before and after list.
 
 ## Code Samples
 
-For formatting and code coloring, wrap code samples in the following format.
-Replace the `xml` with the file extension of the code sample type. Use `bash` for command-line text.
+For formatting and code coloring, wrap code samples in the following format:
+Replace the `xml` with the corresponding language (tupe) of the code sample. Use `bash` for shell commands and `terminal` for terminal output.
 
 ```xml
 <xmlSample>
@@ -48,12 +49,12 @@ Replace the `xml` with the file extension of the code sample type. Use `bash` fo
 | Colume 1 text | Column 2 text|
 | Column 1 text | Column 2 text|
 
-Markdown tables work for simple tables. If you need lists or other complex features within a cell, you may have to use a HTML table.
+Markdown formatting works for simple tables. It does not support multiline content within a cell, or split/merged cells within a row or column.
 
 ## Other tips
 
 - Use spaces instead of tabs.
-- One empty line between content. No duplicate empty lines.
+- Do not duplicate blank lines.
 - Read more about how to [Contribute to Magento Devdocs][]
 
 <!-- For readability, we abstract the link URLS to the bottom of the page. The extra set of square brackets denotes it is a link, rather than plain brackets. >

From a6786ee5fd8f83837a789b5bd4a15fbe3387cfb2 Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:26:22 -0500
Subject: [PATCH 06/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 1171901ac..d7e9d19ab 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -5,7 +5,6 @@ We welcome contributions to the documentation.
 This page describes the process for submitting documentation and serves as a template for a properly written content.
 
 The contribution workflow for the Magento functional testing framework (MFTF) documentation is the same as submitting code.
-To contribute to MFTF docs:
 
 1. Make a branch from the `develop` branch in the [MFTF repo][].
 1. Make edits/additions/deletions as needed.

From a0faa1d21d99651b95c2192bc3c84bf87e49906a Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:26:43 -0500
Subject: [PATCH 07/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index d7e9d19ab..c5668ce2f 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -29,7 +29,7 @@ Any changes to the Table of Contents will need to be made in the regular [devdoc
 - Add one space after the dash.
 - Add a blank line before and after list.
 
-## Code Samples
+## Code samples
 
 For formatting and code coloring, wrap code samples in the following format:
 Replace the `xml` with the corresponding language (tupe) of the code sample. Use `bash` for shell commands and `terminal` for terminal output.

From 5f27f405efd73ee35021b72e358c815420082cf7 Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:27:59 -0500
Subject: [PATCH 08/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index c5668ce2f..d99a64e21 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -11,7 +11,6 @@ The contribution workflow for the Magento functional testing framework (MFTF) do
 1. Submit your PR to the `develop` branch.
 
 Once submitted, a member of the documentation team will review and merge it.
-If approved it will be tested and merged.
 If it needs any work, we will inform you.
 
 Any changes to the Table of Contents will need to be made in the regular [devdocs repo][].

From acc9238b15f4e11a7650eb5a11cb3c0d2d0ca84a Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:28:11 -0500
Subject: [PATCH 09/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index d99a64e21..776040a14 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -11,7 +11,7 @@ The contribution workflow for the Magento functional testing framework (MFTF) do
 1. Submit your PR to the `develop` branch.
 
 Once submitted, a member of the documentation team will review and merge it.
-If it needs any work, we will inform you.
+We will inform you if it needs any additional processing.
 
 Any changes to the Table of Contents will need to be made in the regular [devdocs repo][].
 

From 978d945feefcad7059f88830318aa9b46c0b4a06 Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:29:10 -0500
Subject: [PATCH 10/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 776040a14..933558bab 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -17,6 +17,8 @@ Any changes to the Table of Contents will need to be made in the regular [devdoc
 
 ## H2 Heading - blank line before and after, capitalize first word only
 
+## Ordered lists
+
 1. Number all ordered list items as `1.`
 1. The build process will number them correctly.
 1. Single spafce after the number.

From 6182ec9e0ae3dee88ab332c6573829733749dda3 Mon Sep 17 00:00:00 2001
From: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
Date: Tue, 2 Apr 2019 12:29:27 -0500
Subject: [PATCH 11/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: dobooth <dobooth@adobe.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 933558bab..096049841 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -20,7 +20,6 @@ Any changes to the Table of Contents will need to be made in the regular [devdoc
 ## Ordered lists
 
 1. Number all ordered list items as `1.`
-1. The build process will number them correctly.
 1. Single spafce after the number.
 1. Blank line before and after list.
 

From 447ed0f927bf015c25685bd0c256c30ba220d526 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 2 Apr 2019 12:37:58 -0500
Subject: [PATCH 12/63] More review fixes.

---
 .github/DOCUMENTATION_TEMPLATE.md | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 096049841..f4f904d13 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -31,8 +31,7 @@ Any changes to the Table of Contents will need to be made in the regular [devdoc
 
 ## Code samples
 
-For formatting and code coloring, wrap code samples in the following format:
-Replace the `xml` with the corresponding language (tupe) of the code sample. Use `bash` for shell commands and `terminal` for terminal output.
+For code highlighting, wrap code samples in the following format:
 
 ```xml
 <xmlSample>
@@ -41,6 +40,8 @@ Replace the `xml` with the corresponding language (tupe) of the code sample. Use
 </xmlSample>
 ```
 
+Replace the `xml` with the corresponding language (tupe) of the code sample. Use `bash` for shell commands and `terminal` for terminal output.
+
 ## Markdown tables
 
 | Header      | Header |

From 2099d30d910da89ab318a6f0b3f3be85f0a884e3 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 2 Apr 2019 14:31:38 -0500
Subject: [PATCH 13/63] Fixed link.

---
 .github/CONTRIBUTING.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
index 24dc3b467..271b66244 100644
--- a/.github/CONTRIBUTING.md
+++ b/.github/CONTRIBUTING.md
@@ -160,7 +160,7 @@ Label| Description
 [create issue]: https://help.github.com/articles/creating-an-issue/
 [create pr]: https://help.github.com/articles/creating-a-pull-request/
 [Definition of Done]: https://devdocs.magento.com/guides/v2.2/contributor-guide/contributing_dod.html
-[DOCUMENTATION_TEMPLATE]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
+[DOCUMENTATION_TEMPLATE]: https://github.com/magento/devdocs/blob/master/.github/DOCUMENTATION_TEMPLATE.md
 [existing issues]: https://github.com/magento/magento2-functional-testing-framework/issues?q=is%3Aopen+is%3Aissue
 [existing PRs]: https://github.com/magento/magento2-functional-testing-framework/pulls?q=is%3Aopen+is%3Apr
 [GitHub documentation]: https://help.github.com/articles/syncing-a-fork

From 9c68e7dc3fabc91b7810973f52b3c66358300fd3 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 2 Apr 2019 16:13:05 -0500
Subject: [PATCH 14/63] Fixed header

---
 .github/DOCUMENTATION_TEMPLATE.md | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index f4f904d13..427f0590d 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -15,9 +15,7 @@ We will inform you if it needs any additional processing.
 
 Any changes to the Table of Contents will need to be made in the regular [devdocs repo][].
 
-## H2 Heading - blank line before and after, capitalize first word only
-
-## Ordered lists
+## Ordered lists - all heading have a blank line before and after, capitalize the first word only
 
 1. Number all ordered list items as `1.`
 1. Single spafce after the number.

From 95016141d3f102642001b992063989460ec15afa Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Fri, 5 Apr 2019 15:17:44 -0500
Subject: [PATCH 15/63] Fixes per review.

---
 .github/DOCUMENTATION_TEMPLATE.md | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 427f0590d..96f8c6bf9 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -1,4 +1,4 @@
-# How to conttribute to MFTF docs
+# How to contribute to MFTF docs
 
 MFTF documentation is kept within the /docs/ folder in this repository.
 We welcome contributions to the documentation.
@@ -6,19 +6,20 @@ This page describes the process for submitting documentation and serves as a tem
 
 The contribution workflow for the Magento functional testing framework (MFTF) documentation is the same as submitting code.
 
-1. Make a branch from the `develop` branch in the [MFTF repo][].
+1. Create a branch from the `develop` branch in the [MFTF repo][].
 1. Make edits/additions/deletions as needed.
 1. Submit your PR to the `develop` branch.
 
 Once submitted, a member of the documentation team will review and merge it.
 We will inform you if it needs any additional processing.
 
-Any changes to the Table of Contents will need to be made in the regular [devdocs repo][].
+The documentation in this repository is used as the source for the [MFTF documentation][].
+Any changes to the Table of Contents will need to be made via a separate PR in the regular [devdocs repo][].
 
 ## Ordered lists - all heading have a blank line before and after, capitalize the first word only
 
 1. Number all ordered list items as `1.`
-1. Single spafce after the number.
+1. Single space after the number.
 1. Blank line before and after list.
 
 ## Unordered lists
@@ -29,7 +30,8 @@ Any changes to the Table of Contents will need to be made in the regular [devdoc
 
 ## Code samples
 
-For code highlighting, wrap code samples in the following format:
+For technical words such as a database table name, wrap with single backticks: `tablename`.
+For code block highlighting, wrap code samples in the following format:
 
 ```xml
 <xmlSample>
@@ -44,7 +46,7 @@ Replace the `xml` with the corresponding language (tupe) of the code sample. Use
 
 | Header      | Header |
 | ----------- | ----------- |
-| Colume 1 text | Column 2 text|
+| Column 1 text | Column 2 text|
 | Column 1 text | Column 2 text|
 
 Markdown formatting works for simple tables. It does not support multiline content within a cell, or split/merged cells within a row or column.
@@ -58,6 +60,7 @@ Markdown formatting works for simple tables. It does not support multiline conte
 <!-- For readability, we abstract the link URLS to the bottom of the page. The extra set of square brackets denotes it is a link, rather than plain brackets. >
 
 <!-- Link Definitions -->
-[devdocs repo]: https://github.com/magento/devdocs
+[devdocs repo]: https://github.com/magento/devdocs/blob/master/_data/toc/mftf.yml
 [MFTF repo]: https://github.com/magento/magento2-functional-testing-framework
 [Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
+[MFTF documentation]: https://devdocs.magento.com/mftf/docs/introduction.html

From d0fd11b3dc337adb35e3a3f6db1a70f16a498fdf Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Mon, 15 Apr 2019 13:36:16 -0500
Subject: [PATCH 16/63] Cleaned up header

---
 .github/DOCUMENTATION_TEMPLATE.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 96f8c6bf9..ea5910c91 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -16,7 +16,7 @@ We will inform you if it needs any additional processing.
 The documentation in this repository is used as the source for the [MFTF documentation][].
 Any changes to the Table of Contents will need to be made via a separate PR in the regular [devdocs repo][].
 
-## Ordered lists - all heading have a blank line before and after, capitalize the first word only
+## Ordered lists
 
 1. Number all ordered list items as `1.`
 1. Single space after the number.

From 0b2373413c691efd83750c0cd7dda139bc036e85 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 16 Apr 2019 09:41:50 -0500
Subject: [PATCH 17/63] Grammar fix.

---
 .github/DOCUMENTATION_TEMPLATE.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index ea5910c91..0afecad9e 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -6,7 +6,7 @@ This page describes the process for submitting documentation and serves as a tem
 
 The contribution workflow for the Magento functional testing framework (MFTF) documentation is the same as submitting code.
 
-1. Create a branch from the `develop` branch in the [MFTF repo][].
+1. Create a branch from the `develop` branch in the [MFTF repository][].
 1. Make edits/additions/deletions as needed.
 1. Submit your PR to the `develop` branch.
 
@@ -14,7 +14,7 @@ Once submitted, a member of the documentation team will review and merge it.
 We will inform you if it needs any additional processing.
 
 The documentation in this repository is used as the source for the [MFTF documentation][].
-Any changes to the Table of Contents will need to be made via a separate PR in the regular [devdocs repo][].
+Any changes to the Table of Contents will need to be made via a separate PR in the regular [Magento Developer documentation repository][].
 
 ## Ordered lists
 
@@ -60,7 +60,7 @@ Markdown formatting works for simple tables. It does not support multiline conte
 <!-- For readability, we abstract the link URLS to the bottom of the page. The extra set of square brackets denotes it is a link, rather than plain brackets. >
 
 <!-- Link Definitions -->
-[devdocs repo]: https://github.com/magento/devdocs/blob/master/_data/toc/mftf.yml
-[MFTF repo]: https://github.com/magento/magento2-functional-testing-framework
+[Magento Developer documentation repository]: https://github.com/magento/devdocs/blob/master/_data/toc/mftf.yml
+[MFTF repository]: https://github.com/magento/magento2-functional-testing-framework
 [Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
 [MFTF documentation]: https://devdocs.magento.com/mftf/docs/introduction.html

From 70e59a520d1b774536a8340d0aed93d43792a064 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Mon, 22 Apr 2019 09:16:26 -0500
Subject: [PATCH 18/63] Fixed grammar.

---
 .github/DOCUMENTATION_TEMPLATE.md | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 0afecad9e..fb39105aa 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -1,10 +1,8 @@
 # How to contribute to MFTF docs
 
-MFTF documentation is kept within the /docs/ folder in this repository.
-We welcome contributions to the documentation.
-This page describes the process for submitting documentation and serves as a template for a properly written content.
-
-The contribution workflow for the Magento functional testing framework (MFTF) documentation is the same as submitting code.
+We welcome contributions to the MFTF documentation, which is kept within the /docs/ folder in this repository.
+This page describes the submitting process and serves as a template for a properly written content.
+The contribution workflow is the same as submitting code.
 
 1. Create a branch from the `develop` branch in the [MFTF repository][].
 1. Make edits/additions/deletions as needed.
@@ -14,7 +12,7 @@ Once submitted, a member of the documentation team will review and merge it.
 We will inform you if it needs any additional processing.
 
 The documentation in this repository is used as the source for the [MFTF documentation][].
-Any changes to the Table of Contents will need to be made via a separate PR in the regular [Magento Developer documentation repository][].
+Any changes to the table of contents will need to be made through a separate pull request in the regular [Magento Developer documentation repository][].
 
 ## Ordered lists
 

From 2eef763920df78d96ee25000ecadb9f20359fd72 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Mon, 29 Apr 2019 15:51:28 -0700
Subject: [PATCH 19/63] Indenting so markdown syntax shows.

---
 .github/DOCUMENTATION_TEMPLATE.md | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index fb39105aa..8a5a7d65e 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -31,12 +31,13 @@ Any changes to the table of contents will need to be made through a separate pul
 For technical words such as a database table name, wrap with single backticks: `tablename`.
 For code block highlighting, wrap code samples in the following format:
 
-```xml
-<xmlSample>
-   ...
-   ...
-</xmlSample>
-```
+
+    ```xml
+    <xmlSample>
+     ...
+     ...
+    </xmlSample>
+    ```
 
 Replace the `xml` with the corresponding language (tupe) of the code sample. Use `bash` for shell commands and `terminal` for terminal output.
 

From 42bc8d160744019857edbcf3b354dde9c8b32970 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Mon, 29 Apr 2019 15:51:57 -0700
Subject: [PATCH 20/63] Indenting so markdown syntax shows.


From 51d66ebdf39ae9e9bcc5f09f1155143747350f0d Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Mon, 29 Apr 2019 15:58:47 -0700
Subject: [PATCH 21/63] Linked to Basic Template.

---
 .github/DOCUMENTATION_TEMPLATE.md | 45 +++----------------------------
 1 file changed, 3 insertions(+), 42 deletions(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index 8a5a7d65e..f7e6a8576 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -5,7 +5,7 @@ This page describes the submitting process and serves as a template for a proper
 The contribution workflow is the same as submitting code.
 
 1. Create a branch from the `develop` branch in the [MFTF repository][].
-1. Make edits/additions/deletions as needed.
+1. Make edits/additions/deletions as needed. Read the [Basic Template][] for tips on how to write with Markdown.
 1. Submit your PR to the `develop` branch.
 
 Once submitted, a member of the documentation team will review and merge it.
@@ -14,47 +14,7 @@ We will inform you if it needs any additional processing.
 The documentation in this repository is used as the source for the [MFTF documentation][].
 Any changes to the table of contents will need to be made through a separate pull request in the regular [Magento Developer documentation repository][].
 
-## Ordered lists
-
-1. Number all ordered list items as `1.`
-1. Single space after the number.
-1. Blank line before and after list.
-
-## Unordered lists
-
-- Use dashes in unordered lists.
-- Add one space after the dash.
-- Add a blank line before and after list.
-
-## Code samples
-
-For technical words such as a database table name, wrap with single backticks: `tablename`.
-For code block highlighting, wrap code samples in the following format:
-
-
-    ```xml
-    <xmlSample>
-     ...
-     ...
-    </xmlSample>
-    ```
-
-Replace the `xml` with the corresponding language (tupe) of the code sample. Use `bash` for shell commands and `terminal` for terminal output.
-
-## Markdown tables
-
-| Header      | Header |
-| ----------- | ----------- |
-| Column 1 text | Column 2 text|
-| Column 1 text | Column 2 text|
-
-Markdown formatting works for simple tables. It does not support multiline content within a cell, or split/merged cells within a row or column.
-
-## Other tips
-
-- Use spaces instead of tabs.
-- Do not duplicate blank lines.
-- Read more about how to [Contribute to Magento Devdocs][]
+Read more about how to [Contribute to Magento Devdocs][].
 
 <!-- For readability, we abstract the link URLS to the bottom of the page. The extra set of square brackets denotes it is a link, rather than plain brackets. >
 
@@ -63,3 +23,4 @@ Markdown formatting works for simple tables. It does not support multiline conte
 [MFTF repository]: https://github.com/magento/magento2-functional-testing-framework
 [Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
 [MFTF documentation]: https://devdocs.magento.com/mftf/docs/introduction.html
+[Basic Template]: https://devdocs.magento.com/guides/v2.3/contributor-guide/templates/basic_template.html

From 53f6baec2d5e6881eb09d3a001246f5db0796f69 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 7 May 2019 16:43:54 -0500
Subject: [PATCH 22/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 dev/tests/_bootstrap.php                      |  1 +
 .../Config/MftfApplicationConfig.php          | 36 +++++++++++++++----
 .../Config/Reader/Filesystem.php              |  6 ++++
 .../Config/Reader/MftfFilesystem.php          |  3 ++
 .../Console/GenerateDocsCommand.php           |  1 +
 .../Console/GenerateTestsCommand.php          | 19 ++++++----
 .../Console/RunTestFailedCommand.php          |  1 +
 .../Console/RunTestGroupCommand.php           |  1 +
 .../StaticCheck/TestDependencyCheck.php       |  1 +
 .../Util/Logger/MftfLogger.php                | 19 +++++++++-
 10 files changed, 74 insertions(+), 14 deletions(-)

diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php
index 31b4462f6..41559184a 100644
--- a/dev/tests/_bootstrap.php
+++ b/dev/tests/_bootstrap.php
@@ -32,6 +32,7 @@
     true,
     \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::UNIT_TEST_PHASE,
     true,
+    false,
     false
 );
 
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index eeabdf117..54930df3f 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -36,12 +36,19 @@ class MftfApplicationConfig
     private $verboseEnabled;
 
     /**
-     * Determines whether the user would like to execute mftf in a verbose run.
+     * Determines whether the user would like to execute mftf in a 'per file' debug mode
      *
      * @var boolean
      */
     private $debugEnabled;
 
+    /**
+     * Determines whether the user would like to execute mftf in a 'merged file' debug mode
+     *
+     * @var boolean
+     */
+    private $fastDebugEnabled;
+
     /**
      * MftfApplicationConfig Singelton Instance
      *
@@ -53,16 +60,18 @@ class MftfApplicationConfig
      * MftfApplicationConfig constructor.
      *
      * @param boolean $forceGenerate
-     * @param string  $phase
+     * @param string $phase
      * @param boolean $verboseEnabled
      * @param boolean $debugEnabled
+     * @param null $fastDebugEnabled
      * @throws TestFrameworkException
      */
     private function __construct(
         $forceGenerate = false,
         $phase = self::EXECUTION_PHASE,
         $verboseEnabled = null,
-        $debugEnabled = null
+        $debugEnabled = null,
+        $fastDebugEnabled = null
     ) {
         $this->forceGenerate = $forceGenerate;
 
@@ -73,6 +82,7 @@ private function __construct(
         $this->phase = $phase;
         $this->verboseEnabled = $verboseEnabled;
         $this->debugEnabled = $debugEnabled;
+        $this->fastDebugEnabled = $fastDebugEnabled;
     }
 
     /**
@@ -83,13 +93,14 @@ private function __construct(
      * @param string  $phase
      * @param boolean $verboseEnabled
      * @param boolean $debugEnabled
+     * * @param boolean $fastDebugEnabled
      * @return void
      */
-    public static function create($forceGenerate, $phase, $verboseEnabled, $debugEnabled)
+    public static function create($forceGenerate, $phase, $verboseEnabled, $debugEnabled, $fastDebugEnabled)
     {
         if (self::$MFTF_APPLICATION_CONTEXT == null) {
             self::$MFTF_APPLICATION_CONTEXT =
-                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debugEnabled);
+                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debugEnabled, $fastDebugEnabled);
         }
     }
 
@@ -128,7 +139,7 @@ public function forceGenerateEnabled()
      */
     public function verboseEnabled()
     {
-        return $this->verboseEnabled ?? getenv('MFTF_DEBUG');
+        return $this->verboseEnabled ?? (strcasecmp(getenv('MFTF_DEBUG'),'true') == 0);
     }
 
     /**
@@ -139,7 +150,18 @@ public function verboseEnabled()
      */
     public function debugEnabled()
     {
-        return $this->debugEnabled ?? getenv('MFTF_DEBUG');
+        return $this->debugEnabled ?? (strcasecmp(getenv('MFTF_DEBUG'),'true') == 0);
+    }
+
+    /**
+     * Returns a boolean indicating whether the user has indicated a fast debug run, which will lengthy validation
+     * on merged file instead of 'per file' with some extra error messaging to be run
+     *
+     * @return boolean
+     */
+    public function fastDebugEnabled()
+    {
+        return $this->fastDebugEnabled ?? (strcasecmp(getenv('MFTF_FAST_DEBUG'),'true') == 0);
     }
 
     /**
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index 4c3aa581b..d7ea02fc0 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -232,6 +232,12 @@ protected function validateSchema($configMerger, $filename = null)
             $errors = [];
             if ($configMerger && !$configMerger->validate($this->schemaFile, $errors)) {
                 $message = $filename ? $filename . PHP_EOL . "Invalid Document \n" : PHP_EOL . "Invalid Document \n";
+                foreach ($errors as $error ){
+                    LoggingUtil::getInstance()->getLogger(Filesystem::class)->buildFailure(
+                        "XSD schema validation error",
+                        [ "file"=> $filename ? $filename: ":mergedFile:", "error" => $error]
+                    );
+                }
                 throw new \Exception($message . implode("\n", $errors));
             }
         }
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 728c4cb3a..fd5c3bda3 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -51,6 +51,9 @@ public function readFiles($fileList)
         if ($fileList->valid()) {
             $this->validateSchema($configMerger, $fileList->getFilename());
         }
+        if (MftfApplicationConfig::getConfig()->fastDebugEnabled()) {
+            $this->validateSchema($configMerger);
+        }
 
         $output = [];
         if ($configMerger) {
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
index db77bc74f..c10acd4f9 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
@@ -67,6 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
+            false,
             false
         );
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index fcd977060..00e6ded42 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -55,7 +55,12 @@ protected function configure()
                 'debug',
                 'd',
                 InputOption::VALUE_NONE,
-                'run extra validation when generating tests'
+                'run extra validation per file when generating tests'
+            )->addOption(
+                'fastdebug',
+                'a',
+                InputOption::VALUE_NONE,
+                'run extra validation on merged files when generating tests'
             );
 
         parent::configure();
@@ -79,8 +84,8 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
         $debug = $input->getOption('debug');
+        $fastDebug = $input->getOption('fastdebug');
         $remove = $input->getOption('remove');
-
         $verbose = $output->isVerbose();
 
         if ($json !== null && !json_decode($json)) {
@@ -95,10 +100,10 @@ protected function execute(InputInterface $input, OutputInterface $output)
 
         // Remove previous GENERATED_DIR if --remove option is used
         if ($remove) {
-            $this->removeGeneratedDirectory($output, $verbose || $debug);
+            $this->removeGeneratedDirectory($output, $verbose || $debug || $fastDebug);
         }
 
-        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $verbose);
+        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $fastDebug, $verbose);
 
         // create our manifest file here
         $testManifest = TestManifestFactory::makeManifest($config, $testConfiguration['suites']);
@@ -125,19 +130,21 @@ protected function execute(InputInterface $input, OutputInterface $output)
      * @param array   $tests
      * @param boolean $force
      * @param boolean $debug
+     * @param boolean $fastDebug
      * @param boolean $verbose
      * @return array
      * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException
      * @throws \Magento\FunctionalTestingFramework\Exceptions\XmlException
      */
-    private function createTestConfiguration($json, array $tests, bool $force, bool $debug, bool $verbose)
+    private function createTestConfiguration($json, array $tests, bool $force, bool $debug, bool $fastDebug, bool $verbose)
     {
         // set our application configuration so we can references the user options in our framework
         MftfApplicationConfig::create(
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             $verbose,
-            $debug
+            $debug,
+            $fastDebug
         );
 
         $testConfiguration = [];
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index 8257162b9..b16d49aa7 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -72,6 +72,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
+            false,
             false
         );
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index e895fb66a..6d990e330 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -77,6 +77,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
+            false,
             false
         );
 
diff --git a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
index e0b3cefd6..938d440a9 100644
--- a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
+++ b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
@@ -75,6 +75,7 @@ public function execute(InputInterface $input)
             true,
             MftfApplicationConfig::UNIT_TEST_PHASE,
             false,
+            false,
             false
         );
 
diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index 5844858a0..dea5604fb 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -13,7 +13,7 @@
 class MftfLogger extends Logger
 {
     /**
-     * Prints a deprecation warning, as well as adding a log at the WARNING level.
+     * Prints a deprecation warning, as well as adds a log at the WARNING level.
      *
      * @param  string $message The log message.
      * @param  array  $context The log context.
@@ -28,4 +28,21 @@ public function deprecation($message, array $context = [])
         }
         parent::warning($message, $context);
     }
+
+    /**
+     * Prints a critical failure, as well as adds a log at the CRITICAL level.
+     *
+     * @param  string $message The log message.
+     * @param  array  $context The log context.
+     * @return void
+     */
+    public function buildFailure($message, array $context = [])
+    {
+        $message = "BUILD FAILURE: " . $message;
+        // Suppress print during unit testing
+        if (MftfApplicationConfig::getConfig()->getPhase() !== MftfApplicationConfig::UNIT_TEST_PHASE) {
+            print ($message . json_encode($context) . "\n");
+        }
+        parent::critical($message, $context);
+    }
 }

From b3e59129b6b7264302ad51bf3013b9b6a59f982f Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 7 May 2019 16:53:18 -0500
Subject: [PATCH 23/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 composer.lock | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/composer.lock b/composer.lock
index 2e539fcf2..4309569de 100644
--- a/composer.lock
+++ b/composer.lock
@@ -2644,16 +2644,16 @@
         },
         {
             "name": "phpunit/phpunit",
-            "version": "6.5.13",
+            "version": "6.5.14",
             "source": {
                 "type": "git",
                 "url": "https://github.com/sebastianbergmann/phpunit.git",
-                "reference": "0973426fb012359b2f18d3bd1e90ef1172839693"
+                "reference": "bac23fe7ff13dbdb461481f706f0e9fe746334b7"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/0973426fb012359b2f18d3bd1e90ef1172839693",
-                "reference": "0973426fb012359b2f18d3bd1e90ef1172839693",
+                "url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/bac23fe7ff13dbdb461481f706f0e9fe746334b7",
+                "reference": "bac23fe7ff13dbdb461481f706f0e9fe746334b7",
                 "shasum": ""
             },
             "require": {
@@ -2724,7 +2724,7 @@
                 "testing",
                 "xunit"
             ],
-            "time": "2018-09-08T15:10:43+00:00"
+            "time": "2019-02-01T05:22:47+00:00"
         },
         {
             "name": "phpunit/phpunit-mock-objects",

From a4f9ff8675c7924c3f46e2c581aef5b932546a2c Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 8 May 2019 10:24:32 -0500
Subject: [PATCH 24/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 .../Config/Reader/Filesystem.php                           | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index d7ea02fc0..e895180f7 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -231,14 +231,13 @@ protected function validateSchema($configMerger, $filename = null)
         if ($this->validationState->isValidationRequired()) {
             $errors = [];
             if ($configMerger && !$configMerger->validate($this->schemaFile, $errors)) {
-                $message = $filename ? $filename . PHP_EOL . "Invalid Document \n" : PHP_EOL . "Invalid Document \n";
-                foreach ($errors as $error ){
+                foreach ($errors as $error){
                     LoggingUtil::getInstance()->getLogger(Filesystem::class)->buildFailure(
-                        "XSD schema validation error",
+                        "XSD schema error ",
                         [ "file"=> $filename ? $filename: ":mergedFile:", "error" => $error]
                     );
                 }
-                throw new \Exception($message . implode("\n", $errors));
+                throw new \Exception("Error: XSD schema issues found in file(s) " . $filename . "\n");
             }
         }
     }

From e76354c3b2d5c51bc2e11a60e11c6f04468907d3 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 8 May 2019 12:07:35 -0500
Subject: [PATCH 25/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 etc/config/.env.example                                       | 3 ++-
 .../Config/Reader/MftfFilesystem.php                          | 4 +++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/etc/config/.env.example b/etc/config/.env.example
index 5dc7168be..103f9c0ce 100644
--- a/etc/config/.env.example
+++ b/etc/config/.env.example
@@ -42,8 +42,9 @@ BROWSER=chrome
 MODULE_WHITELIST=Magento_Framework,Magento_ConfigurableProductWishlist,Magento_ConfigurableProductCatalogSearch
 #CUSTOM_MODULE_PATHS=
 
-#*** Bool property which allows the user to toggle debug output during test execution
+#*** Bool properties which allows the user to toggle debug output during test execution
 #MFTF_DEBUG=
+#MFTF_FAST_DEBUG=
 
 #*** Default timeout for wait actions
 #WAIT_TIMEOUT=10
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index fd5c3bda3..d6b1d15ae 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -6,6 +6,7 @@
 
 namespace Magento\FunctionalTestingFramework\Config\Reader;
 
+use Magento\AdminNotification\Block\Inbox;
 use Magento\FunctionalTestingFramework\Config\MftfApplicationConfig;
 use Magento\FunctionalTestingFramework\Exceptions\Collector\ExceptionCollector;
 use Magento\FunctionalTestingFramework\Util\Iterator\File;
@@ -51,7 +52,8 @@ public function readFiles($fileList)
         if ($fileList->valid()) {
             $this->validateSchema($configMerger, $fileList->getFilename());
         }
-        if (MftfApplicationConfig::getConfig()->fastDebugEnabled()) {
+        if (MftfApplicationConfig::getConfig()->fastDebugEnabled() &&
+            !MftfApplicationConfig::getConfig()->debugEnabled()) {
             $this->validateSchema($configMerger);
         }
 

From 6480ccc58c1cbdcf8c35f8d5f7f11db262beaad7 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 8 May 2019 12:59:00 -0500
Subject: [PATCH 26/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 composer.lock                                          | 10 +++++-----
 .../Config/Reader/MftfFilesystem.php                   |  1 -
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/composer.lock b/composer.lock
index 4309569de..2e539fcf2 100644
--- a/composer.lock
+++ b/composer.lock
@@ -2644,16 +2644,16 @@
         },
         {
             "name": "phpunit/phpunit",
-            "version": "6.5.14",
+            "version": "6.5.13",
             "source": {
                 "type": "git",
                 "url": "https://github.com/sebastianbergmann/phpunit.git",
-                "reference": "bac23fe7ff13dbdb461481f706f0e9fe746334b7"
+                "reference": "0973426fb012359b2f18d3bd1e90ef1172839693"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/bac23fe7ff13dbdb461481f706f0e9fe746334b7",
-                "reference": "bac23fe7ff13dbdb461481f706f0e9fe746334b7",
+                "url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/0973426fb012359b2f18d3bd1e90ef1172839693",
+                "reference": "0973426fb012359b2f18d3bd1e90ef1172839693",
                 "shasum": ""
             },
             "require": {
@@ -2724,7 +2724,7 @@
                 "testing",
                 "xunit"
             ],
-            "time": "2019-02-01T05:22:47+00:00"
+            "time": "2018-09-08T15:10:43+00:00"
         },
         {
             "name": "phpunit/phpunit-mock-objects",
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index d6b1d15ae..59c6d0df3 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -6,7 +6,6 @@
 
 namespace Magento\FunctionalTestingFramework\Config\Reader;
 
-use Magento\AdminNotification\Block\Inbox;
 use Magento\FunctionalTestingFramework\Config\MftfApplicationConfig;
 use Magento\FunctionalTestingFramework\Exceptions\Collector\ExceptionCollector;
 use Magento\FunctionalTestingFramework\Util\Iterator\File;

From f1f2381284ceb9771a72e2cd61df70400c2ad2af Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 8 May 2019 13:05:00 -0500
Subject: [PATCH 27/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

fixed comments
---
 .../Config/MftfApplicationConfig.php                          | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index 54930df3f..4a7064f21 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -143,7 +143,7 @@ public function verboseEnabled()
     }
 
     /**
-     * Returns a boolean indicating whether the user has indicated a debug run, which will lengthy validation
+     * Returns a boolean indicating whether the user has indicated a debug run, which will run lengthy validation
      * with some extra error messaging to be run
      *
      * @return boolean
@@ -154,7 +154,7 @@ public function debugEnabled()
     }
 
     /**
-     * Returns a boolean indicating whether the user has indicated a fast debug run, which will lengthy validation
+     * Returns a boolean indicating whether the user has indicated a fast debug run, which will run lengthy validation
      * on merged file instead of 'per file' with some extra error messaging to be run
      *
      * @return boolean

From baad7505e34a09db1fbda7426e6fdff6881b9305 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Thu, 9 May 2019 12:27:10 -0500
Subject: [PATCH 28/63] Add version table and update instructions.

---
 docs/versioning.md | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/docs/versioning.md b/docs/versioning.md
index 026cab8f8..55ae4dda6 100644
--- a/docs/versioning.md
+++ b/docs/versioning.md
@@ -30,7 +30,6 @@ X.Y.Z
 | | +-- Backward Compatible changes (Patch release - bug fixes, small additions)
 | +---- Backward Compatible changes (Minor release - small new features, bug fixes)
 +------ Backward Incompatible changes (Major release - new features and/or major changes)
-
 ```
 
 For example:
@@ -56,3 +55,32 @@ It MAY include patch level changes. Patch version MUST be reset to 0 when minor
 Major version **X** MUST be incremented for a release that introduces backward incompatible changes.
 A major release can also include minor and patch level changes.
 You must reset the patch and minor version to 0 when you change the major version.
+
+## MFTF version table
+
+This table lists the version of the MFTF that was released with a particular version of Magento.
+
+|||
+|---|---|
+| Magento | MFTF |
+| 2.3.1 | 2.3.13 |
+| 2.3.0 | 2.3.9 |
+| 2.2.8 | 2.3.13 |
+| 2.2.7 | 2.3.8 |
+
+## Updating MFTF
+
+The MFTF and the Magento product have differing release cycles, with the MFTF releasing more frequently. To get the latest version of the MFTF, update the version number in `composer.json` and update the package.
+
+1. Open `composer.json` at the <Magento root>.
+1. Find the MFTF listing:
+
+    ```json
+    "require-dev": {
+        "friendsofphp/php-cs-fixer": "~2.13.0",
+        "lusitanian/oauth": "~0.8.10",
+        "magento/magento2-functional-testing-framework": "2.3.13",
+    ```
+
+1. Update the version number and save.
+1. Run the `composer update` command.
\ No newline at end of file

From 3a7c942e8f8de6c79f81140fb28c1d225b091197 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Fri, 10 May 2019 11:06:36 -0500
Subject: [PATCH 29/63] Removed dupe upgrade instructions.

---
 docs/versioning.md | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/docs/versioning.md b/docs/versioning.md
index 55ae4dda6..6c9104230 100644
--- a/docs/versioning.md
+++ b/docs/versioning.md
@@ -67,20 +67,3 @@ This table lists the version of the MFTF that was released with a particular ver
 | 2.3.0 | 2.3.9 |
 | 2.2.8 | 2.3.13 |
 | 2.2.7 | 2.3.8 |
-
-## Updating MFTF
-
-The MFTF and the Magento product have differing release cycles, with the MFTF releasing more frequently. To get the latest version of the MFTF, update the version number in `composer.json` and update the package.
-
-1. Open `composer.json` at the <Magento root>.
-1. Find the MFTF listing:
-
-    ```json
-    "require-dev": {
-        "friendsofphp/php-cs-fixer": "~2.13.0",
-        "lusitanian/oauth": "~0.8.10",
-        "magento/magento2-functional-testing-framework": "2.3.13",
-    ```
-
-1. Update the version number and save.
-1. Run the `composer update` command.
\ No newline at end of file

From 741e81d5e131fd53359556d16e055a771c8fb447 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 14 May 2019 15:29:17 -0500
Subject: [PATCH 30/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

Updates for default debugging
---
 etc/config/.env.example                       |  3 +-
 .../Config/MftfApplicationConfig.php          | 58 +++++++------------
 .../Config/Reader/Filesystem.php              | 11 ++--
 .../Config/Reader/MftfFilesystem.php          | 10 +++-
 .../Console/GenerateDocsCommand.php           |  3 +-
 .../Console/GenerateTestsCommand.php          | 28 ++++-----
 .../Console/RunTestCommand.php                |  4 +-
 .../Console/RunTestFailedCommand.php          |  3 +-
 .../Console/RunTestGroupCommand.php           |  3 +-
 .../StaticCheck/TestDependencyCheck.php       |  3 +-
 .../Test/etc/Actions/commonAttributes.xsd     |  8 +--
 .../Test/etc/mergedTestSchema.xsd             |  8 ++-
 .../Util/Logger/MftfLogger.php                |  4 +-
 13 files changed, 62 insertions(+), 84 deletions(-)

diff --git a/etc/config/.env.example b/etc/config/.env.example
index 103f9c0ce..5dc7168be 100644
--- a/etc/config/.env.example
+++ b/etc/config/.env.example
@@ -42,9 +42,8 @@ BROWSER=chrome
 MODULE_WHITELIST=Magento_Framework,Magento_ConfigurableProductWishlist,Magento_ConfigurableProductCatalogSearch
 #CUSTOM_MODULE_PATHS=
 
-#*** Bool properties which allows the user to toggle debug output during test execution
+#*** Bool property which allows the user to toggle debug output during test execution
 #MFTF_DEBUG=
-#MFTF_FAST_DEBUG=
 
 #*** Default timeout for wait actions
 #WAIT_TIMEOUT=10
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index 4a7064f21..7806406db 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -14,6 +14,11 @@ class MftfApplicationConfig
     const UNIT_TEST_PHASE = "testing";
     const MFTF_PHASES = [self::GENERATION_PHASE, self::EXECUTION_PHASE, self::UNIT_TEST_PHASE];
 
+    const DEFAULT_DEBUG_MODE = "default";
+    const PER_FILE_DEBUG_MODE = "perFile";
+    const DISABLE_DEBUG_MODE = "ignore";
+    const MFTF_DEBUG_MODES = [self::DEFAULT_DEBUG_MODE, self::PER_FILE_DEBUG_MODE, self::DISABLE_DEBUG_MODE];
+
     /**
      * Determines whether the user has specified a force option for generation
      *
@@ -36,18 +41,11 @@ class MftfApplicationConfig
     private $verboseEnabled;
 
     /**
-     * Determines whether the user would like to execute mftf in a 'per file' debug mode
+     * String which identifies the current debug mode of mftf execution
      *
-     * @var boolean
-     */
-    private $debugEnabled;
-
-    /**
-     * Determines whether the user would like to execute mftf in a 'merged file' debug mode
-     *
-     * @var boolean
+     * @var string
      */
-    private $fastDebugEnabled;
+    private $debug;
 
     /**
      * MftfApplicationConfig Singelton Instance
@@ -60,18 +58,16 @@ class MftfApplicationConfig
      * MftfApplicationConfig constructor.
      *
      * @param boolean $forceGenerate
-     * @param string $phase
+     * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param boolean $debugEnabled
-     * @param null $fastDebugEnabled
+     * @param boolean $debug
      * @throws TestFrameworkException
      */
     private function __construct(
         $forceGenerate = false,
         $phase = self::EXECUTION_PHASE,
         $verboseEnabled = null,
-        $debugEnabled = null,
-        $fastDebugEnabled = null
+        $debug = null
     ) {
         $this->forceGenerate = $forceGenerate;
 
@@ -81,8 +77,7 @@ private function __construct(
 
         $this->phase = $phase;
         $this->verboseEnabled = $verboseEnabled;
-        $this->debugEnabled = $debugEnabled;
-        $this->fastDebugEnabled = $fastDebugEnabled;
+        $this->debug = $debug;
     }
 
     /**
@@ -92,15 +87,14 @@ private function __construct(
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param boolean $debugEnabled
-     * * @param boolean $fastDebugEnabled
+     * @param string $debug
      * @return void
      */
-    public static function create($forceGenerate, $phase, $verboseEnabled, $debugEnabled, $fastDebugEnabled)
+    public static function create($forceGenerate, $phase, $verboseEnabled, $debug)
     {
         if (self::$MFTF_APPLICATION_CONTEXT == null) {
             self::$MFTF_APPLICATION_CONTEXT =
-                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debugEnabled, $fastDebugEnabled);
+                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debug);
         }
     }
 
@@ -139,29 +133,17 @@ public function forceGenerateEnabled()
      */
     public function verboseEnabled()
     {
-        return $this->verboseEnabled ?? (strcasecmp(getenv('MFTF_DEBUG'),'true') == 0);
+        return $this->verboseEnabled ?? getenv('MFTF_DEBUG');
     }
 
     /**
-     * Returns a boolean indicating whether the user has indicated a debug run, which will run lengthy validation
-     * with some extra error messaging to be run
+     * Returns a string which indicates the debug mode of mftf execution.
      *
-     * @return boolean
-     */
-    public function debugEnabled()
-    {
-        return $this->debugEnabled ?? (strcasecmp(getenv('MFTF_DEBUG'),'true') == 0);
-    }
-
-    /**
-     * Returns a boolean indicating whether the user has indicated a fast debug run, which will run lengthy validation
-     * on merged file instead of 'per file' with some extra error messaging to be run
-     *
-     * @return boolean
+     * @return string
      */
-    public function fastDebugEnabled()
+    public function getDebugMode()
     {
-        return $this->fastDebugEnabled ?? (strcasecmp(getenv('MFTF_FAST_DEBUG'),'true') == 0);
+        return $this->debug ?? getenv('MFTF_DEBUG');
     }
 
     /**
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index e895180f7..26e431b76 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -157,7 +157,7 @@ protected function readFiles($fileList)
                 } else {
                     $configMerger->merge($content);
                 }
-                if (MftfApplicationConfig::getConfig()->debugEnabled()) {
+                if (MftfApplicationConfig::getConfig()->getDebugMode() == MftfApplicationConfig::PER_FILE_DEBUG_MODE) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -232,12 +232,13 @@ protected function validateSchema($configMerger, $filename = null)
             $errors = [];
             if ($configMerger && !$configMerger->validate($this->schemaFile, $errors)) {
                 foreach ($errors as $error){
-                    LoggingUtil::getInstance()->getLogger(Filesystem::class)->buildFailure(
-                        "XSD schema error ",
-                        [ "file"=> $filename ? $filename: ":mergedFile:", "error" => $error]
+                    $error = str_replace("\n", "", $error);
+                    LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
+                        "Schema validation error. ",
+                        ($filename ? [ "file"=> $filename, "error" => $error]: ["error" => $error])
                     );
                 }
-                throw new \Exception("Error: XSD schema issues found in file(s) " . $filename . "\n");
+                throw new \Exception("Error: schema validation errors found in xml file(s) " . $filename . "\n");
             }
         }
     }
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 59c6d0df3..2e5f38828 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -24,6 +24,7 @@ public function readFiles($fileList)
         $exceptionCollector = new ExceptionCollector();
         /** @var \Magento\FunctionalTestingFramework\Test\Config\Dom $configMerger */
         $configMerger = null;
+        $debugMode = MftfApplicationConfig::getConfig()->getDebugMode();
         foreach ($fileList as $key => $content) {
             //check if file is empty and continue to next if it is
             if (!parent::verifyFileEmpty($content, $fileList->getFilename())) {
@@ -40,7 +41,9 @@ public function readFiles($fileList)
                 } else {
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
-                if (MftfApplicationConfig::getConfig()->debugEnabled()) {
+                // per file debug mode - run per file validation for generate:tests -d
+                if (!in_array($debugMode, MftfApplicationConfig::MFTF_DEBUG_MODES) ||
+                    $debugMode == MftfApplicationConfig::PER_FILE_DEBUG_MODE) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -51,8 +54,9 @@ public function readFiles($fileList)
         if ($fileList->valid()) {
             $this->validateSchema($configMerger, $fileList->getFilename());
         }
-        if (MftfApplicationConfig::getConfig()->fastDebugEnabled() &&
-            !MftfApplicationConfig::getConfig()->debugEnabled()) {
+
+        // default debug mode - run validation on merged file for generate:tests
+        if ($debugMode == MftfApplicationConfig::DEFAULT_DEBUG_MODE) {
             $this->validateSchema($configMerger);
         }
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
index c10acd4f9..b555694c4 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
@@ -67,8 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            false,
-            false
+            MftfApplicationConfig::DISABLE_DEBUG_MODE
         );
 
         $allActionGroups = ActionGroupObjectHandler::getInstance()->getAllObjects();
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index 00e6ded42..3913fd2d7 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -29,7 +29,7 @@ class GenerateTestsCommand extends BaseGenerateCommand
     protected function configure()
     {
         $this->setName('generate:tests')
-            ->setDescription('This command generates all test files and suites based on xml declarations')
+            ->setDescription('This command runs validation and generates all test files and suites based on xml declarations')
             ->addArgument(
                 'name',
                 InputArgument::OPTIONAL | InputArgument::IS_ARRAY,
@@ -54,13 +54,9 @@ protected function configure()
             )->addOption(
                 'debug',
                 'd',
-                InputOption::VALUE_NONE,
-                'run extra validation per file when generating tests'
-            )->addOption(
-                'fastdebug',
-                'a',
-                InputOption::VALUE_NONE,
-                'run extra validation on merged files when generating tests'
+                InputOption::VALUE_OPTIONAL,
+                'run per file validation while running tests. Use option \'ignore\' to skip debugging',
+                'default'
             );
 
         parent::configure();
@@ -83,8 +79,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $json = $input->getOption('tests');
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
-        $debug = $input->getOption('debug');
-        $fastDebug = $input->getOption('fastdebug');
+        $debug = $input->getOption('debug')?? MftfApplicationConfig::PER_FILE_DEBUG_MODE; //set to per file debug as default
         $remove = $input->getOption('remove');
         $verbose = $output->isVerbose();
 
@@ -100,10 +95,11 @@ protected function execute(InputInterface $input, OutputInterface $output)
 
         // Remove previous GENERATED_DIR if --remove option is used
         if ($remove) {
-            $this->removeGeneratedDirectory($output, $verbose || $debug || $fastDebug);
+            $this->removeGeneratedDirectory($output, $verbose ||
+                ($debug !== MftfApplicationConfig::DISABLE_DEBUG_MODE));
         }
 
-        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $fastDebug, $verbose);
+        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $verbose);
 
         // create our manifest file here
         $testManifest = TestManifestFactory::makeManifest($config, $testConfiguration['suites']);
@@ -129,22 +125,20 @@ protected function execute(InputInterface $input, OutputInterface $output)
      * @param string  $json
      * @param array   $tests
      * @param boolean $force
-     * @param boolean $debug
-     * @param boolean $fastDebug
+     * @param string $debug
      * @param boolean $verbose
      * @return array
      * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException
      * @throws \Magento\FunctionalTestingFramework\Exceptions\XmlException
      */
-    private function createTestConfiguration($json, array $tests, bool $force, bool $debug, bool $fastDebug, bool $verbose)
+    private function createTestConfiguration($json, array $tests, bool $force, string $debug, bool $verbose)
     {
         // set our application configuration so we can references the user options in our framework
         MftfApplicationConfig::create(
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             $verbose,
-            $debug,
-            $fastDebug
+            $debug
         );
 
         $testConfiguration = [];
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
index 33a29b2e8..815daa419 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
@@ -7,6 +7,7 @@
 
 namespace Magento\FunctionalTestingFramework\Console;
 
+use Magento\FunctionalTestingFramework\Config\MftfApplicationConfig;
 use Symfony\Component\Console\Input\ArrayInput;
 use Symfony\Component\Console\Input\InputArgument;
 use Symfony\Component\Console\Input\InputInterface;
@@ -73,7 +74,8 @@ protected function execute(InputInterface $input, OutputInterface $output): int
                     'suites' => null
                 ]),
                 '--force' => $force,
-                '--remove' => $remove
+                '--remove' => $remove,
+                '--debug' => MftfApplicationConfig::DISABLE_DEBUG_MODE
             ];
             $command->run(new ArrayInput($args), $output);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index b16d49aa7..ccea7797e 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -72,8 +72,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            false,
-            false
+            MftfApplicationConfig::DISABLE_DEBUG_MODE
         );
 
         $testConfiguration = $this->getFailedTestList();
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index 6d990e330..244f3f30c 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -77,8 +77,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            false,
-            false
+            MftfApplicationConfig::DISABLE_DEBUG_MODE
         );
 
         if (!$skipGeneration) {
diff --git a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
index 938d440a9..d1b55623e 100644
--- a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
+++ b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
@@ -75,8 +75,7 @@ public function execute(InputInterface $input)
             true,
             MftfApplicationConfig::UNIT_TEST_PHASE,
             false,
-            false,
-            false
+            MftfApplicationConfig::DISABLE_DEBUG_MODE
         );
 
         ModuleResolver::getInstance()->getModulesPath();
diff --git a/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd b/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
index 22c05ea7d..e90759ee5 100644
--- a/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
+++ b/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
@@ -31,13 +31,6 @@
             </xs:annotation>
         </xs:attribute>
 
-        <xs:attribute type="xs:boolean" name="skipReadiness">
-            <xs:annotation>
-                <xs:documentation>
-                    Flag for skipping readiness check
-                </xs:documentation>
-            </xs:annotation>
-        </xs:attribute>
     </xs:attributeGroup>
 
     <xs:attribute type="xs:string" name="userInput" id="userInput">
@@ -87,4 +80,5 @@
             </xs:documentation>
         </xs:annotation>
     </xs:attribute>
+
 </xs:schema>
\ No newline at end of file
diff --git a/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd b/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd
index b663bb99d..fe45945ee 100644
--- a/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd
+++ b/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd
@@ -129,7 +129,13 @@
                 </xs:documentation>
             </xs:annotation>
         </xs:attribute>
+        <xs:attribute type="xs:boolean" name="skipReadiness" use="prohibited">
+            <xs:annotation>
+                <xs:documentation>
+                    Flag for skipping readiness check.
+                </xs:documentation>
+            </xs:annotation>
+        </xs:attribute>
         <xs:attributeGroup ref="commonActionAttributes"/>
-        <xs:attribute type="xs:boolean" name="skipReadiness" use="prohibited"/>
     </xs:complexType>
 </xs:schema>
diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index dea5604fb..dd0bcb8c4 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -36,9 +36,9 @@ public function deprecation($message, array $context = [])
      * @param  array  $context The log context.
      * @return void
      */
-    public function buildFailure($message, array $context = [])
+    public function criticalFailure($message, array $context = [])
     {
-        $message = "BUILD FAILURE: " . $message;
+        $message = "CRITICAL FAILURE: " . $message;
         // Suppress print during unit testing
         if (MftfApplicationConfig::getConfig()->getPhase() !== MftfApplicationConfig::UNIT_TEST_PHASE) {
             print ($message . json_encode($context) . "\n");

From 5e96fb4a13d37fefdf24e2366aab4efc754b2a8b Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 14 May 2019 15:54:37 -0500
Subject: [PATCH 31/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 .../Config/Reader/MftfFilesystem.php                          | 4 ++--
 .../Console/GenerateTestsCommand.php                          | 2 +-
 .../Test/etc/Actions/commonAttributes.xsd                     | 1 -
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 2e5f38828..0d2bbfdb1 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -41,7 +41,7 @@ public function readFiles($fileList)
                 } else {
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
-                // per file debug mode - run per file validation for generate:tests -d
+                // run per file validation with generate:tests -d
                 if (!in_array($debugMode, MftfApplicationConfig::MFTF_DEBUG_MODES) ||
                     $debugMode == MftfApplicationConfig::PER_FILE_DEBUG_MODE) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
@@ -55,7 +55,7 @@ public function readFiles($fileList)
             $this->validateSchema($configMerger, $fileList->getFilename());
         }
 
-        // default debug mode - run validation on merged file for generate:tests
+        // run validation on merged file with generate:tests
         if ($debugMode == MftfApplicationConfig::DEFAULT_DEBUG_MODE) {
             $this->validateSchema($configMerger);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index 3913fd2d7..b311aa063 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -79,7 +79,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $json = $input->getOption('tests');
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
-        $debug = $input->getOption('debug')?? MftfApplicationConfig::PER_FILE_DEBUG_MODE; //set to per file debug as default
+        $debug = $input->getOption('debug')?? MftfApplicationConfig::PER_FILE_DEBUG_MODE; // set to per file if no option specified
         $remove = $input->getOption('remove');
         $verbose = $output->isVerbose();
 
diff --git a/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd b/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
index e90759ee5..c39d779ed 100644
--- a/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
+++ b/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
@@ -80,5 +80,4 @@
             </xs:documentation>
         </xs:annotation>
     </xs:attribute>
-
 </xs:schema>
\ No newline at end of file

From f5d00407e0625ad693614531445ce0c633cd3ca0 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 15 May 2019 09:05:40 -0500
Subject: [PATCH 32/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 dev/tests/_bootstrap.php                              |  3 +--
 .../TestModule/ActionGroup/BasicActionGroup.xml       |  4 ----
 .../TestModule/Test/BasicFunctionalTest.xml           |  1 -
 .../verification/Tests/ActionGroupGenerationTest.php  | 11 -----------
 .../Config/MftfApplicationConfig.php                  |  6 +++---
 .../Config/Reader/Filesystem.php                      |  4 ++--
 .../Config/Reader/MftfFilesystem.php                  | 10 +++++-----
 .../Console/GenerateTestsCommand.php                  | 11 ++++++-----
 8 files changed, 17 insertions(+), 33 deletions(-)

diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php
index 41559184a..a4498a5cf 100644
--- a/dev/tests/_bootstrap.php
+++ b/dev/tests/_bootstrap.php
@@ -32,8 +32,7 @@
     true,
     \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::UNIT_TEST_PHASE,
     true,
-    false,
-    false
+    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::DISABLE_DEBUG_MODE
 );
 
 // Load needed framework env params
diff --git a/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml b/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml
index 245447f7d..45ccdbd24 100644
--- a/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml
+++ b/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml
@@ -115,10 +115,6 @@
         <see selector=".selector" userInput="{{sameStepKeyAsArg}}" stepKey="arg1" />
     </actionGroup>
 
-    <actionGroup name="actionGroupWithSkipReadinessActions">
-        <comment userInput="ActionGroupSkipReadiness" stepKey="skip" skipReadiness="true"/>
-    </actionGroup>
-
     <actionGroup name="actionGroupWithSectionAndData">
         <arguments>
             <argument name="content" type="string"/>
diff --git a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
index d120e5c31..bbe56e373 100644
--- a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
+++ b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
@@ -24,7 +24,6 @@
         </after>
         <comment stepKey="basicCommentWithNoData" userInput="{{emptyData.noData}}"/>
         <comment stepKey="basicCommentWithDefinitelyNoData" userInput="{{emptyData.definitelyNoData}}"/>
-        <comment stepKey="ReadinessCheckSkipped" userInput="skipReadiness" skipReadiness="true"/>
         <grabValueFrom stepKey="someVarDefinition"/>
         <acceptPopup stepKey="acceptPopupKey1"/>
         <amOnPage stepKey="amOnPageKey1" url="/test/url"/>
diff --git a/dev/tests/verification/Tests/ActionGroupGenerationTest.php b/dev/tests/verification/Tests/ActionGroupGenerationTest.php
index dec8c6d14..613d6cb72 100644
--- a/dev/tests/verification/Tests/ActionGroupGenerationTest.php
+++ b/dev/tests/verification/Tests/ActionGroupGenerationTest.php
@@ -185,17 +185,6 @@ public function testActionGroupWithArgContainingStepKey()
         $this->generateAndCompareTest('ActionGroupContainsStepKeyInArgText');
     }
 
-    /**
-     * Test an action group with an arg containing stepKey text
-     *
-     * @throws \Exception
-     * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException
-     */
-    public function testActionGroupWithSkipReadiness()
-    {
-        $this->generateAndCompareTest('ActionGroupSkipReadiness');
-    }
-
     /**
      * Test an action group with an arg containing stepKey text
      *
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index 7806406db..2c35fbe21 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -15,9 +15,9 @@ class MftfApplicationConfig
     const MFTF_PHASES = [self::GENERATION_PHASE, self::EXECUTION_PHASE, self::UNIT_TEST_PHASE];
 
     const DEFAULT_DEBUG_MODE = "default";
-    const PER_FILE_DEBUG_MODE = "perFile";
+    const DEVELOPER_MODE = "developer";
     const DISABLE_DEBUG_MODE = "ignore";
-    const MFTF_DEBUG_MODES = [self::DEFAULT_DEBUG_MODE, self::PER_FILE_DEBUG_MODE, self::DISABLE_DEBUG_MODE];
+    const MFTF_DEBUG_MODES = [self::DEFAULT_DEBUG_MODE, self::DEVELOPER_MODE, self::DISABLE_DEBUG_MODE];
 
     /**
      * Determines whether the user has specified a force option for generation
@@ -87,7 +87,7 @@ private function __construct(
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param string $debug
+     * @param string  $debug
      * @return void
      */
     public static function create($forceGenerate, $phase, $verboseEnabled, $debug)
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index 26e431b76..89699eff6 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -157,7 +157,7 @@ protected function readFiles($fileList)
                 } else {
                     $configMerger->merge($content);
                 }
-                if (MftfApplicationConfig::getConfig()->getDebugMode() == MftfApplicationConfig::PER_FILE_DEBUG_MODE) {
+                if (MftfApplicationConfig::getConfig()->getDebugMode() === MftfApplicationConfig::DEVELOPER_MODE) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -231,7 +231,7 @@ protected function validateSchema($configMerger, $filename = null)
         if ($this->validationState->isValidationRequired()) {
             $errors = [];
             if ($configMerger && !$configMerger->validate($this->schemaFile, $errors)) {
-                foreach ($errors as $error){
+                foreach ($errors as $error) {
                     $error = str_replace("\n", "", $error);
                     LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
                         "Schema validation error. ",
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 0d2bbfdb1..5ab447e57 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -41,9 +41,9 @@ public function readFiles($fileList)
                 } else {
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
-                // run per file validation with generate:tests -d
+                 // run per file validation with generate:tests -d
                 if (!in_array($debugMode, MftfApplicationConfig::MFTF_DEBUG_MODES) ||
-                    $debugMode == MftfApplicationConfig::PER_FILE_DEBUG_MODE) {
+                    $debugMode === MftfApplicationConfig::DEVELOPER_MODE) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -55,9 +55,9 @@ public function readFiles($fileList)
             $this->validateSchema($configMerger, $fileList->getFilename());
         }
 
-        // run validation on merged file with generate:tests
-        if ($debugMode == MftfApplicationConfig::DEFAULT_DEBUG_MODE) {
-            $this->validateSchema($configMerger);
+         //run validation on merged file with generate:tests
+        if ($debugMode === MftfApplicationConfig::DEFAULT_DEBUG_MODE) {
+            $this->validateSchema($configMerger, $debugMode);
         }
 
         $output = [];
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index b311aa063..5ff9f8783 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -29,7 +29,7 @@ class GenerateTestsCommand extends BaseGenerateCommand
     protected function configure()
     {
         $this->setName('generate:tests')
-            ->setDescription('This command runs validation and generates all test files and suites based on xml declarations')
+            ->setDescription('Run validation and generate all test files and suites based on xml declarations')
             ->addArgument(
                 'name',
                 InputArgument::OPTIONAL | InputArgument::IS_ARRAY,
@@ -39,7 +39,7 @@ protected function configure()
                 "force",
                 'f',
                 InputOption::VALUE_NONE,
-                'force generation of tests regardless of Magento Instance Configuration'
+                'Force generation of tests regardless of Magento Instance Configuration'
             )->addOption(
                 'time',
                 'i',
@@ -55,7 +55,8 @@ protected function configure()
                 'debug',
                 'd',
                 InputOption::VALUE_OPTIONAL,
-                'run per file validation while running tests. Use option \'ignore\' to skip debugging',
+                'Run per file validation while running tests. Use option \'ignore\' to skip debugging -- 
+                 added for backward compatibility, will be removed in the next MAJOR release',
                 'default'
             );
 
@@ -79,7 +80,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $json = $input->getOption('tests');
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
-        $debug = $input->getOption('debug')?? MftfApplicationConfig::PER_FILE_DEBUG_MODE; // set to per file if no option specified
+        $debug = $input->getOption('debug')?? MftfApplicationConfig::DEVELOPER_MODE;
         $remove = $input->getOption('remove');
         $verbose = $output->isVerbose();
 
@@ -125,7 +126,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
      * @param string  $json
      * @param array   $tests
      * @param boolean $force
-     * @param string $debug
+     * @param string  $debug
      * @param boolean $verbose
      * @return array
      * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException

From 7ad4084cb27193dba5c055afdf6ec6395ca77949 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 15 May 2019 11:16:32 -0500
Subject: [PATCH 33/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 dev/tests/verification/Resources/BasicFunctionalTest.txt     | 4 +---
 .../verification/TestModule/Test/BasicFunctionalTest.xml     | 1 +
 dev/tests/verification/Tests/SchemaValidationTest.php        | 2 +-
 .../Config/Reader/MftfFilesystem.php                         | 5 +----
 4 files changed, 4 insertions(+), 8 deletions(-)

diff --git a/dev/tests/verification/Resources/BasicFunctionalTest.txt b/dev/tests/verification/Resources/BasicFunctionalTest.txt
index 9970e81ec..fd5f0f165 100644
--- a/dev/tests/verification/Resources/BasicFunctionalTest.txt
+++ b/dev/tests/verification/Resources/BasicFunctionalTest.txt
@@ -60,9 +60,7 @@ class BasicFunctionalTestCest
 	{
 		$I->comment("");
 		$I->comment("");
-		$I->skipReadinessCheck(true);
-		$I->comment("skipReadiness");
-		$I->skipReadinessCheck(false);
+		$I->comment("seeComment");
 		$someVarDefinition = $I->grabValueFrom();
 		$I->acceptPopup();
 		$I->amOnPage("/test/url");
diff --git a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
index bbe56e373..bdeb4e0c6 100644
--- a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
+++ b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
@@ -24,6 +24,7 @@
         </after>
         <comment stepKey="basicCommentWithNoData" userInput="{{emptyData.noData}}"/>
         <comment stepKey="basicCommentWithDefinitelyNoData" userInput="{{emptyData.definitelyNoData}}"/>
+        <comment stepKey="basicCommentWithData" userInput="seeComment"/>
         <grabValueFrom stepKey="someVarDefinition"/>
         <acceptPopup stepKey="acceptPopupKey1"/>
         <amOnPage stepKey="amOnPageKey1" url="/test/url"/>
diff --git a/dev/tests/verification/Tests/SchemaValidationTest.php b/dev/tests/verification/Tests/SchemaValidationTest.php
index 86828e9a5..14b2d7149 100644
--- a/dev/tests/verification/Tests/SchemaValidationTest.php
+++ b/dev/tests/verification/Tests/SchemaValidationTest.php
@@ -19,7 +19,7 @@ class SchemaValidationTest extends MftfTestCase
      */
     public function testInvalidTestSchema()
     {
-        AspectMock::double(MftfApplicationConfig::class, ['debugEnabled' => true]);
+        AspectMock::double(MftfApplicationConfig::class, ['getDebugMode' => MftfApplicationConfig::DEVELOPER_MODE]);
         $testFile = ['testFile.xml' => "<tests><test name='testName'><annotations>a</annotations></test></tests>"];
         $expectedError = TESTS_MODULE_PATH .
             DIRECTORY_SEPARATOR .
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 5ab447e57..7960b4e3b 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -51,13 +51,10 @@ public function readFiles($fileList)
             }
         }
         $exceptionCollector->throwException();
-        if ($fileList->valid()) {
-            $this->validateSchema($configMerger, $fileList->getFilename());
-        }
 
          //run validation on merged file with generate:tests
         if ($debugMode === MftfApplicationConfig::DEFAULT_DEBUG_MODE) {
-            $this->validateSchema($configMerger, $debugMode);
+            $this->validateSchema($configMerger);
         }
 
         $output = [];

From 3f67629b52ef6d1b9f6d11d463b3d1ac0b437bd9 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 15 May 2019 13:04:52 -0500
Subject: [PATCH 34/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 .../FunctionalTestingFramework/Console/GenerateTestsCommand.php | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index 5ff9f8783..3443de2aa 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -55,7 +55,7 @@ protected function configure()
                 'debug',
                 'd',
                 InputOption::VALUE_OPTIONAL,
-                'Run per file validation while running tests. Use option \'ignore\' to skip debugging -- 
+                'Run extra validation when generating tests. Use option \'ignore\' to skip debugging -- 
                  added for backward compatibility, will be removed in the next MAJOR release',
                 'default'
             );

From 50396644021313f47245f8b231a7293125803a3e Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Wed, 15 May 2019 16:24:57 -0500
Subject: [PATCH 35/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 .../Config/Reader/Filesystem.php                           | 6 +++---
 .../FunctionalTestingFramework/Util/Logger/MftfLogger.php  | 7 ++++---
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index 89699eff6..e82e53894 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -232,13 +232,13 @@ protected function validateSchema($configMerger, $filename = null)
             $errors = [];
             if ($configMerger && !$configMerger->validate($this->schemaFile, $errors)) {
                 foreach ($errors as $error) {
-                    $error = str_replace("\n", "", $error);
+                    $error = str_replace(PHP_EOL, "", $error);
                     LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
-                        "Schema validation error. ",
+                        "Schema validation error:",
                         ($filename ? [ "file"=> $filename, "error" => $error]: ["error" => $error])
                     );
                 }
-                throw new \Exception("Error: schema validation errors found in xml file(s) " . $filename . "\n");
+                throw new \Exception("Schema validation errors found in xml file(s).");
             }
         }
     }
diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index dd0bcb8c4..20a360b2e 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -34,14 +34,15 @@ public function deprecation($message, array $context = [])
      *
      * @param  string $message The log message.
      * @param  array  $context The log context.
+     * @param  string  $filename
      * @return void
      */
-    public function criticalFailure($message, array $context = [])
+    public function criticalFailure($message, array $context = [], $filename = null)
     {
-        $message = "CRITICAL FAILURE: " . $message;
+        $message = "FAILURE: " . $message . $filename;
         // Suppress print during unit testing
         if (MftfApplicationConfig::getConfig()->getPhase() !== MftfApplicationConfig::UNIT_TEST_PHASE) {
-            print ($message . json_encode($context) . "\n");
+            print ($message . implode( "\n", $context) . "\n");
         }
         parent::critical($message, $context);
     }

From ea5b72ece351bfcd7d844d2bff29e9aa4bb12dd4 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Thu, 16 May 2019 09:19:08 -0500
Subject: [PATCH 36/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 .../Config/Reader/Filesystem.php                          | 4 ++--
 .../FunctionalTestingFramework/Util/Logger/MftfLogger.php | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index e82e53894..6d6e8f1b6 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -234,11 +234,11 @@ protected function validateSchema($configMerger, $filename = null)
                 foreach ($errors as $error) {
                     $error = str_replace(PHP_EOL, "", $error);
                     LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
-                        "Schema validation error:",
+                        "Schema validation error ",
                         ($filename ? [ "file"=> $filename, "error" => $error]: ["error" => $error])
                     );
                 }
-                throw new \Exception("Schema validation errors found in xml file(s).");
+                throw new \Exception("Schema validation errors found in xml file(s)" . $filename);
             }
         }
     }
diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index 20a360b2e..d762ddf20 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -32,14 +32,14 @@ public function deprecation($message, array $context = [])
     /**
      * Prints a critical failure, as well as adds a log at the CRITICAL level.
      *
-     * @param  string $message The log message.
-     * @param  array  $context The log context.
+     * @param  string $message  The log message.
+     * @param  array  $context  The log context.
      * @param  string  $filename
      * @return void
      */
-    public function criticalFailure($message, array $context = [], $filename = null)
+    public function criticalFailure($message, array $context = [])
     {
-        $message = "FAILURE: " . $message . $filename;
+        $message = "FAILURE: " . $message;
         // Suppress print during unit testing
         if (MftfApplicationConfig::getConfig()->getPhase() !== MftfApplicationConfig::UNIT_TEST_PHASE) {
             print ($message . implode( "\n", $context) . "\n");

From 29d5435b95ab26629ab3894bff6dde2d8ea30e26 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Thu, 16 May 2019 09:23:33 -0500
Subject: [PATCH 37/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 .../FunctionalTestingFramework/Util/Logger/MftfLogger.php        | 1 -
 1 file changed, 1 deletion(-)

diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index d762ddf20..9c2f0c457 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -34,7 +34,6 @@ public function deprecation($message, array $context = [])
      *
      * @param  string $message  The log message.
      * @param  array  $context  The log context.
-     * @param  string  $filename
      * @return void
      */
     public function criticalFailure($message, array $context = [])

From 592aba911e4ef92485a9148d281c0788953d8cf5 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Thu, 16 May 2019 10:00:49 -0500
Subject: [PATCH 38/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files' fxing code sniffer errors

---
 .../FunctionalTestingFramework/Util/Logger/MftfLogger.php     | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index 9c2f0c457..1cb9139b9 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -32,8 +32,8 @@ public function deprecation($message, array $context = [])
     /**
      * Prints a critical failure, as well as adds a log at the CRITICAL level.
      *
-     * @param  string $message  The log message.
-     * @param  array  $context  The log context.
+     * @param  string $message The log message.
+     * @param  array  $context The log context.
      * @return void
      */
     public function criticalFailure($message, array $context = [])

From 3f39f4496f9a67ee1ad0b46d44373304d3109a82 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Thu, 16 May 2019 10:29:48 -0500
Subject: [PATCH 39/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

fixed code sniffer errors
---
 .../FunctionalTestingFramework/Util/Logger/MftfLogger.php       | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index 1cb9139b9..0a52f6b6b 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -41,7 +41,7 @@ public function criticalFailure($message, array $context = [])
         $message = "FAILURE: " . $message;
         // Suppress print during unit testing
         if (MftfApplicationConfig::getConfig()->getPhase() !== MftfApplicationConfig::UNIT_TEST_PHASE) {
-            print ($message . implode( "\n", $context) . "\n");
+            print ($message . implode("\n", $context) . "\n");
         }
         parent::critical($message, $context);
     }

From 489ca5e8465f4ef88bf22e5df4d8e6ba8165c4e2 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Fri, 17 May 2019 15:30:00 -0500
Subject: [PATCH 40/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 dev/tests/_bootstrap.php                      |  2 +-
 .../Tests/SchemaValidationTest.php            |  2 +-
 .../Config/MftfApplicationConfig.php          | 37 ++++++++++++-------
 .../Config/Reader/Filesystem.php              |  4 +-
 .../Config/Reader/MftfFilesystem.php          |  7 ++--
 .../Console/GenerateDocsCommand.php           |  2 +-
 .../Console/GenerateTestsCommand.php          | 10 ++---
 .../Console/RunTestCommand.php                |  2 +-
 .../Console/RunTestFailedCommand.php          |  2 +-
 .../Console/RunTestGroupCommand.php           |  2 +-
 .../StaticCheck/TestDependencyCheck.php       |  2 +-
 11 files changed, 41 insertions(+), 31 deletions(-)

diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php
index a4498a5cf..35260691f 100644
--- a/dev/tests/_bootstrap.php
+++ b/dev/tests/_bootstrap.php
@@ -32,7 +32,7 @@
     true,
     \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::UNIT_TEST_PHASE,
     true,
-    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::DISABLE_DEBUG_MODE
+    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::MODE_PRODUCTION
 );
 
 // Load needed framework env params
diff --git a/dev/tests/verification/Tests/SchemaValidationTest.php b/dev/tests/verification/Tests/SchemaValidationTest.php
index 14b2d7149..b01aaea27 100644
--- a/dev/tests/verification/Tests/SchemaValidationTest.php
+++ b/dev/tests/verification/Tests/SchemaValidationTest.php
@@ -19,7 +19,7 @@ class SchemaValidationTest extends MftfTestCase
      */
     public function testInvalidTestSchema()
     {
-        AspectMock::double(MftfApplicationConfig::class, ['getDebugMode' => MftfApplicationConfig::DEVELOPER_MODE]);
+        AspectMock::double(MftfApplicationConfig::class, ['getMode' => MftfApplicationConfig::MODE_DEVELOPER]);
         $testFile = ['testFile.xml' => "<tests><test name='testName'><annotations>a</annotations></test></tests>"];
         $expectedError = TESTS_MODULE_PATH .
             DIRECTORY_SEPARATOR .
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index 2c35fbe21..4a3c6345d 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -14,10 +14,13 @@ class MftfApplicationConfig
     const UNIT_TEST_PHASE = "testing";
     const MFTF_PHASES = [self::GENERATION_PHASE, self::EXECUTION_PHASE, self::UNIT_TEST_PHASE];
 
-    const DEFAULT_DEBUG_MODE = "default";
-    const DEVELOPER_MODE = "developer";
-    const DISABLE_DEBUG_MODE = "ignore";
-    const MFTF_DEBUG_MODES = [self::DEFAULT_DEBUG_MODE, self::DEVELOPER_MODE, self::DISABLE_DEBUG_MODE];
+    /**
+     * Mftf debug modes
+     */
+    const MODE_DEFAULT = "default";
+    const MODE_DEVELOPER = "developer";
+    const MODE_PRODUCTION = "off";
+    const MFTF_DEBUG_MODES = [self::MODE_DEFAULT, self::MODE_DEVELOPER, self::MODE_PRODUCTION];
 
     /**
      * Determines whether the user has specified a force option for generation
@@ -45,7 +48,7 @@ class MftfApplicationConfig
      *
      * @var string
      */
-    private $debug;
+    private $mode;
 
     /**
      * MftfApplicationConfig Singelton Instance
@@ -60,14 +63,14 @@ class MftfApplicationConfig
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param boolean $debug
+     * @param string $mode
      * @throws TestFrameworkException
      */
     private function __construct(
         $forceGenerate = false,
         $phase = self::EXECUTION_PHASE,
         $verboseEnabled = null,
-        $debug = null
+        $mode = self::MODE_PRODUCTION
     ) {
         $this->forceGenerate = $forceGenerate;
 
@@ -77,7 +80,15 @@ private function __construct(
 
         $this->phase = $phase;
         $this->verboseEnabled = $verboseEnabled;
-        $this->debug = $debug;
+        switch ($mode) {
+            case self::MODE_DEVELOPER:
+            case self::MODE_DEFAULT:
+            case self::MODE_PRODUCTION:
+                $this->mode = $mode;
+                break;
+            default:
+                $this->mode = self::MODE_DEVELOPER;
+        }
     }
 
     /**
@@ -87,14 +98,14 @@ private function __construct(
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param string  $debug
+     * @param string  $mode
      * @return void
      */
-    public static function create($forceGenerate, $phase, $verboseEnabled, $debug)
+    public static function create($forceGenerate, $phase, $verboseEnabled, $mode)
     {
         if (self::$MFTF_APPLICATION_CONTEXT == null) {
             self::$MFTF_APPLICATION_CONTEXT =
-                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debug);
+                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $mode);
         }
     }
 
@@ -141,9 +152,9 @@ public function verboseEnabled()
      *
      * @return string
      */
-    public function getDebugMode()
+    public function getMode()
     {
-        return $this->debug ?? getenv('MFTF_DEBUG');
+        return $this->mode ?? getenv('MFTF_DEBUG');
     }
 
     /**
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index 6d6e8f1b6..707c54eb9 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -157,7 +157,7 @@ protected function readFiles($fileList)
                 } else {
                     $configMerger->merge($content);
                 }
-                if (MftfApplicationConfig::getConfig()->getDebugMode() === MftfApplicationConfig::DEVELOPER_MODE) {
+                if (MftfApplicationConfig::getConfig()->getMode() === MftfApplicationConfig::MODE_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -235,7 +235,7 @@ protected function validateSchema($configMerger, $filename = null)
                     $error = str_replace(PHP_EOL, "", $error);
                     LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
                         "Schema validation error ",
-                        ($filename ? [ "file"=> $filename, "error" => $error]: ["error" => $error])
+                        ($filename ? [ "file"=> $filename, "error" => $error]: ["schema" => $this->schemaFile, "error" => $error])
                     );
                 }
                 throw new \Exception("Schema validation errors found in xml file(s)" . $filename);
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 7960b4e3b..d86f26da1 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -24,7 +24,7 @@ public function readFiles($fileList)
         $exceptionCollector = new ExceptionCollector();
         /** @var \Magento\FunctionalTestingFramework\Test\Config\Dom $configMerger */
         $configMerger = null;
-        $debugMode = MftfApplicationConfig::getConfig()->getDebugMode();
+        $mode = MftfApplicationConfig::getConfig()->getMode();
         foreach ($fileList as $key => $content) {
             //check if file is empty and continue to next if it is
             if (!parent::verifyFileEmpty($content, $fileList->getFilename())) {
@@ -42,8 +42,7 @@ public function readFiles($fileList)
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
                  // run per file validation with generate:tests -d
-                if (!in_array($debugMode, MftfApplicationConfig::MFTF_DEBUG_MODES) ||
-                    $debugMode === MftfApplicationConfig::DEVELOPER_MODE) {
+                if ($mode == MftfApplicationConfig::MODE_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -53,7 +52,7 @@ public function readFiles($fileList)
         $exceptionCollector->throwException();
 
          //run validation on merged file with generate:tests
-        if ($debugMode === MftfApplicationConfig::DEFAULT_DEBUG_MODE) {
+        if ($mode == MftfApplicationConfig::MODE_DEFAULT) {
             $this->validateSchema($configMerger);
         }
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
index b555694c4..db747d160 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
@@ -67,7 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::DISABLE_DEBUG_MODE
+            MftfApplicationConfig::MODE_PRODUCTION
         );
 
         $allActionGroups = ActionGroupObjectHandler::getInstance()->getAllObjects();
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index 3443de2aa..e97ecdbaf 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -55,7 +55,7 @@ protected function configure()
                 'debug',
                 'd',
                 InputOption::VALUE_OPTIONAL,
-                'Run extra validation when generating tests. Use option \'ignore\' to skip debugging -- 
+                'Run extra validation when generating tests. Use option \'off\' to turn off debugging -- 
                  added for backward compatibility, will be removed in the next MAJOR release',
                 'default'
             );
@@ -80,7 +80,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $json = $input->getOption('tests');
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
-        $debug = $input->getOption('debug')?? MftfApplicationConfig::DEVELOPER_MODE;
+        $mode = $input->getOption('debug');
         $remove = $input->getOption('remove');
         $verbose = $output->isVerbose();
 
@@ -97,10 +97,10 @@ protected function execute(InputInterface $input, OutputInterface $output)
         // Remove previous GENERATED_DIR if --remove option is used
         if ($remove) {
             $this->removeGeneratedDirectory($output, $verbose ||
-                ($debug !== MftfApplicationConfig::DISABLE_DEBUG_MODE));
+                ($mode !== MftfApplicationConfig::MODE_PRODUCTION));
         }
 
-        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $verbose);
+        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $mode, $verbose);
 
         // create our manifest file here
         $testManifest = TestManifestFactory::makeManifest($config, $testConfiguration['suites']);
@@ -132,7 +132,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
      * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException
      * @throws \Magento\FunctionalTestingFramework\Exceptions\XmlException
      */
-    private function createTestConfiguration($json, array $tests, bool $force, string $debug, bool $verbose)
+    private function createTestConfiguration($json, array $tests, bool $force, $debug, bool $verbose)
     {
         // set our application configuration so we can references the user options in our framework
         MftfApplicationConfig::create(
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
index 815daa419..82294307d 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
@@ -75,7 +75,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
                 ]),
                 '--force' => $force,
                 '--remove' => $remove,
-                '--debug' => MftfApplicationConfig::DISABLE_DEBUG_MODE
+                '--debug' => MftfApplicationConfig::MODE_PRODUCTION
             ];
             $command->run(new ArrayInput($args), $output);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index ccea7797e..b39419adc 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -72,7 +72,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::DISABLE_DEBUG_MODE
+            MftfApplicationConfig::MODE_PRODUCTION
         );
 
         $testConfiguration = $this->getFailedTestList();
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index 244f3f30c..ef88fdabc 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -77,7 +77,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::DISABLE_DEBUG_MODE
+            MftfApplicationConfig::MODE_PRODUCTION
         );
 
         if (!$skipGeneration) {
diff --git a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
index d1b55623e..6813e4e89 100644
--- a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
+++ b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
@@ -75,7 +75,7 @@ public function execute(InputInterface $input)
             true,
             MftfApplicationConfig::UNIT_TEST_PHASE,
             false,
-            MftfApplicationConfig::DISABLE_DEBUG_MODE
+            MftfApplicationConfig::MODE_PRODUCTION
         );
 
         ModuleResolver::getInstance()->getModulesPath();

From 27bd6b32fd99521b1c24990fd70c102a4fcacb18 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Fri, 17 May 2019 16:30:51 -0500
Subject: [PATCH 41/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

fixed static-check errors
---
 .../FunctionalTestingFramework/Config/MftfApplicationConfig.php | 2 +-
 .../FunctionalTestingFramework/Config/Reader/Filesystem.php     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index 4a3c6345d..a5859891e 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -63,7 +63,7 @@ class MftfApplicationConfig
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param string $mode
+     * @param string  $mode
      * @throws TestFrameworkException
      */
     private function __construct(
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index 707c54eb9..c40db4029 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -235,7 +235,7 @@ protected function validateSchema($configMerger, $filename = null)
                     $error = str_replace(PHP_EOL, "", $error);
                     LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
                         "Schema validation error ",
-                        ($filename ? [ "file"=> $filename, "error" => $error]: ["schema" => $this->schemaFile, "error" => $error])
+                        ($filename ? [ "file"=> $filename, "error" => $error]: ["error" => $error])
                     );
                 }
                 throw new \Exception("Schema validation errors found in xml file(s)" . $filename);

From eb83bc5195a982c75e7937c17bb4a6b6ea817570 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Mon, 20 May 2019 16:30:32 -0500
Subject: [PATCH 42/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

Changes from review comments
---
 dev/tests/_bootstrap.php                      |  2 +-
 .../Tests/SchemaValidationTest.php            |  2 +-
 .../Config/MftfApplicationConfig.php          | 42 +++++++++----------
 .../Config/Reader/Filesystem.php              |  2 +-
 .../Config/Reader/MftfFilesystem.php          |  6 +--
 .../Console/GenerateDocsCommand.php           |  2 +-
 .../Console/GenerateTestsCommand.php          |  8 ++--
 .../Console/RunTestCommand.php                |  2 +-
 .../Console/RunTestFailedCommand.php          |  2 +-
 .../Console/RunTestGroupCommand.php           |  2 +-
 .../StaticCheck/TestDependencyCheck.php       |  2 +-
 11 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php
index 35260691f..2b6fc0934 100644
--- a/dev/tests/_bootstrap.php
+++ b/dev/tests/_bootstrap.php
@@ -32,7 +32,7 @@
     true,
     \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::UNIT_TEST_PHASE,
     true,
-    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::MODE_PRODUCTION
+    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::DEBUG_NONE
 );
 
 // Load needed framework env params
diff --git a/dev/tests/verification/Tests/SchemaValidationTest.php b/dev/tests/verification/Tests/SchemaValidationTest.php
index b01aaea27..92442bd37 100644
--- a/dev/tests/verification/Tests/SchemaValidationTest.php
+++ b/dev/tests/verification/Tests/SchemaValidationTest.php
@@ -19,7 +19,7 @@ class SchemaValidationTest extends MftfTestCase
      */
     public function testInvalidTestSchema()
     {
-        AspectMock::double(MftfApplicationConfig::class, ['getMode' => MftfApplicationConfig::MODE_DEVELOPER]);
+        AspectMock::double(MftfApplicationConfig::class, ['getDebugLevel' => MftfApplicationConfig::LEVEL_DEVELOPER]);
         $testFile = ['testFile.xml' => "<tests><test name='testName'><annotations>a</annotations></test></tests>"];
         $expectedError = TESTS_MODULE_PATH .
             DIRECTORY_SEPARATOR .
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index a5859891e..64c4ee7aa 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -15,12 +15,12 @@ class MftfApplicationConfig
     const MFTF_PHASES = [self::GENERATION_PHASE, self::EXECUTION_PHASE, self::UNIT_TEST_PHASE];
 
     /**
-     * Mftf debug modes
+     * Mftf debug levels
      */
-    const MODE_DEFAULT = "default";
-    const MODE_DEVELOPER = "developer";
-    const MODE_PRODUCTION = "off";
-    const MFTF_DEBUG_MODES = [self::MODE_DEFAULT, self::MODE_DEVELOPER, self::MODE_PRODUCTION];
+    const LEVEL_DEFAULT = "default";
+    const LEVEL_DEVELOPER = "developer";
+    const DEBUG_NONE = "none";
+    const MFTF_DEBUG_LEVEL = [self::LEVEL_DEFAULT, self::LEVEL_DEVELOPER, self::DEBUG_NONE];
 
     /**
      * Determines whether the user has specified a force option for generation
@@ -44,11 +44,11 @@ class MftfApplicationConfig
     private $verboseEnabled;
 
     /**
-     * String which identifies the current debug mode of mftf execution
+     * String which identifies the current debug level of mftf execution
      *
      * @var string
      */
-    private $mode;
+    private $debugLevel;
 
     /**
      * MftfApplicationConfig Singelton Instance
@@ -63,14 +63,14 @@ class MftfApplicationConfig
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param string  $mode
+     * @param string  $debugLevel
      * @throws TestFrameworkException
      */
     private function __construct(
         $forceGenerate = false,
         $phase = self::EXECUTION_PHASE,
         $verboseEnabled = null,
-        $mode = self::MODE_PRODUCTION
+        $debugLevel = self::DEBUG_NONE
     ) {
         $this->forceGenerate = $forceGenerate;
 
@@ -80,14 +80,14 @@ private function __construct(
 
         $this->phase = $phase;
         $this->verboseEnabled = $verboseEnabled;
-        switch ($mode) {
-            case self::MODE_DEVELOPER:
-            case self::MODE_DEFAULT:
-            case self::MODE_PRODUCTION:
-                $this->mode = $mode;
+        switch ($debugLevel) {
+            case self::LEVEL_DEVELOPER:
+            case self::LEVEL_DEFAULT:
+            case self::DEBUG_NONE:
+                $this->debugLevel = $debugLevel;
                 break;
             default:
-                $this->mode = self::MODE_DEVELOPER;
+                $this->debugLevel = self::LEVEL_DEVELOPER;
         }
     }
 
@@ -98,14 +98,14 @@ private function __construct(
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param string  $mode
+     * @param string  $debugLevel
      * @return void
      */
-    public static function create($forceGenerate, $phase, $verboseEnabled, $mode)
+    public static function create($forceGenerate, $phase, $verboseEnabled, $debugLevel)
     {
         if (self::$MFTF_APPLICATION_CONTEXT == null) {
             self::$MFTF_APPLICATION_CONTEXT =
-                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $mode);
+                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debugLevel);
         }
     }
 
@@ -148,13 +148,13 @@ public function verboseEnabled()
     }
 
     /**
-     * Returns a string which indicates the debug mode of mftf execution.
+     * Returns a string which indicates the debug level of mftf execution.
      *
      * @return string
      */
-    public function getMode()
+    public function getDebugLevel()
     {
-        return $this->mode ?? getenv('MFTF_DEBUG');
+        return $this->debugLevel ?? getenv('MFTF_DEBUG');
     }
 
     /**
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index c40db4029..2f2cb8ad3 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -157,7 +157,7 @@ protected function readFiles($fileList)
                 } else {
                     $configMerger->merge($content);
                 }
-                if (MftfApplicationConfig::getConfig()->getMode() === MftfApplicationConfig::MODE_DEVELOPER) {
+                if (MftfApplicationConfig::getConfig()->getDebugLevel() === MftfApplicationConfig::LEVEL_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index d86f26da1..6fbc2fdfc 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -24,7 +24,7 @@ public function readFiles($fileList)
         $exceptionCollector = new ExceptionCollector();
         /** @var \Magento\FunctionalTestingFramework\Test\Config\Dom $configMerger */
         $configMerger = null;
-        $mode = MftfApplicationConfig::getConfig()->getMode();
+        $debugLevel = MftfApplicationConfig::getConfig()->getDebugLevel();
         foreach ($fileList as $key => $content) {
             //check if file is empty and continue to next if it is
             if (!parent::verifyFileEmpty($content, $fileList->getFilename())) {
@@ -42,7 +42,7 @@ public function readFiles($fileList)
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
                  // run per file validation with generate:tests -d
-                if ($mode == MftfApplicationConfig::MODE_DEVELOPER) {
+                if ($debugLevel == MftfApplicationConfig::LEVEL_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -52,7 +52,7 @@ public function readFiles($fileList)
         $exceptionCollector->throwException();
 
          //run validation on merged file with generate:tests
-        if ($mode == MftfApplicationConfig::MODE_DEFAULT) {
+        if ($debugLevel == MftfApplicationConfig::LEVEL_DEFAULT) {
             $this->validateSchema($configMerger);
         }
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
index db747d160..b578ba26a 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
@@ -67,7 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::MODE_PRODUCTION
+            MftfApplicationConfig::DEBUG_NONE
         );
 
         $allActionGroups = ActionGroupObjectHandler::getInstance()->getAllObjects();
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index e97ecdbaf..f4314b4f0 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -55,7 +55,7 @@ protected function configure()
                 'debug',
                 'd',
                 InputOption::VALUE_OPTIONAL,
-                'Run extra validation when generating tests. Use option \'off\' to turn off debugging -- 
+                'Run extra validation when generating tests. Use option \'none\' to turn off debugging -- 
                  added for backward compatibility, will be removed in the next MAJOR release',
                 'default'
             );
@@ -80,7 +80,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $json = $input->getOption('tests');
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
-        $mode = $input->getOption('debug');
+        $debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
         $remove = $input->getOption('remove');
         $verbose = $output->isVerbose();
 
@@ -97,10 +97,10 @@ protected function execute(InputInterface $input, OutputInterface $output)
         // Remove previous GENERATED_DIR if --remove option is used
         if ($remove) {
             $this->removeGeneratedDirectory($output, $verbose ||
-                ($mode !== MftfApplicationConfig::MODE_PRODUCTION));
+                ($debug !== MftfApplicationConfig::DEBUG_NONE));
         }
 
-        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $mode, $verbose);
+        $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $verbose);
 
         // create our manifest file here
         $testManifest = TestManifestFactory::makeManifest($config, $testConfiguration['suites']);
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
index 82294307d..e2023d883 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
@@ -75,7 +75,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
                 ]),
                 '--force' => $force,
                 '--remove' => $remove,
-                '--debug' => MftfApplicationConfig::MODE_PRODUCTION
+                '--debug' => MftfApplicationConfig::DEBUG_NONE
             ];
             $command->run(new ArrayInput($args), $output);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index b39419adc..6f63f82ca 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -72,7 +72,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::MODE_PRODUCTION
+            MftfApplicationConfig::DEBUG_NONE
         );
 
         $testConfiguration = $this->getFailedTestList();
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index ef88fdabc..df81debec 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -77,7 +77,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::MODE_PRODUCTION
+            MftfApplicationConfig::DEBUG_NONE
         );
 
         if (!$skipGeneration) {
diff --git a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
index 6813e4e89..30db1fa78 100644
--- a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
+++ b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
@@ -75,7 +75,7 @@ public function execute(InputInterface $input)
             true,
             MftfApplicationConfig::UNIT_TEST_PHASE,
             false,
-            MftfApplicationConfig::MODE_PRODUCTION
+            MftfApplicationConfig::DEBUG_NONE
         );
 
         ModuleResolver::getInstance()->getModulesPath();

From 81414c8d9b8d4c2ceedabe6397a566945395e67e Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 21 May 2019 09:32:19 -0500
Subject: [PATCH 43/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 dev/tests/_bootstrap.php                                  | 2 +-
 .../functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml    | 2 +-
 .../Config/MftfApplicationConfig.php                      | 8 ++++----
 .../Console/GenerateDocsCommand.php                       | 2 +-
 .../Console/GenerateTestsCommand.php                      | 2 +-
 .../FunctionalTestingFramework/Console/RunTestCommand.php | 2 +-
 .../Console/RunTestFailedCommand.php                      | 2 +-
 .../Console/RunTestGroupCommand.php                       | 2 +-
 .../StaticCheck/TestDependencyCheck.php                   | 2 +-
 9 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php
index 2b6fc0934..d9a4208b2 100644
--- a/dev/tests/_bootstrap.php
+++ b/dev/tests/_bootstrap.php
@@ -32,7 +32,7 @@
     true,
     \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::UNIT_TEST_PHASE,
     true,
-    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::DEBUG_NONE
+    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::LEVEL_NONE
 );
 
 // Load needed framework env params
diff --git a/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml b/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml
index eceeb3f58..1d97adce4 100644
--- a/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml
+++ b/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml
@@ -19,6 +19,6 @@
         </annotations>
 
         <amOnPage stepKey="openMFTFDevDocPage" url="{{MFTFDocPage.url}}" />
-        <see stepKey="verifyPageIntroText" selector="{{contentSection.pageIntro}}" userInput="Introduction to the Magento Functional Testing Framework" />
+        <see stepKey="verifyPageIntroText" selector="{{contentSection.pageIntro}}" userInput="Introduction to the Magento Functional Testing Framework" skipReadiness=""/>
     </test>
 </tests>
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index 64c4ee7aa..29278f761 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -19,8 +19,8 @@ class MftfApplicationConfig
      */
     const LEVEL_DEFAULT = "default";
     const LEVEL_DEVELOPER = "developer";
-    const DEBUG_NONE = "none";
-    const MFTF_DEBUG_LEVEL = [self::LEVEL_DEFAULT, self::LEVEL_DEVELOPER, self::DEBUG_NONE];
+    const LEVEL_NONE = "none";
+    const MFTF_DEBUG_LEVEL = [self::LEVEL_DEFAULT, self::LEVEL_DEVELOPER, self::LEVEL_NONE];
 
     /**
      * Determines whether the user has specified a force option for generation
@@ -70,7 +70,7 @@ private function __construct(
         $forceGenerate = false,
         $phase = self::EXECUTION_PHASE,
         $verboseEnabled = null,
-        $debugLevel = self::DEBUG_NONE
+        $debugLevel = self::LEVEL_NONE
     ) {
         $this->forceGenerate = $forceGenerate;
 
@@ -83,7 +83,7 @@ private function __construct(
         switch ($debugLevel) {
             case self::LEVEL_DEVELOPER:
             case self::LEVEL_DEFAULT:
-            case self::DEBUG_NONE:
+            case self::LEVEL_NONE:
                 $this->debugLevel = $debugLevel;
                 break;
             default:
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
index b578ba26a..4ed2b6b29 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
@@ -67,7 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::DEBUG_NONE
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         $allActionGroups = ActionGroupObjectHandler::getInstance()->getAllObjects();
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index f4314b4f0..bc8c76142 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -97,7 +97,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         // Remove previous GENERATED_DIR if --remove option is used
         if ($remove) {
             $this->removeGeneratedDirectory($output, $verbose ||
-                ($debug !== MftfApplicationConfig::DEBUG_NONE));
+                ($debug !== MftfApplicationConfig::LEVEL_NONE));
         }
 
         $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $verbose);
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
index e2023d883..bd9b4b99b 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
@@ -75,7 +75,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
                 ]),
                 '--force' => $force,
                 '--remove' => $remove,
-                '--debug' => MftfApplicationConfig::DEBUG_NONE
+                '--debug' => MftfApplicationConfig::LEVEL_NONE
             ];
             $command->run(new ArrayInput($args), $output);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index 6f63f82ca..899fa8312 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -72,7 +72,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::DEBUG_NONE
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         $testConfiguration = $this->getFailedTestList();
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index df81debec..174fd8130 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -77,7 +77,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::DEBUG_NONE
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         if (!$skipGeneration) {
diff --git a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
index 30db1fa78..1a149a657 100644
--- a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
+++ b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
@@ -75,7 +75,7 @@ public function execute(InputInterface $input)
             true,
             MftfApplicationConfig::UNIT_TEST_PHASE,
             false,
-            MftfApplicationConfig::DEBUG_NONE
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         ModuleResolver::getInstance()->getModulesPath();

From a3f25bd87bfdf69787ba52e0089e5429a94bc169 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 21 May 2019 09:53:32 -0500
Subject: [PATCH 44/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml b/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml
index 1d97adce4..eceeb3f58 100644
--- a/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml
+++ b/dev/tests/functional/tests/MFTF/DevDocs/Test/DevDocsTest.xml
@@ -19,6 +19,6 @@
         </annotations>
 
         <amOnPage stepKey="openMFTFDevDocPage" url="{{MFTFDocPage.url}}" />
-        <see stepKey="verifyPageIntroText" selector="{{contentSection.pageIntro}}" userInput="Introduction to the Magento Functional Testing Framework" skipReadiness=""/>
+        <see stepKey="verifyPageIntroText" selector="{{contentSection.pageIntro}}" userInput="Introduction to the Magento Functional Testing Framework" />
     </test>
 </tests>

From 958e9c36618d1c61ae059734e3c3eab9ae950183 Mon Sep 17 00:00:00 2001
From: soumyau <sunnikri@adobe.com>
Date: Tue, 21 May 2019 10:20:19 -0500
Subject: [PATCH 45/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files' (#349)

- added input options to debug argument
---
 dev/tests/_bootstrap.php                      |  2 +-
 .../Resources/BasicFunctionalTest.txt         |  4 +-
 .../ActionGroup/BasicActionGroup.xml          |  4 --
 .../TestModule/Test/BasicFunctionalTest.xml   |  2 +-
 .../Tests/ActionGroupGenerationTest.php       | 11 -----
 .../Tests/SchemaValidationTest.php            |  2 +-
 .../Config/MftfApplicationConfig.php          | 43 +++++++++++++------
 .../Config/Reader/Filesystem.php              | 12 ++++--
 .../Config/Reader/MftfFilesystem.php          | 10 +++--
 .../Console/GenerateDocsCommand.php           |  2 +-
 .../Console/GenerateTestsCommand.php          | 20 +++++----
 .../Console/RunTestCommand.php                |  4 +-
 .../Console/RunTestFailedCommand.php          |  2 +-
 .../Console/RunTestGroupCommand.php           |  2 +-
 .../StaticCheck/TestDependencyCheck.php       |  2 +-
 .../Test/etc/Actions/commonAttributes.xsd     |  7 ---
 .../Test/etc/mergedTestSchema.xsd             |  8 +++-
 .../Util/Logger/MftfLogger.php                | 19 +++++++-
 18 files changed, 92 insertions(+), 64 deletions(-)

diff --git a/dev/tests/_bootstrap.php b/dev/tests/_bootstrap.php
index 31b4462f6..d9a4208b2 100644
--- a/dev/tests/_bootstrap.php
+++ b/dev/tests/_bootstrap.php
@@ -32,7 +32,7 @@
     true,
     \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::UNIT_TEST_PHASE,
     true,
-    false
+    \Magento\FunctionalTestingFramework\Config\MftfApplicationConfig::LEVEL_NONE
 );
 
 // Load needed framework env params
diff --git a/dev/tests/verification/Resources/BasicFunctionalTest.txt b/dev/tests/verification/Resources/BasicFunctionalTest.txt
index 9970e81ec..fd5f0f165 100644
--- a/dev/tests/verification/Resources/BasicFunctionalTest.txt
+++ b/dev/tests/verification/Resources/BasicFunctionalTest.txt
@@ -60,9 +60,7 @@ class BasicFunctionalTestCest
 	{
 		$I->comment("");
 		$I->comment("");
-		$I->skipReadinessCheck(true);
-		$I->comment("skipReadiness");
-		$I->skipReadinessCheck(false);
+		$I->comment("seeComment");
 		$someVarDefinition = $I->grabValueFrom();
 		$I->acceptPopup();
 		$I->amOnPage("/test/url");
diff --git a/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml b/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml
index 245447f7d..45ccdbd24 100644
--- a/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml
+++ b/dev/tests/verification/TestModule/ActionGroup/BasicActionGroup.xml
@@ -115,10 +115,6 @@
         <see selector=".selector" userInput="{{sameStepKeyAsArg}}" stepKey="arg1" />
     </actionGroup>
 
-    <actionGroup name="actionGroupWithSkipReadinessActions">
-        <comment userInput="ActionGroupSkipReadiness" stepKey="skip" skipReadiness="true"/>
-    </actionGroup>
-
     <actionGroup name="actionGroupWithSectionAndData">
         <arguments>
             <argument name="content" type="string"/>
diff --git a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
index d120e5c31..bdeb4e0c6 100644
--- a/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
+++ b/dev/tests/verification/TestModule/Test/BasicFunctionalTest.xml
@@ -24,7 +24,7 @@
         </after>
         <comment stepKey="basicCommentWithNoData" userInput="{{emptyData.noData}}"/>
         <comment stepKey="basicCommentWithDefinitelyNoData" userInput="{{emptyData.definitelyNoData}}"/>
-        <comment stepKey="ReadinessCheckSkipped" userInput="skipReadiness" skipReadiness="true"/>
+        <comment stepKey="basicCommentWithData" userInput="seeComment"/>
         <grabValueFrom stepKey="someVarDefinition"/>
         <acceptPopup stepKey="acceptPopupKey1"/>
         <amOnPage stepKey="amOnPageKey1" url="/test/url"/>
diff --git a/dev/tests/verification/Tests/ActionGroupGenerationTest.php b/dev/tests/verification/Tests/ActionGroupGenerationTest.php
index dec8c6d14..613d6cb72 100644
--- a/dev/tests/verification/Tests/ActionGroupGenerationTest.php
+++ b/dev/tests/verification/Tests/ActionGroupGenerationTest.php
@@ -185,17 +185,6 @@ public function testActionGroupWithArgContainingStepKey()
         $this->generateAndCompareTest('ActionGroupContainsStepKeyInArgText');
     }
 
-    /**
-     * Test an action group with an arg containing stepKey text
-     *
-     * @throws \Exception
-     * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException
-     */
-    public function testActionGroupWithSkipReadiness()
-    {
-        $this->generateAndCompareTest('ActionGroupSkipReadiness');
-    }
-
     /**
      * Test an action group with an arg containing stepKey text
      *
diff --git a/dev/tests/verification/Tests/SchemaValidationTest.php b/dev/tests/verification/Tests/SchemaValidationTest.php
index 86828e9a5..92442bd37 100644
--- a/dev/tests/verification/Tests/SchemaValidationTest.php
+++ b/dev/tests/verification/Tests/SchemaValidationTest.php
@@ -19,7 +19,7 @@ class SchemaValidationTest extends MftfTestCase
      */
     public function testInvalidTestSchema()
     {
-        AspectMock::double(MftfApplicationConfig::class, ['debugEnabled' => true]);
+        AspectMock::double(MftfApplicationConfig::class, ['getDebugLevel' => MftfApplicationConfig::LEVEL_DEVELOPER]);
         $testFile = ['testFile.xml' => "<tests><test name='testName'><annotations>a</annotations></test></tests>"];
         $expectedError = TESTS_MODULE_PATH .
             DIRECTORY_SEPARATOR .
diff --git a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
index eeabdf117..29278f761 100644
--- a/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
+++ b/src/Magento/FunctionalTestingFramework/Config/MftfApplicationConfig.php
@@ -14,6 +14,14 @@ class MftfApplicationConfig
     const UNIT_TEST_PHASE = "testing";
     const MFTF_PHASES = [self::GENERATION_PHASE, self::EXECUTION_PHASE, self::UNIT_TEST_PHASE];
 
+    /**
+     * Mftf debug levels
+     */
+    const LEVEL_DEFAULT = "default";
+    const LEVEL_DEVELOPER = "developer";
+    const LEVEL_NONE = "none";
+    const MFTF_DEBUG_LEVEL = [self::LEVEL_DEFAULT, self::LEVEL_DEVELOPER, self::LEVEL_NONE];
+
     /**
      * Determines whether the user has specified a force option for generation
      *
@@ -36,11 +44,11 @@ class MftfApplicationConfig
     private $verboseEnabled;
 
     /**
-     * Determines whether the user would like to execute mftf in a verbose run.
+     * String which identifies the current debug level of mftf execution
      *
-     * @var boolean
+     * @var string
      */
-    private $debugEnabled;
+    private $debugLevel;
 
     /**
      * MftfApplicationConfig Singelton Instance
@@ -55,14 +63,14 @@ class MftfApplicationConfig
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param boolean $debugEnabled
+     * @param string  $debugLevel
      * @throws TestFrameworkException
      */
     private function __construct(
         $forceGenerate = false,
         $phase = self::EXECUTION_PHASE,
         $verboseEnabled = null,
-        $debugEnabled = null
+        $debugLevel = self::LEVEL_NONE
     ) {
         $this->forceGenerate = $forceGenerate;
 
@@ -72,7 +80,15 @@ private function __construct(
 
         $this->phase = $phase;
         $this->verboseEnabled = $verboseEnabled;
-        $this->debugEnabled = $debugEnabled;
+        switch ($debugLevel) {
+            case self::LEVEL_DEVELOPER:
+            case self::LEVEL_DEFAULT:
+            case self::LEVEL_NONE:
+                $this->debugLevel = $debugLevel;
+                break;
+            default:
+                $this->debugLevel = self::LEVEL_DEVELOPER;
+        }
     }
 
     /**
@@ -82,14 +98,14 @@ private function __construct(
      * @param boolean $forceGenerate
      * @param string  $phase
      * @param boolean $verboseEnabled
-     * @param boolean $debugEnabled
+     * @param string  $debugLevel
      * @return void
      */
-    public static function create($forceGenerate, $phase, $verboseEnabled, $debugEnabled)
+    public static function create($forceGenerate, $phase, $verboseEnabled, $debugLevel)
     {
         if (self::$MFTF_APPLICATION_CONTEXT == null) {
             self::$MFTF_APPLICATION_CONTEXT =
-                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debugEnabled);
+                new MftfApplicationConfig($forceGenerate, $phase, $verboseEnabled, $debugLevel);
         }
     }
 
@@ -132,14 +148,13 @@ public function verboseEnabled()
     }
 
     /**
-     * Returns a boolean indicating whether the user has indicated a debug run, which will lengthy validation
-     * with some extra error messaging to be run
+     * Returns a string which indicates the debug level of mftf execution.
      *
-     * @return boolean
+     * @return string
      */
-    public function debugEnabled()
+    public function getDebugLevel()
     {
-        return $this->debugEnabled ?? getenv('MFTF_DEBUG');
+        return $this->debugLevel ?? getenv('MFTF_DEBUG');
     }
 
     /**
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
index 4c3aa581b..2f2cb8ad3 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/Filesystem.php
@@ -157,7 +157,7 @@ protected function readFiles($fileList)
                 } else {
                     $configMerger->merge($content);
                 }
-                if (MftfApplicationConfig::getConfig()->debugEnabled()) {
+                if (MftfApplicationConfig::getConfig()->getDebugLevel() === MftfApplicationConfig::LEVEL_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -231,8 +231,14 @@ protected function validateSchema($configMerger, $filename = null)
         if ($this->validationState->isValidationRequired()) {
             $errors = [];
             if ($configMerger && !$configMerger->validate($this->schemaFile, $errors)) {
-                $message = $filename ? $filename . PHP_EOL . "Invalid Document \n" : PHP_EOL . "Invalid Document \n";
-                throw new \Exception($message . implode("\n", $errors));
+                foreach ($errors as $error) {
+                    $error = str_replace(PHP_EOL, "", $error);
+                    LoggingUtil::getInstance()->getLogger(Filesystem::class)->criticalFailure(
+                        "Schema validation error ",
+                        ($filename ? [ "file"=> $filename, "error" => $error]: ["error" => $error])
+                    );
+                }
+                throw new \Exception("Schema validation errors found in xml file(s)" . $filename);
             }
         }
     }
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 728c4cb3a..6fbc2fdfc 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -24,6 +24,7 @@ public function readFiles($fileList)
         $exceptionCollector = new ExceptionCollector();
         /** @var \Magento\FunctionalTestingFramework\Test\Config\Dom $configMerger */
         $configMerger = null;
+        $debugLevel = MftfApplicationConfig::getConfig()->getDebugLevel();
         foreach ($fileList as $key => $content) {
             //check if file is empty and continue to next if it is
             if (!parent::verifyFileEmpty($content, $fileList->getFilename())) {
@@ -40,7 +41,8 @@ public function readFiles($fileList)
                 } else {
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
-                if (MftfApplicationConfig::getConfig()->debugEnabled()) {
+                 // run per file validation with generate:tests -d
+                if ($debugLevel == MftfApplicationConfig::LEVEL_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -48,8 +50,10 @@ public function readFiles($fileList)
             }
         }
         $exceptionCollector->throwException();
-        if ($fileList->valid()) {
-            $this->validateSchema($configMerger, $fileList->getFilename());
+
+         //run validation on merged file with generate:tests
+        if ($debugLevel == MftfApplicationConfig::LEVEL_DEFAULT) {
+            $this->validateSchema($configMerger);
         }
 
         $output = [];
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
index db77bc74f..4ed2b6b29 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateDocsCommand.php
@@ -67,7 +67,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            false
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         $allActionGroups = ActionGroupObjectHandler::getInstance()->getAllObjects();
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index fcd977060..bc8c76142 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -29,7 +29,7 @@ class GenerateTestsCommand extends BaseGenerateCommand
     protected function configure()
     {
         $this->setName('generate:tests')
-            ->setDescription('This command generates all test files and suites based on xml declarations')
+            ->setDescription('Run validation and generate all test files and suites based on xml declarations')
             ->addArgument(
                 'name',
                 InputArgument::OPTIONAL | InputArgument::IS_ARRAY,
@@ -39,7 +39,7 @@ protected function configure()
                 "force",
                 'f',
                 InputOption::VALUE_NONE,
-                'force generation of tests regardless of Magento Instance Configuration'
+                'Force generation of tests regardless of Magento Instance Configuration'
             )->addOption(
                 'time',
                 'i',
@@ -54,8 +54,10 @@ protected function configure()
             )->addOption(
                 'debug',
                 'd',
-                InputOption::VALUE_NONE,
-                'run extra validation when generating tests'
+                InputOption::VALUE_OPTIONAL,
+                'Run extra validation when generating tests. Use option \'none\' to turn off debugging -- 
+                 added for backward compatibility, will be removed in the next MAJOR release',
+                'default'
             );
 
         parent::configure();
@@ -78,9 +80,8 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $json = $input->getOption('tests');
         $force = $input->getOption('force');
         $time = $input->getOption('time') * 60 * 1000; // convert from minutes to milliseconds
-        $debug = $input->getOption('debug');
+        $debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
         $remove = $input->getOption('remove');
-
         $verbose = $output->isVerbose();
 
         if ($json !== null && !json_decode($json)) {
@@ -95,7 +96,8 @@ protected function execute(InputInterface $input, OutputInterface $output)
 
         // Remove previous GENERATED_DIR if --remove option is used
         if ($remove) {
-            $this->removeGeneratedDirectory($output, $verbose || $debug);
+            $this->removeGeneratedDirectory($output, $verbose ||
+                ($debug !== MftfApplicationConfig::LEVEL_NONE));
         }
 
         $testConfiguration = $this->createTestConfiguration($json, $tests, $force, $debug, $verbose);
@@ -124,13 +126,13 @@ protected function execute(InputInterface $input, OutputInterface $output)
      * @param string  $json
      * @param array   $tests
      * @param boolean $force
-     * @param boolean $debug
+     * @param string  $debug
      * @param boolean $verbose
      * @return array
      * @throws \Magento\FunctionalTestingFramework\Exceptions\TestReferenceException
      * @throws \Magento\FunctionalTestingFramework\Exceptions\XmlException
      */
-    private function createTestConfiguration($json, array $tests, bool $force, bool $debug, bool $verbose)
+    private function createTestConfiguration($json, array $tests, bool $force, $debug, bool $verbose)
     {
         // set our application configuration so we can references the user options in our framework
         MftfApplicationConfig::create(
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
index 33a29b2e8..bd9b4b99b 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
@@ -7,6 +7,7 @@
 
 namespace Magento\FunctionalTestingFramework\Console;
 
+use Magento\FunctionalTestingFramework\Config\MftfApplicationConfig;
 use Symfony\Component\Console\Input\ArrayInput;
 use Symfony\Component\Console\Input\InputArgument;
 use Symfony\Component\Console\Input\InputInterface;
@@ -73,7 +74,8 @@ protected function execute(InputInterface $input, OutputInterface $output): int
                     'suites' => null
                 ]),
                 '--force' => $force,
-                '--remove' => $remove
+                '--remove' => $remove,
+                '--debug' => MftfApplicationConfig::LEVEL_NONE
             ];
             $command->run(new ArrayInput($args), $output);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index 8257162b9..899fa8312 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -72,7 +72,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            false
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         $testConfiguration = $this->getFailedTestList();
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index e895fb66a..174fd8130 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -77,7 +77,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            false
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         if (!$skipGeneration) {
diff --git a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
index e0b3cefd6..1a149a657 100644
--- a/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
+++ b/src/Magento/FunctionalTestingFramework/StaticCheck/TestDependencyCheck.php
@@ -75,7 +75,7 @@ public function execute(InputInterface $input)
             true,
             MftfApplicationConfig::UNIT_TEST_PHASE,
             false,
-            false
+            MftfApplicationConfig::LEVEL_NONE
         );
 
         ModuleResolver::getInstance()->getModulesPath();
diff --git a/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd b/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
index 22c05ea7d..c39d779ed 100644
--- a/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
+++ b/src/Magento/FunctionalTestingFramework/Test/etc/Actions/commonAttributes.xsd
@@ -31,13 +31,6 @@
             </xs:annotation>
         </xs:attribute>
 
-        <xs:attribute type="xs:boolean" name="skipReadiness">
-            <xs:annotation>
-                <xs:documentation>
-                    Flag for skipping readiness check
-                </xs:documentation>
-            </xs:annotation>
-        </xs:attribute>
     </xs:attributeGroup>
 
     <xs:attribute type="xs:string" name="userInput" id="userInput">
diff --git a/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd b/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd
index b663bb99d..fe45945ee 100644
--- a/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd
+++ b/src/Magento/FunctionalTestingFramework/Test/etc/mergedTestSchema.xsd
@@ -129,7 +129,13 @@
                 </xs:documentation>
             </xs:annotation>
         </xs:attribute>
+        <xs:attribute type="xs:boolean" name="skipReadiness" use="prohibited">
+            <xs:annotation>
+                <xs:documentation>
+                    Flag for skipping readiness check.
+                </xs:documentation>
+            </xs:annotation>
+        </xs:attribute>
         <xs:attributeGroup ref="commonActionAttributes"/>
-        <xs:attribute type="xs:boolean" name="skipReadiness" use="prohibited"/>
     </xs:complexType>
 </xs:schema>
diff --git a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
index 5844858a0..0a52f6b6b 100644
--- a/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
+++ b/src/Magento/FunctionalTestingFramework/Util/Logger/MftfLogger.php
@@ -13,7 +13,7 @@
 class MftfLogger extends Logger
 {
     /**
-     * Prints a deprecation warning, as well as adding a log at the WARNING level.
+     * Prints a deprecation warning, as well as adds a log at the WARNING level.
      *
      * @param  string $message The log message.
      * @param  array  $context The log context.
@@ -28,4 +28,21 @@ public function deprecation($message, array $context = [])
         }
         parent::warning($message, $context);
     }
+
+    /**
+     * Prints a critical failure, as well as adds a log at the CRITICAL level.
+     *
+     * @param  string $message The log message.
+     * @param  array  $context The log context.
+     * @return void
+     */
+    public function criticalFailure($message, array $context = [])
+    {
+        $message = "FAILURE: " . $message;
+        // Suppress print during unit testing
+        if (MftfApplicationConfig::getConfig()->getPhase() !== MftfApplicationConfig::UNIT_TEST_PHASE) {
+            print ($message . implode("\n", $context) . "\n");
+        }
+        parent::critical($message, $context);
+    }
 }

From 016332c53577d980f4cf2d973cceef89beebe5a1 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 10:44:52 -0500
Subject: [PATCH 46/63] Updated title.

---
 docs/versioning.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/versioning.md b/docs/versioning.md
index 6c9104230..c601a4a57 100644
--- a/docs/versioning.md
+++ b/docs/versioning.md
@@ -56,7 +56,7 @@ Major version **X** MUST be incremented for a release that introduces backward i
 A major release can also include minor and patch level changes.
 You must reset the patch and minor version to 0 when you change the major version.
 
-## MFTF version table
+## Magento2 - MFTF versions
 
 This table lists the version of the MFTF that was released with a particular version of Magento.
 

From 41f8039807cabd52a235cf47d59b1e8af3ecbb71 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 12:32:47 -0500
Subject: [PATCH 47/63] Update docs/versioning.md

Co-Authored-By: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
---
 docs/versioning.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/versioning.md b/docs/versioning.md
index c601a4a57..5497e339f 100644
--- a/docs/versioning.md
+++ b/docs/versioning.md
@@ -60,7 +60,7 @@ You must reset the patch and minor version to 0 when you change the major versio
 
 This table lists the version of the MFTF that was released with a particular version of Magento.
 
-|||
+|Magento version| MFTF  version|
 |---|---|
 | Magento | MFTF |
 | 2.3.1 | 2.3.13 |

From 4929a593e1c38301303895274cbcf66383664d23 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 12:32:55 -0500
Subject: [PATCH 48/63] Update docs/versioning.md

Co-Authored-By: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
---
 docs/versioning.md | 1 -
 1 file changed, 1 deletion(-)

diff --git a/docs/versioning.md b/docs/versioning.md
index 5497e339f..b40e913f9 100644
--- a/docs/versioning.md
+++ b/docs/versioning.md
@@ -62,7 +62,6 @@ This table lists the version of the MFTF that was released with a particular ver
 
 |Magento version| MFTF  version|
 |---|---|
-| Magento | MFTF |
 | 2.3.1 | 2.3.13 |
 | 2.3.0 | 2.3.9 |
 | 2.2.8 | 2.3.13 |

From 645eb051abf7ebfd24c508e527cfbc745186ea3a Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 12:33:12 -0500
Subject: [PATCH 49/63] Update docs/versioning.md

Co-Authored-By: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
---
 docs/versioning.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/versioning.md b/docs/versioning.md
index b40e913f9..660b88a40 100644
--- a/docs/versioning.md
+++ b/docs/versioning.md
@@ -56,7 +56,7 @@ Major version **X** MUST be incremented for a release that introduces backward i
 A major release can also include minor and patch level changes.
 You must reset the patch and minor version to 0 when you change the major version.
 
-## Magento2 - MFTF versions
+## Magento 2  compatibility
 
 This table lists the version of the MFTF that was released with a particular version of Magento.
 

From 8c7c139d414d1ec322260cc2510d9f0feaac8041 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 12:35:31 -0500
Subject: [PATCH 50/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index f7e6a8576..cab4787c7 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -1,6 +1,6 @@
 # How to contribute to MFTF docs
 
-We welcome contributions to the MFTF documentation, which is kept within the /docs/ folder in this repository.
+We welcome contributions to the MFTF documentation, which is kept within the `docs/` folder in this repository.
 This page describes the submitting process and serves as a template for a properly written content.
 The contribution workflow is the same as submitting code.
 

From f03a564a9bee7c77982204905bb8ce9c325ebced Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 12:35:42 -0500
Subject: [PATCH 51/63] Update .github/DOCUMENTATION_TEMPLATE.md

Co-Authored-By: Dmitry Shevtsov <12731225+dshevtsov@users.noreply.github.com>
---
 .github/DOCUMENTATION_TEMPLATE.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/DOCUMENTATION_TEMPLATE.md b/.github/DOCUMENTATION_TEMPLATE.md
index cab4787c7..222ff23be 100644
--- a/.github/DOCUMENTATION_TEMPLATE.md
+++ b/.github/DOCUMENTATION_TEMPLATE.md
@@ -6,7 +6,7 @@ The contribution workflow is the same as submitting code.
 
 1. Create a branch from the `develop` branch in the [MFTF repository][].
 1. Make edits/additions/deletions as needed. Read the [Basic Template][] for tips on how to write with Markdown.
-1. Submit your PR to the `develop` branch.
+1. Submit your pull request to the `develop` branch.
 
 Once submitted, a member of the documentation team will review and merge it.
 We will inform you if it needs any additional processing.

From 2c9b92cc5e4ed4b335d6372f7c2cc5328cffa490 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 21 May 2019 13:17:32 -0500
Subject: [PATCH 52/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

Added documentation
---
 docs/commands/mftf.md | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 4d91bb37f..545383deb 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -105,7 +105,7 @@ vendor/bin/mftf build:project --MAGENTO_BASE_URL=http://magento.local/ --MAGENTO
 
 #### Description
 
-Generate PHP code from the tests defined in XML files.
+Perform XML schema validation and generate PHP code from the tests defined in XML files.
 The path is set in the `TESTS_MODULE_PATH` [configuration] parameter.
 
 #### Usage
@@ -122,9 +122,22 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
-| `--debug` | Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This parameter takes extra processing time. Use it after test generation has failed once. |
+| `--debug=[<developer> or <none>]`| Set debug level to `default` when the option is not specified, to `developer` with `--debug developer` or `--debug`, to `none` with `--debug none` |   |
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
+#### Debugging levels
+
+You can run generate:tests with any of the following debug levels. XML schema validation errors will be logged as CRITICAL failures.
+
+| Debug level | Description |
+| ---| --- |
+|`default` `[generate:tests]` ` | perform XML schema validation on merged files. Does not indicate the file name where error is encountered. Use `developer` level for enhanced debugging. |
+|`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once. |
+|`none` `[generate:tests --debug none]`|  skip debugging during test generation. Added for backward compatibility, will be removed with the next MAJOR release|
+
+
+
+
 #### Examples of the JSON configuration
 
 The configuration to generate a single test with no suites:

From 2e92c25167eb4607ec6b288a01f57ef3b89e91e6 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 21 May 2019 13:20:40 -0500
Subject: [PATCH 53/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

---
 docs/commands/mftf.md | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 545383deb..73a44e19d 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -135,9 +135,6 @@ You can run generate:tests with any of the following debug levels. XML schema va
 |`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once. |
 |`none` `[generate:tests --debug none]`|  skip debugging during test generation. Added for backward compatibility, will be removed with the next MAJOR release|
 
-
-
-
 #### Examples of the JSON configuration
 
 The configuration to generate a single test with no suites:

From 5e90be5957683bb4fff6b63197c0311e44179640 Mon Sep 17 00:00:00 2001
From: Donald Booth <dobooth@adobe.com>
Date: Tue, 21 May 2019 15:57:02 -0500
Subject: [PATCH 54/63] Grammar and formatting.

---
 docs/commands/mftf.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 73a44e19d..3027d1929 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -127,13 +127,13 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 
 #### Debugging levels
 
-You can run generate:tests with any of the following debug levels. XML schema validation errors will be logged as CRITICAL failures.
+You can run `generate:tests` with any of the following debug levels. XML schema validation errors will be logged as CRITICAL failures.
 
 | Debug level | Description |
 | ---| --- |
-|`default` `[generate:tests]` ` | perform XML schema validation on merged files. Does not indicate the file name where error is encountered. Use `developer` level for enhanced debugging. |
-|`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once. |
-|`none` `[generate:tests --debug none]`|  skip debugging during test generation. Added for backward compatibility, will be removed with the next MAJOR release|
+|`default` `[generate:tests]` | Perform XML schema validation on merged files. Does not indicate the file name where trhe error is encountered. Use `developer` level for enhanced debugging. |
+|`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. Note: This option takes extra processing time. Use it after test generation has failed once. |
+|`none` `[generate:tests --debug none]`|  Skip debugging during test generation. Added for backward compatibility, it will be removed with the next MAJOR release.|
 
 #### Examples of the JSON configuration
 

From 9150d91bd46422800271a7adced82caa99b2e69f Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 21 May 2019 16:07:06 -0500
Subject: [PATCH 55/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

fixed spell error
---
 docs/commands/mftf.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 3027d1929..12e1ead28 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -131,7 +131,7 @@ You can run `generate:tests` with any of the following debug levels. XML schema
 
 | Debug level | Description |
 | ---| --- |
-|`default` `[generate:tests]` | Perform XML schema validation on merged files. Does not indicate the file name where trhe error is encountered. Use `developer` level for enhanced debugging. |
+|`default` `[generate:tests]` | Perform XML schema validation on merged files. Does not indicate the file name where the error is encountered. Use `developer` level for enhanced debugging. |
 |`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. Note: This option takes extra processing time. Use it after test generation has failed once. |
 |`none` `[generate:tests --debug none]`|  Skip debugging during test generation. Added for backward compatibility, it will be removed with the next MAJOR release.|
 

From e11a5d020f566cafcc927910abd5e4a0428ce26d Mon Sep 17 00:00:00 2001
From: soumyau <sunnikri@adobe.com>
Date: Thu, 23 May 2019 09:58:05 -0500
Subject: [PATCH 56/63] MQE-1541: Documentation for debug changes (#353)

- Added documentation
---
 docs/commands/mftf.md | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 4d91bb37f..12e1ead28 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -105,7 +105,7 @@ vendor/bin/mftf build:project --MAGENTO_BASE_URL=http://magento.local/ --MAGENTO
 
 #### Description
 
-Generate PHP code from the tests defined in XML files.
+Perform XML schema validation and generate PHP code from the tests defined in XML files.
 The path is set in the `TESTS_MODULE_PATH` [configuration] parameter.
 
 #### Usage
@@ -122,9 +122,19 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
-| `--debug` | Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This parameter takes extra processing time. Use it after test generation has failed once. |
+| `--debug=[<developer> or <none>]`| Set debug level to `default` when the option is not specified, to `developer` with `--debug developer` or `--debug`, to `none` with `--debug none` |   |
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
+#### Debugging levels
+
+You can run `generate:tests` with any of the following debug levels. XML schema validation errors will be logged as CRITICAL failures.
+
+| Debug level | Description |
+| ---| --- |
+|`default` `[generate:tests]` | Perform XML schema validation on merged files. Does not indicate the file name where the error is encountered. Use `developer` level for enhanced debugging. |
+|`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. Note: This option takes extra processing time. Use it after test generation has failed once. |
+|`none` `[generate:tests --debug none]`|  Skip debugging during test generation. Added for backward compatibility, it will be removed with the next MAJOR release.|
+
 #### Examples of the JSON configuration
 
 The configuration to generate a single test with no suites:

From 5c1536df07e4629a5f50a852c6af7be89469aa1c Mon Sep 17 00:00:00 2001
From: Kevin Kozan <kkozan@magento.com>
Date: Fri, 24 May 2019 08:59:25 -0500
Subject: [PATCH 57/63] MQE-1570: Add release to CHANGELOG.MD

- Version bump
- Changelog Notes
---
 CHANGELOG.md  | 10 ++++++++++
 bin/mftf      |  2 +-
 composer.json |  2 +-
 composer.lock |  2 +-
 4 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index afcb80adc..de0310a4a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,6 +1,16 @@
 Magento Functional Testing Framework Changelog
 ================================================
 
+2.4.1
+-----
+* Traceability
+    * XSD Schema validation is now enabled by default in `generate:tests`
+    * `generate:tests --debug` option has been updated to include different debug levels
+        * See DevDocs for details
+
+### Fixes
+* Fixed an issue where `skipReadiness` attribute would cause false XSD Schema validation errors.
+
 2.4.0
 -----
 ### Enhancements
diff --git a/bin/mftf b/bin/mftf
index bebc55b94..207dab6ff 100755
--- a/bin/mftf
+++ b/bin/mftf
@@ -29,7 +29,7 @@ try {
 try {
     $application = new Symfony\Component\Console\Application();
     $application->setName('Magento Functional Testing Framework CLI');
-    $application->setVersion('2.4.0');
+    $application->setVersion('2.4.1');
     /** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */
     $commandList = new \Magento\FunctionalTestingFramework\Console\CommandList;
     foreach ($commandList->getCommands() as $command) {
diff --git a/composer.json b/composer.json
index 21a2670c0..d226371ea 100755
--- a/composer.json
+++ b/composer.json
@@ -2,7 +2,7 @@
     "name": "magento/magento2-functional-testing-framework",
     "description": "Magento2 Functional Testing Framework",
     "type": "library",
-    "version": "2.4.0",
+    "version": "2.4.1",
     "license": "AGPL-3.0",
     "keywords": ["magento", "automation", "functional", "testing"],
     "config": {
diff --git a/composer.lock b/composer.lock
index 2e539fcf2..c858ceff4 100644
--- a/composer.lock
+++ b/composer.lock
@@ -4,7 +4,7 @@
         "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
         "This file is @generated automatically"
     ],
-    "content-hash": "4ae7e6856deb8d93cae8f3d7c2fbb3c0",
+    "content-hash": "6b2b6a074f053fe28df2633e8bce095f",
     "packages": [
         {
             "name": "allure-framework/allure-codeception",

From c83f2e90c2d4d4151c0952c3dc924181b39792ca Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 28 May 2019 22:48:12 -0500
Subject: [PATCH 58/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

added debug levels to run:test, run:failed, run:group
---
 docs/commands/mftf.md                         | 24 +++++++++----------
 .../Config/Reader/MftfFilesystem.php          |  4 ++--
 .../Console/GenerateTestsCommand.php          |  2 +-
 .../Console/RunTestCommand.php                | 10 +++++++-
 .../Console/RunTestFailedCommand.php          | 20 +++++++++++++---
 .../Console/RunTestGroupCommand.php           | 13 ++++++++--
 6 files changed, 52 insertions(+), 21 deletions(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 12e1ead28..3b3b94a50 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -122,19 +122,9 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
-| `--debug=[<developer> or <none>]`| Set debug level to `default` when the option is not specified, to `developer` with `--debug developer` or `--debug`, to `none` with `--debug none` |   |
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files </br> <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
-#### Debugging levels
-
-You can run `generate:tests` with any of the following debug levels. XML schema validation errors will be logged as CRITICAL failures.
-
-| Debug level | Description |
-| ---| --- |
-|`default` `[generate:tests]` | Perform XML schema validation on merged files. Does not indicate the file name where the error is encountered. Use `developer` level for enhanced debugging. |
-|`developer` `[generate:tests --debug or generate:tests --debug developer]` |  Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. Note: This option takes extra processing time. Use it after test generation has failed once. |
-|`none` `[generate:tests --debug none]`|  Skip debugging during test generation. Added for backward compatibility, it will be removed with the next MAJOR release.|
-
 #### Examples of the JSON configuration
 
 The configuration to generate a single test with no suites:
@@ -305,6 +295,9 @@ vendor/bin/mftf run:group [--skip-generate|--remove] [--] <group1> [<group2>]
 | --------------------- | --------------------------------------------------------------------------------------------------------- |
 | `-k, --skip-generate` | Skips generating from the source XML. Instead, the command executes previously-generated groups of tests. |
 | `-r, --remove`        | Removes previously generated suites and tests before the actual generation and run.                       |
+| `--debug or --debug=[<none>]`| <br/>Performs schema validations on XML files </br> <br/> DEFAULT: `run:group` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test run fails because of an invalid XML schema. This option takes extra processing time. Use it after test run has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+
+
 
 #### Examples
 
@@ -327,7 +320,7 @@ Generates and executes tests by name using Codeception.
 #### Usage
 
 ```bash
-vendor/bin/mftf run:test [--skip-generate|--remove] [--] <name1> [<name2>]
+vendor/bin/mftf run:test [--skip-generate|--remove] [--] <name1> [<name2>] [--debug|
 ```
 
 #### Options
@@ -336,6 +329,7 @@ vendor/bin/mftf run:test [--skip-generate|--remove] [--] <name1> [<name2>]
 |-----------------------|-----------------------------------------------------------------------------------------------------------|
 | `-k, --skip-generate` | Skips generating from the source XML. Instead, the command executes previously-generated groups of tests. |
 | `-r, --remove`        | Remove previously generated suites and tests.                                                             |
+| `--debug or --debug=[<none>]`| <br/>Performs schema validations on XML files </br> <br/> DEFAULT: `run:test` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test run fails because of an invalid XML schema. This option takes extra processing time. Use it after test run has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
 
 #### Examples
 
@@ -357,6 +351,12 @@ For more details about `failed`, refer to [Reporting][].
 ```bash
 vendor/bin/mftf run:failed
 ```
+#### Options
+
+| Option                | Description                                                                                               |
+|-----------------------|-----------------------------------------------------------------------------------------------------------|
+| `--debug or --debug=[<none>]`| <br/>Performs schema validations on XML files </br> <br/> DEFAULT: `run:failed` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test run fails because of an invalid XML schema. This option takes extra processing time. Use it after test run has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+
 
 #### Examples
 
diff --git a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
index 6fbc2fdfc..cd075e57c 100644
--- a/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
+++ b/src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php
@@ -42,7 +42,7 @@ public function readFiles($fileList)
                     $configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
                 }
                  // run per file validation with generate:tests -d
-                if ($debugLevel == MftfApplicationConfig::LEVEL_DEVELOPER) {
+                if ($debugLevel === MftfApplicationConfig::LEVEL_DEVELOPER) {
                     $this->validateSchema($configMerger, $fileList->getFilename());
                 }
             } catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -52,7 +52,7 @@ public function readFiles($fileList)
         $exceptionCollector->throwException();
 
          //run validation on merged file with generate:tests
-        if ($debugLevel == MftfApplicationConfig::LEVEL_DEFAULT) {
+        if ($debugLevel === MftfApplicationConfig::LEVEL_DEFAULT) {
             $this->validateSchema($configMerger);
         }
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
index bc8c76142..23651b492 100644
--- a/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php
@@ -57,7 +57,7 @@ protected function configure()
                 InputOption::VALUE_OPTIONAL,
                 'Run extra validation when generating tests. Use option \'none\' to turn off debugging -- 
                  added for backward compatibility, will be removed in the next MAJOR release',
-                'default'
+                MftfApplicationConfig::LEVEL_DEFAULT
             );
 
         parent::configure();
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
index bd9b4b99b..76a8bc8f4 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php
@@ -37,6 +37,13 @@ protected function configure()
                 'f',
                 InputOption::VALUE_NONE,
                 'force generation of tests regardless of Magento Instance Configuration'
+            )->addOption(
+                'debug',
+                'd',
+                InputOption::VALUE_OPTIONAL,
+                'Run extra validation when running tests. Use option \'none\' to turn off debugging -- 
+                 added for backward compatibility, will be removed in the next MAJOR release',
+                MftfApplicationConfig::LEVEL_DEFAULT
             );
 
         parent::configure();
@@ -58,6 +65,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
         $skipGeneration = $input->getOption('skip-generate');
         $force = $input->getOption('force');
         $remove = $input->getOption('remove');
+        $debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
 
         if ($skipGeneration and $remove) {
             // "skip-generate" and "remove" options cannot be used at the same time
@@ -75,7 +83,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
                 ]),
                 '--force' => $force,
                 '--remove' => $remove,
-                '--debug' => MftfApplicationConfig::LEVEL_NONE
+                '--debug' => $debug
             ];
             $command->run(new ArrayInput($args), $output);
         }
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
index 899fa8312..d6a580d52 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php
@@ -13,6 +13,7 @@
 use Symfony\Component\Console\Output\OutputInterface;
 use Symfony\Component\Process\Process;
 use Magento\FunctionalTestingFramework\Exceptions\TestFrameworkException;
+use Symfony\Component\Console\Input\InputOption;
 
 class RunTestFailedCommand extends BaseGenerateCommand
 {
@@ -49,7 +50,15 @@ class RunTestFailedCommand extends BaseGenerateCommand
     protected function configure()
     {
         $this->setName('run:failed')
-            ->setDescription('Execute a set of tests referenced via failed file');
+            ->setDescription('Execute a set of tests referenced via failed file')
+            ->addOption(
+                'debug',
+                'd',
+                InputOption::VALUE_OPTIONAL,
+                'Run extra validation when running failed tests. Use option \'none\' to turn off debugging -- 
+                 added for backward compatibility, will be removed in the next MAJOR release',
+                MftfApplicationConfig::LEVEL_DEFAULT
+            );
 
         parent::configure();
     }
@@ -67,12 +76,13 @@ protected function configure()
      */
     protected function execute(InputInterface $input, OutputInterface $output): int
     {
+        $debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
         // Create Mftf Configuration
         MftfApplicationConfig::create(
             false,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::LEVEL_NONE
+            $debug
         );
 
         $testConfiguration = $this->getFailedTestList();
@@ -83,7 +93,11 @@ protected function execute(InputInterface $input, OutputInterface $output): int
         }
 
         $command = $this->getApplication()->find('generate:tests');
-        $args = ['--tests' => $testConfiguration, '--remove' => true];
+        $args = [
+            '--tests' => $testConfiguration,
+            '--remove' => true,
+            '--debug' => $debug
+        ];
 
         $command->run(new ArrayInput($args), $output);
 
diff --git a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
index 174fd8130..d714a1dd1 100644
--- a/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
+++ b/src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php
@@ -39,6 +39,13 @@ protected function configure()
                 'f',
                 InputOption::VALUE_NONE,
                 'force generation of tests regardless of Magento Instance Configuration'
+            )->addOption(
+                'debug',
+                'd',
+                InputOption::VALUE_OPTIONAL,
+                'Run extra validation when running tests. Use option \'none\' to turn off debugging -- 
+                 added for backward compatibility, will be removed in the next MAJOR release',
+                MftfApplicationConfig::LEVEL_DEFAULT
             )->addArgument(
                 'groups',
                 InputArgument::IS_ARRAY | InputArgument::REQUIRED,
@@ -64,6 +71,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
         $force = $input->getOption('force');
         $groups = $input->getArgument('groups');
         $remove = $input->getOption('remove');
+        $debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
 
         if ($skipGeneration and $remove) {
             // "skip-generate" and "remove" options cannot be used at the same time
@@ -77,7 +85,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $force,
             MftfApplicationConfig::GENERATION_PHASE,
             false,
-            MftfApplicationConfig::LEVEL_NONE
+            $debug
         );
 
         if (!$skipGeneration) {
@@ -86,7 +94,8 @@ protected function execute(InputInterface $input, OutputInterface $output): int
             $args = [
                 '--tests' => $testConfiguration,
                 '--force' => $force,
-                '--remove' => $remove
+                '--remove' => $remove,
+                '--debug' => $debug
             ];
 
             $command->run(new ArrayInput($args), $output);

From f3f53c597bd3443c0cc17770e35e3c34345a7e00 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Tue, 28 May 2019 23:31:03 -0500
Subject: [PATCH 59/63] MQE-1541: Add option to generate:tests for XSD
 validation on 'merged files'

updated documentation for --debug
---
 docs/commands/mftf.md | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index fbca2c8de..fa6f1646c 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -122,7 +122,7 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
-| `--debug or --debug=[<none>]`| Performs schema validations on XML files </br> <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. </br> <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
 #### Examples of the JSON configuration
@@ -295,9 +295,7 @@ vendor/bin/mftf run:group [--skip-generate|--remove] [--] <group1> [<group2>]
 | --------------------- | --------------------------------------------------------------------------------------------------------- |
 | `-k, --skip-generate` | Skips generating from the source XML. Instead, the command executes previously-generated groups of tests. |
 | `-r, --remove`        | Removes previously generated suites and tests before the actual generation and run.                       |
-| `--debug or --debug=[<none>]`| <br/>Performs schema validations on XML files </br> <br/> DEFAULT: `run:group` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test run fails because of an invalid XML schema. This option takes extra processing time. Use it after test run has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
-
-
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. `run:group` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred). `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.|
 
 #### Examples
 
@@ -329,7 +327,7 @@ vendor/bin/mftf run:test [--skip-generate|--remove] [--] <name1> [<name2>]
 |-----------------------|-----------------------------------------------------------------------------------------------------------|
 | `-k, --skip-generate` | Skips generating from the source XML. Instead, the command executes previously-generated groups of tests. |
 | `-r, --remove`        | Remove previously generated suites and tests.                                                             |
-| `--debug or --debug=[<none>]`| <br/>Performs schema validations on XML files </br> <br/> DEFAULT: `run:test` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test run fails because of an invalid XML schema. This option takes extra processing time. Use it after test run has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. `run:test` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred). `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.
 
 #### Examples
 
@@ -355,8 +353,7 @@ vendor/bin/mftf run:failed
 
 | Option                | Description                                                                                               |
 |-----------------------|-----------------------------------------------------------------------------------------------------------|
-| `--debug or --debug=[<none>]`| <br/>Performs schema validations on XML files </br> <br/> DEFAULT: `run:failed` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test run fails because of an invalid XML schema. This option takes extra processing time. Use it after test run has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
-
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. `run:failed` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred). Use it after test run has failed once. `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.|
 
 #### Examples
 

From 9422f83d9d25f951f605c2839120092f2c4a85a7 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Thu, 6 Jun 2019 16:19:14 -0500
Subject: [PATCH 60/63] MQE-1570: Add release to CHANGELOG.MD

---
 CHANGELOG.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index de0310a4a..b317a4cd3 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,8 +4,8 @@ Magento Functional Testing Framework Changelog
 2.4.1
 -----
 * Traceability
-    * XSD Schema validation is now enabled by default in `generate:tests`
-    * `generate:tests --debug` option has been updated to include different debug levels
+    * XSD Schema validation is now enabled by default in `generate:tests`, `run:test`,`run:failed`, `run:group`
+    * `--debug` option for the above commands has been updated to include different debug levels
         * See DevDocs for details
 
 ### Fixes

From d972c20a28f72f410a72058051192ce654769d98 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Thu, 6 Jun 2019 16:24:59 -0500
Subject: [PATCH 61/63] MQE-1570: Add release to CHANGELOG.MD

updated as per latest changes
---
 CHANGELOG.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index b317a4cd3..ee0d665e5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,7 @@ Magento Functional Testing Framework Changelog
 2.4.1
 -----
 * Traceability
-    * XSD Schema validation is now enabled by default in `generate:tests`, `run:test`,`run:failed`, `run:group`
+    * XSD Schema validation is now enabled by default in `generate:tests`, `run:test`, `run:failed`, `run:group`
     * `--debug` option for the above commands has been updated to include different debug levels
         * See DevDocs for details
 

From 588298e8a4ea3657f5e559f3e0eaf400d255fee2 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Fri, 7 Jun 2019 14:06:55 -0500
Subject: [PATCH 62/63] MQE-1569: [2.4.1 - Release] Checklist

fixed line break issues in devdocs
---
 docs/commands/mftf.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index fa6f1646c..4bd599e78 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -122,7 +122,7 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
-| `--debug or --debug=[<none>]`| Performs schema validations on XML files. </br> <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br><br> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
 #### Examples of the JSON configuration

From 48dd236159c79820aed56ebf6bf5edfbb4744193 Mon Sep 17 00:00:00 2001
From: Soumya Unnikrishnan <sunnikri@adobe.com>
Date: Fri, 7 Jun 2019 14:11:05 -0500
Subject: [PATCH 63/63] MQE-1569: [2.4.1 - Release] Checklist

fixed line break issues in devdocs
---
 docs/commands/mftf.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
index 4bd599e78..2301d6f44 100644
--- a/docs/commands/mftf.md
+++ b/docs/commands/mftf.md
@@ -122,7 +122,7 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
-| `--debug or --debug=[<none>]`| Performs schema validations on XML files. <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br><br> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br><br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
 #### Examples of the JSON configuration