diff --git a/docs/ui-coverage/results-api.mdx b/docs/ui-coverage/results-api.mdx index 1b533edef4..ec796464b8 100644 --- a/docs/ui-coverage/results-api.mdx +++ b/docs/ui-coverage/results-api.mdx @@ -41,7 +41,7 @@ If you check this in as a dependency, your installation will fail when we update ## Usage -### **1. Fetch UI Coverage Results** +### **1. Get the UI Coverage Results** Write a script to fetch UI Coverage results and assert test coverage criteria. This script will be executed in CI. @@ -106,9 +106,77 @@ getUICoverageResults({ }) ``` -## 2. Integrate into CI workflow +#### Result Details + +The `getUICoverageResults` utility returns the following data: + +```javascript +{ + // The run number of the identified build. + runNumber: number + // The run url for the identified build. + runUrl: 'https://cloud.cypress.io/projects/:project_id/runs/:run_number' + // The status of the identified build. + runStatus: 'passed' | 'failed' | 'errored' | 'timedOut' | 'cancelled' | 'noTests' + // The url that links to UI Coverage report for the identified build. + uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' + summary: { + // Indicates whether a complete UI Coverage report was generated. + // For example, if a run was cancelled and the report expected to run + // for 20 specs, but only 10 ran, this would result in a partial report. + isPartialReport: boolean + // The report coverage from 0-100 with 2 decimal precision (e.g 92.45). + coverage: number + // The number of views tested and analyzed. + viewCount: number + // The number of interactive elements that were tested. + testedElementsCount:number + // The number of interactive elements that were not tested. + untestedElementsCount: number + } + // The list of tested views and the coverage of each page. + views: [{ + // The sanatized URL pattern shown in the report. + displayName: string + // The view coverage from 0-100 with 2 decimal precision (e.g 92.45). + coverage: number + // The number of interactive elements that were tested on this view. + testedElementsCount:number + // The number of interactive elements that were not tested on this view. + untestedElementsCount: number + // The url that links the report for this view. + uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' + }] +} +``` + +### **2. Add to CI Workflow** + +In your CI workflow that runs your Cypress tests, + +1. Update your install job to install the `@cypress/extract-cloud-results` module. +2. Pass in the necessary arguments to `getUICoverageResults`. +3. Add a new step to the job that runs your Cypress tests to verify the UI Coverage results. + +:::info -Add steps to your CI configuration to install the module, fetch results, and verify coverage: +If you record multiple runs in a single CI build, you must record these runs using the `--tag` parameter and then call `getUICoverageResults` with the `runTags` argument for each run. + +This is necessary to identify each unique run and return a corresponding set of results. The tags are how each run is uniquely identified. + +**Example** + +- Let's imagine that within a single CI build you call `cypress run --record` multiple times because you're running one set of tests against a `staging` environment, followed by a `production` environment. +- In this scenario, you pass a different `--tag` to each cypress run + - `cypress run --record --tag staging` + - `cypress run --record --tag production` +- When calling `getUICoverageResults` you would then pass these same tags to get the unique set of results for each run + - `getUICoverageResults({ runTags: ['staging']})` + - `getUICoverageResults({ runTags: ['production']})` + +::: + +#### Example Job Workflow Update: @@ -258,73 +326,3 @@ workflows: - -#### Result Details - -The `getUICoverageResults` utility returns the following data: - -```javascript -{ - // The run number of the identified build. - runNumber: number - // The run url for the identified build. - runUrl: 'https://cloud.cypress.io/projects/:project_id/runs/:run_number' - // The status of the identified build. - runStatus: 'passed' | 'failed' | 'errored' | 'timedOut' | 'cancelled' | 'noTests' - // The url that links to UI Coverage report for the identified build. - uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' - summary: { - // Indicates whether a complete UI Coverage report was generated. - // For example, if a run was cancelled and the report expected to run - // for 20 specs, but only 10 ran, this would result in a partial report. - isPartialReport: boolean - // The report coverage from 0-100 with 2 decimal precision (e.g 92.45). - coverage: float - // The number of views tested and analyzed. - viewCount: number - // The number of interactive elements that were tested. - testedElementsCount:number - // The number of interactive elements that were not tested. - untestedElementsCount: number - } - // The list of tested views and the coverage of each page. - views: [{ - // The sanatized URL pattern shown in the report. - displayName: string - // The view coverage from 0-100 with 2 decimal precision (e.g 92.45). - coverage: float - // The number of interactive elements that were tested on this view. - testedElementsCount:number - // The number of interactive elements that were not tested on this view. - untestedElementsCount: number - // The url that links the report for this view. - uiCoverageReportUrl: 'https://cloud.cypress.io/[...]' - }] -} -``` - -### **2. Add to CI Workflow** - -In your CI workflow that runs your Cypress tests, - -1. Update your install job to install the `@cypress/extract-cloud-results` module. -2. Pass in the necessary arguments to `getUICoverageResults`. -3. Add a new step to the job that runs your Cypress tests to verify the UI Coverage results. - -:::info - -If you record multiple runs in a single CI build, you must record these runs using the `--tag` parameter and then call `getUICoverageResults` with the `runTags` argument for each run. - -This is necessary to identify each unique run and return a corresponding set of results. The tags are how each run is uniquely identified. - -**Example** - -- Let's imagine that within a single CI build you call `cypress run --record` multiple times because you're running one set of tests against a `staging` environment, followed by a `production` environment. -- In this scenario, you pass a different `--tag` to each cypress run - - `cypress run --record --tag staging` - - `cypress run --record --tag production` -- When calling `getUICoverageResults` you would then pass these same tags to get the unique set of results for each run - - `getUICoverageResults({ runTags: ['staging']})` - - `getUICoverageResults({ runTags: ['production']})` - -:::