Skip to content

Update results-api.mdx #6150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 17, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 71 additions & 73 deletions docs/ui-coverage/results-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ If you check this in as a dependency, your installation will fail when we update

## Usage

### **1. Fetch UI Coverage Results**
### **1. Get the UI Coverage Results**

Write a script to fetch UI Coverage results and assert test coverage criteria. This script will be executed in CI.

Expand Down Expand Up @@ -106,9 +106,77 @@ getUICoverageResults({
})
```

## 2. Integrate into CI workflow
#### Result Details

The `getUICoverageResults` utility returns the following data:

```javascript
{
// The run number of the identified build.
runNumber: number
// The run url for the identified build.
runUrl: 'https://cloud.cypress.io/projects/:project_id/runs/:run_number'
// The status of the identified build.
runStatus: 'passed' | 'failed' | 'errored' | 'timedOut' | 'cancelled' | 'noTests'
// The url that links to UI Coverage report for the identified build.
uiCoverageReportUrl: 'https://cloud.cypress.io/[...]'
summary: {
// Indicates whether a complete UI Coverage report was generated.
// For example, if a run was cancelled and the report expected to run
// for 20 specs, but only 10 ran, this would result in a partial report.
isPartialReport: boolean
// The report coverage from 0-100 with 2 decimal precision (e.g 92.45).
coverage: number
// The number of views tested and analyzed.
viewCount: number
// The number of interactive elements that were tested.
testedElementsCount:number
// The number of interactive elements that were not tested.
untestedElementsCount: number
}
// The list of tested views and the coverage of each page.
views: [{
// The sanatized URL pattern shown in the report.
displayName: string
// The view coverage from 0-100 with 2 decimal precision (e.g 92.45).
coverage: number
// The number of interactive elements that were tested on this view.
testedElementsCount:number
// The number of interactive elements that were not tested on this view.
untestedElementsCount: number
// The url that links the report for this view.
uiCoverageReportUrl: 'https://cloud.cypress.io/[...]'
}]
}
```

### **2. Add to CI Workflow**

In your CI workflow that runs your Cypress tests,

1. Update your install job to install the `@cypress/extract-cloud-results` module.
2. Pass in the necessary arguments to `getUICoverageResults`.
3. Add a new step to the job that runs your Cypress tests to verify the UI Coverage results.

:::info

Add steps to your CI configuration to install the module, fetch results, and verify coverage:
If you record multiple runs in a single CI build, you must record these runs using the `--tag` parameter and then call `getUICoverageResults` with the `runTags` argument for each run.

This is necessary to identify each unique run and return a corresponding set of results. The tags are how each run is uniquely identified.

**Example**

- Let's imagine that within a single CI build you call `cypress run --record` multiple times because you're running one set of tests against a `staging` environment, followed by a `production` environment.
- In this scenario, you pass a different `--tag` to each cypress run
- `cypress run --record --tag staging`
- `cypress run --record --tag production`
- When calling `getUICoverageResults` you would then pass these same tags to get the unique set of results for each run
- `getUICoverageResults({ runTags: ['staging']})`
- `getUICoverageResults({ runTags: ['production']})`

:::

#### Example Job Workflow Update:

<Tabs groupId="ui-cov-results-api">
<TabItem value="GitHub Actions" active>
Expand Down Expand Up @@ -258,73 +326,3 @@ workflows:

</TabItem>
</Tabs>

#### Result Details

The `getUICoverageResults` utility returns the following data:

```javascript
{
// The run number of the identified build.
runNumber: number
// The run url for the identified build.
runUrl: 'https://cloud.cypress.io/projects/:project_id/runs/:run_number'
// The status of the identified build.
runStatus: 'passed' | 'failed' | 'errored' | 'timedOut' | 'cancelled' | 'noTests'
// The url that links to UI Coverage report for the identified build.
uiCoverageReportUrl: 'https://cloud.cypress.io/[...]'
summary: {
// Indicates whether a complete UI Coverage report was generated.
// For example, if a run was cancelled and the report expected to run
// for 20 specs, but only 10 ran, this would result in a partial report.
isPartialReport: boolean
// The report coverage from 0-100 with 2 decimal precision (e.g 92.45).
coverage: float
// The number of views tested and analyzed.
viewCount: number
// The number of interactive elements that were tested.
testedElementsCount:number
// The number of interactive elements that were not tested.
untestedElementsCount: number
}
// The list of tested views and the coverage of each page.
views: [{
// The sanatized URL pattern shown in the report.
displayName: string
// The view coverage from 0-100 with 2 decimal precision (e.g 92.45).
coverage: float
// The number of interactive elements that were tested on this view.
testedElementsCount:number
// The number of interactive elements that were not tested on this view.
untestedElementsCount: number
// The url that links the report for this view.
uiCoverageReportUrl: 'https://cloud.cypress.io/[...]'
}]
}
```

### **2. Add to CI Workflow**

In your CI workflow that runs your Cypress tests,

1. Update your install job to install the `@cypress/extract-cloud-results` module.
2. Pass in the necessary arguments to `getUICoverageResults`.
3. Add a new step to the job that runs your Cypress tests to verify the UI Coverage results.

:::info

If you record multiple runs in a single CI build, you must record these runs using the `--tag` parameter and then call `getUICoverageResults` with the `runTags` argument for each run.

This is necessary to identify each unique run and return a corresponding set of results. The tags are how each run is uniquely identified.

**Example**

- Let's imagine that within a single CI build you call `cypress run --record` multiple times because you're running one set of tests against a `staging` environment, followed by a `production` environment.
- In this scenario, you pass a different `--tag` to each cypress run
- `cypress run --record --tag staging`
- `cypress run --record --tag production`
- When calling `getUICoverageResults` you would then pass these same tags to get the unique set of results for each run
- `getUICoverageResults({ runTags: ['staging']})`
- `getUICoverageResults({ runTags: ['production']})`

:::