diff --git a/docs/accessibility/changelog.mdx b/docs/accessibility/changelog.mdx index 1d819e439a..397e85725d 100644 --- a/docs/accessibility/changelog.mdx +++ b/docs/accessibility/changelog.mdx @@ -9,6 +9,10 @@ sidebar_position: 200 # Changelog +## Week of 3/24/2025 + +- Cypress Accessibility results are now included in the [Data Extract API](/cloud/integrations/data-extract-api) so that you can retrieve data over time. + ## Week of 3/10/2025 - The [Cypress Accessibility Results API](/accessibility/results-api) has been updated with additional details about accessibility scores and failed element counts at the run level. Also, view-level reporting has been added, breaking out accessibility report data by the pages and components covered in the report. These improvements support more advanced workflows based on the accessibility data in the API, such as alerting a specific team about violations in a part of the app they own, or implementing different standards for different parts of the application. diff --git a/docs/cloud/integrations/data-extract-api.mdx b/docs/cloud/integrations/data-extract-api.mdx index adb3fa8799..efbbe3e82c 100644 --- a/docs/cloud/integrations/data-extract-api.mdx +++ b/docs/cloud/integrations/data-extract-api.mdx @@ -69,14 +69,14 @@ The following query parameters are available for use: - start_date. **Required** and should be passed in either yyyy-MM-dd or yyyy-MM-ddTHH:mm:ss format. - For example, to request a start date of September 1, 2024, use: `?start_date=2024-09-01`. To request a - start date of 12:30p UTC on that day, use: `?start_date=2024-09-01T12%3A30%3A00` + For example, to request a start date of January 1, 2025, use: `?start_date=2025-01-01`. To request a + start date of 12:30p UTC on that day, use: `?start_date=2025-01-01T12%3A30%3A00` - end_date. Optional and should be passed in either yyyy-MM-dd or yyyy-MM-ddTHH:mm:ss format. If no end date is specified the query will default to today. - For example, to request an end date of start of day September 1, 2024, use: `?end_date=2024-09-01`. To request an - end date of 12:30p UTC on that day, use: `?end_date=2024-09-01T12%3A30%3A00` + For example, to request an end date of start of day March 1, 2025, use: `?end_date=2025-03-01`. To request an + end date of 12:30p UTC on that day, use: `?end_date=2025-03-01T12%3A30%3A00` - projects. Optional, and accepts the name of your Cypress project. The project name should be URL encoded. You can pass as many projects parameters as needed. @@ -90,6 +90,10 @@ The following query parameters are available for use: - export_format. Specify the type of data format that will be returned in the response file. Valid options include: `csv`, `json`, `xlsx` + For example, to request that the data be returned in JSON format, use: `&export_format=json` + +## Usage & projects + ### Project list Retrieves a list of all projects in Cypress Cloud that have had tests recorded since the start_date. @@ -104,7 +108,7 @@ Data returned includes: - id Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=project-list&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=project-list&export_format=csv&start_date=2025-03-01 ### Tests per project @@ -119,7 +123,7 @@ Data returned includes: - test_run_count Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=usage-per-project-summary&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=usage-per-project-summary&export_format=csv&start_date=2025-03-01 ### Tests per project over time @@ -136,7 +140,86 @@ Data returned includes: - day Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=usage-per-project-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=usage-per-project-over-time&export_format=csv&start_date=2025-03-01 + +### Cypress test types + +This data helps you to grow and manage your Cypress test suite by understanding how +you are adopting end-to-end and component testing in your organization. + +The report_id is: `cypress-test-types` + +Data returned includes: + +- testing_type (e2e or component) +- total_tests +- total_e2e +- total_ct +- week +- day + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-test-types&export_format=csv&start_date=2025-03-01 + +### Cypress test suite size + +This data helps you to understand and manage the growth of your test suite size across +all projects in your organization. It calculates the average number of test cases executed per run +and excludes runs that errored or timed out since they don't accurately represent the size of your +test suite. + +The report_id is: `test-suite-size-summary` + +Data returned includes: + +- total_tests +- total_e2e +- total_ct +- week + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-suite-size-summary&export_format=csv&start_date=2025-03-01 + +### Cypress test suite over time + +This data helps you to understand and manage the growth of your test suite size across +all projects in your organization. It calculates the average number of test cases executed per run +and excludes runs that errored or timed out since they don't accurately represent the size of your +test suite. This data is reported over time. + +The report_id is: `test-suite-size-over-time` + +Data returned includes: + +- project_name +- total_tests +- total_e2e +- total_ct +- week + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-suite-size-over-time&export_format=csv&start_date=2025-03-01 + +### Project test count and status + +This data allows you to extract consolidated run status at the project level. +Data is accumulated from the status of each individual test result. This data is +reported over time. + +The report_id is: `status-per-test-daily` + +Data returned includes: + +- project_name +- status +- test_run_count +- week +- day + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-test-daily&export_format=csv&start_date=2025-03-01 + +## Run status ### Status by run @@ -151,7 +234,7 @@ Data returned includes: - test_run_count Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-build-summary&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-build-summary&export_format=csv&start_date=2025-03-01 ### Status by run over time @@ -168,7 +251,7 @@ Data returned includes: - day Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-build-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-build-over-time&export_format=csv&start_date=2025-03-01 ### Status by spec @@ -183,7 +266,7 @@ Data returned includes: - test_run_count Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-spec-summary&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-spec-summary&export_format=csv&start_date=2025-03-01 ### Status by spec over time @@ -200,7 +283,7 @@ Data returned includes: - day Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-spec-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-spec-over-time&export_format=csv&start_date=2025-03-01 ### Status by test run @@ -214,7 +297,7 @@ Data returned includes: - passed Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-test-summary&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-test-summary&export_format=csv&start_date=2025-03-01 ### Status by test run over time @@ -231,7 +314,9 @@ Data returned includes: - status Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-test-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-test-over-time&export_format=csv&start_date=2025-03-01 + +## Versions & browsers ### Cypress run versions @@ -246,7 +331,7 @@ Data returned includes: - cypress_version Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-build-versions-summary&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-build-versions-summary&export_format=csv&start_date=2025-03-01 ### Cypress run versions over time @@ -262,7 +347,7 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-build-versions-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-build-versions-over-time&export_format=csv&start_date=2025-03-01 ### Cypress run versions per project over time @@ -279,7 +364,7 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-build-versions-per-project-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-build-versions-per-project-over-time&export_format=csv&start_date=2025-03-01 ### Browsers tested @@ -293,7 +378,7 @@ Data returned includes: - browser Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browsers-tested&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browsers-tested&export_format=csv&start_date=2025-03-01 ### Browser versions tested @@ -308,7 +393,7 @@ Data returned includes: - version Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browser-versions&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browser-versions&export_format=csv&start_date=2025-03-01 ### Browser versions tested over time @@ -326,7 +411,7 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browser-versions-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browser-versions-over-time&export_format=csv&start_date=2025-03-01 ### Browser versions tested per project over time @@ -345,84 +430,11 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browser-versions-per-project-over-time&export_format=csv&start_date=2024-01-01 - -### Cypress test types - -This data helps you to grow and manage your Cypress test suite by understanding how -you are adopting end-to-end and component testing in your organization. +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=browser-versions-per-project-over-time&export_format=csv&start_date=2025-03-01 -The report_id is: `cypress-test-types` +## Test-level data -Data returned includes: - -- testing_type (e2e or component) -- total_tests -- total_e2e -- total_ct -- week -- day - -Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=cypress-test-types&export_format=csv&start_date=2024-01-01 - -### Cypress test suite size - -This data helps you to understand and manage the growth of your test suite size across -all projects in your organization. It calculates the average number of test cases executed per run -and excludes runs that errored or timed out since they don't accurately represent the size of your -test suite. - -The report_id is: `test-suite-size-summary` - -Data returned includes: - -- total_tests -- total_e2e -- total_ct -- week - -Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-suite-size-summary&export_format=csv&start_date=2024-01-01 - -### Cypress test suite over time - -This data helps you to understand and manage the growth of your test suite size across -all projects in your organization. It calculates the average number of test cases executed per run -and excludes runs that errored or timed out since they don't accurately represent the size of your -test suite. This data is reported over time. - -The report_id is: `test-suite-size-over-time` - -Data returned includes: - -- project_name -- total_tests -- total_e2e -- total_ct -- week - -Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-suite-size-over-time&export_format=csv&start_date=2024-01-01 - -### Project test count and status - -This data allows you to extract consolidated run status at the project level. -Data is accumulated from the status of each individual test result. This data is -reported over time. - -The report_id is: `status-per-test-daily` - -Data returned includes: - -- project_name -- status -- test_run_count -- week -- day - -Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=status-per-test-daily&export_format=csv&start_date=2024-01-01 +All data for these APIs is available near real-time, 30 minutes after the run has completed. ### Individual spec results @@ -457,7 +469,7 @@ Data returned includes: - os_version Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=spec-details&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=spec-details&export_format=csv&start_date=2025-03-01 ### Individual failed test results @@ -488,7 +500,7 @@ Data returned includes: - test_name (the test name matches what is shown in Cypress Cloud and a '///' delimiter is used between fragments of the test name) Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=failed-test-details&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=failed-test-details&export_format=csv&start_date=2025-03-01 ### Individual test results @@ -520,7 +532,9 @@ Data returned includes: - test_name (the test name matches what is shown in Cypress Cloud and a '///' delimiter is used between fragments of the test name) Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-details&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-details&export_format=csv&start_date=2025-03-01 + +## Run duration ### Average run duration over time @@ -540,7 +554,7 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=average-passing-build-duration&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=average-passing-build-duration&export_format=csv&start_date=2025-03-01 ### Average spec duration over time @@ -563,7 +577,9 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=average-passing-spec-duration&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=average-passing-spec-duration&export_format=csv&start_date=2025-03-01 + +## Flaky ### Test flake detail over time @@ -580,7 +596,7 @@ Data returned includes: - day Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-per-project-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-per-project-over-time&export_format=csv&start_date=2025-03-01 ### Flaky rate per project @@ -599,7 +615,7 @@ Data returned includes: - flaky_rate Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-rate-per-project&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-rate-per-project&export_format=csv&start_date=2025-03-01 ### Flaky rate per project over time @@ -620,7 +636,7 @@ Data returned includes: - week Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-rate-per-project-over-time&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-rate-per-project-over-time&export_format=csv&start_date=2025-03-01 ### Flaky test details @@ -649,7 +665,7 @@ Data returned includes: - test_replay_url Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flaky-test-details&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flaky-test-details&export_format=csv&start_date=2025-03-01 ### Top flaky tests per project @@ -666,7 +682,9 @@ Data returned includes: - test_replay_url Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=top-flaky-per-project&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=top-flaky-per-project&export_format=csv&start_date=2025-03-01 + +## Aggregated data ### Top failures per project @@ -687,7 +705,7 @@ Data returned includes: - test_replay_url Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=top-failures-per-project&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=top-failures-per-project&export_format=csv&start_date=2025-03-01 ### Top errors per project @@ -707,4 +725,122 @@ Data returned includes: - test_replay_url Example: -https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=top-errors-per-project&export_format=csv&start_date=2024-01-01 +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=top-errors-per-project&export_format=csv&start_date=2025-03-01 + +## UI Coverage + +Learn more about Cypress UI Coverage [here](/ui-coverage/get-started/introduction). + +### UI Coverage per project + +This data helps you understand which projects in your organization have the +highest or lowest UI Coverage %. Data is returned in a simple summary per project. + +The report_id is: `ui-coverage-per-project-summary` + +Data returned includes: + +- project_name +- ui_coverage_pct + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=ui-coverage-per-project-summary&export_format=csv&start_date=2025-03-01 + +### UI Coverage per project over time + +This data helps you understand which projects in your organization have the +highest or lowest UI Coverage % and how those measurements look over time. + +The report_id is: `ui-coverage-per-project-over-time` + +Data returned includes: + +- project_name +- ui_coverage_pct +- week +- day + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=ui-coverage-per-project-over-time&export_format=csv&start_date=2025-03-01 + +### UI Coverage details + +This report allows you to extract the UI Coverage % for each run along with other attributes of the run. The maximum number of records +returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will +exceed 500k for a single request. + +The report_id is: `ui-coverage-details` + +Data returned includes: + +- project_name +- created_at (date and time) +- run_number +- status +- ui_coverage_pct +- commit_author_name +- commit_branch +- ci_build_id +- run_tags (string array) + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=ui-coverage-details&export_format=csv&start_date=2025-03-01 + +## Cypress Accessibility + +Learn more about Cypress Accessibility [here](/accessibility/get-started/introduction). + +### Accessibility score per project + +This data helps you understand which projects in your organization have the +highest or lowest Accessibility scores. Data is returned in a simple summary per project. + +The report_id is: `accessibility-per-project-summary` + +Data returned includes: + +- project_name +- accessibility_score + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=accessibility-per-project-summary&export_format=csv&start_date=2025-03-01 + +### Accessibility score per project over time + +This data helps you understand which projects in your organization have the +highest or lowest Accessibility scores and how those scores looks over time. + +The report_id is: `accessibility-per-project-over-time` + +Data returned includes: + +- project_name +- accessibility_score +- week +- day + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=accessibility-per-project-over-time&export_format=csv&start_date=2025-03-01 + +### Accessibility score details + +This report allows you to extract Cypress Accessibility scores for each run along with other attributes of the run. The +maximum number of records returned is 500,000. Make multiple requests with different start_date and end_date parameter +values if your volume of records will exceed 500k for a single request. + +The report_id is: `accessibility-details` + +Data returned includes: + +- project_name +- created_at (date and time) +- run_number +- status +- accessibility_score +- commit_author_name +- commit_branch +- ci_build_id +- run_tags (string array) + +Example: +https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=accessibility-details&export_format=csv&start_date=2025-03-01 diff --git a/docs/ui-coverage/changelog.mdx b/docs/ui-coverage/changelog.mdx index 94f3e81e56..c430af2d1c 100644 --- a/docs/ui-coverage/changelog.mdx +++ b/docs/ui-coverage/changelog.mdx @@ -9,6 +9,10 @@ sidebar_position: 200 # Changelog +## Week of 3/24/2025 + +- UI Coverage results are now included in the [Data Extract API](/cloud/integrations/data-extract-api) so that you can retrieve data over time. + ## Week of 1/15/2024 - **Branch Review Integration (Beta):** Added UI Coverage comparisons to Branch Review, enabling teams to see how code changes impact test coverage across branches. Available for all Git providers and includes manual run selection for customized comparisons.