diff --git a/ci/builders/README.md b/ci/builders/README.md index 645cc2d731ff5..7c17c868f8c39 100644 --- a/ci/builders/README.md +++ b/ci/builders/README.md @@ -1,7 +1,7 @@ # Flutter Engine Build Definition Language -The ***Flutter Engine Build Definition Language*** describes a build on CI -by defining a combination of *sub-builds*, *archives*, *generators* and *dependencies*. It +The **_Flutter Engine Build Definition Language_** describes a build on CI +by defining a combination of _sub-builds_, _archives_, _generators_ and _dependencies_. It makes it simple to shard sub-builds by mapping build inputs to workflows, and listing the sub-build-generated artifacts explicitly. The Build Definition Language, Engine Recipes V2 and the generation of artifacts using GN+Ninja set the groundwork @@ -9,26 +9,26 @@ for efficient builds with dependency reusability. **Author: Godofredo Contreras (godofredoc)**\ **Go Link: flutter.dev/go/engine-build-definition-language**\ -**Created:** 01/2023 / **Last updated:** 04/2023 +**Created:** 01/2023 / **Last updated:** 04/2023 ## Glossary -* **[recipes](https://github.com/luci/recipes-py)** - domain specific -language for specifying sequences of subprocess calls in a cross-platform and -testable way. -* **Generator** - scripts in Dart, python or bash that combine the output of -sub-builds to generate artifacts. -* **Builder** - a combination of configuration, recipes and a given commit to -build and test artifacts. -* **Build** - a builder running with specific properties, repository and -commit. -* **[GN](https://gn.googlesource.com/gn/)** - a meta-build system that -generates build files for [Ninja](https://ninja-build.org/). -* **[Ninja](https://ninja-build.org)** - Ninja is a small build system with a -focus on speed. -* **CAS** - a service that stores arbitrary binary blobs addressed by (hash of) -their content. It is specialized for low latency, high volume query/read/write -operations. +- **[recipes](https://github.com/luci/recipes-py)** - domain specific + language for specifying sequences of subprocess calls in a cross-platform and + testable way. +- **Generator** - scripts in Dart, python or bash that combine the output of + sub-builds to generate artifacts. +- **Builder** - a combination of configuration, recipes and a given commit to + build and test artifacts. +- **Build** - a builder running with specific properties, repository and + commit. +- **[GN](https://gn.googlesource.com/gn/)** - a meta-build system that + generates build files for [Ninja](https://ninja-build.org/). +- **[Ninja](https://ninja-build.org)** - Ninja is a small build system with a + focus on speed. +- **CAS** - a service that stores arbitrary binary blobs addressed by (hash of) + their content. It is specialized for low latency, high volume query/read/write + operations. ## USAGE EXAMPLES @@ -42,22 +42,21 @@ uploads them to the Google Cloud Storage bucket. The [.ci.yaml file](https://github.com/flutter/engine/blob/main/.ci.yaml) at the root of the `flutter/engine` repository puts all the components together. Builds are specified in that file using a property pointing to the build definition -file to be used by engine\_v2 recipes. Full documentation of the `.ci.yaml` file format +file to be used by engine_v2 recipes. Full documentation of the `.ci.yaml` file format can be found [in the Cocoon repository here](https://github.com/flutter/cocoon/blob/main/CI_YAML.md). The following is a sample build configuration referencing -[android\_aot\_engine.json](https://github.com/flutter/engine/blob/main/ci/builders/mac_android_aot_engine.json) +[android_aot_engine.json](https://github.com/flutter/engine/blob/main/ci/builders/mac_android_aot_engine.json) in the `config_name` under `properties`: ```yaml - - name: Mac mac_android_aot_engine - recipe: engine_v2/engine_v2 - timeout: 60 - properties: - config_name: mac_android_aot_engine - $flutter/osx_sdk : >- - { "sdk_version": "14e300c" } - +- name: Mac mac_android_aot_engine + recipe: engine_v2/engine_v2 + timeout: 60 + properties: + config_name: mac_android_aot_engine + $flutter/osx_sdk: >- + { "sdk_version": "14e300c" } ``` ## Build Definition Language Assumptions @@ -65,13 +64,13 @@ in the `config_name` under `properties`: To keep the build definition language simple the following assumptions were made during its design: -* A build can be expressed as a set of independent sub-builds. -* A sub-build can be defined as a sequence of a `gn` configuration step, +- A build can be expressed as a set of independent sub-builds. +- A sub-build can be defined as a sequence of a `gn` configuration step, a `ninja` build step, followed by self-contained test scripts, and self-contained generator scripts. All the elements are optional allowing to use gn+ninja without generators or generators without gn+ninja. -* All the sub-builds required by a global generator are defined within the same -configuration file. +- All the sub-builds required by a global generator are defined within the same + configuration file. ## Build configuration file @@ -81,13 +80,12 @@ file: ```json { - "builds": [], - "tests": [], - "generators": { - "tasks": [] - }, - "archives": [ - ] + "builds": [], + "tests": [], + "generators": { + "tasks": [] + }, + "archives": [] } ``` @@ -115,14 +113,14 @@ The following is the high level structure of the build component: ```json { - "archives": [], - "drone_dimensions": [], - "gclient_variables": {}, - "gn": [], - "name": "host_debug", - "generators": [], - "ninja": {}, - "tests": [] + "archives": [], + "drone_dimensions": [], + "gclient_variables": {}, + "gn": [], + "name": "host_debug", + "generators": [], + "ninja": {}, + "tests": [] } ``` @@ -148,41 +146,39 @@ configuration. "name": "host_debug", "base_path": "out/host_debug/zip_archives/", "type": "gcs", - "include_paths": [ - "out/host_debug/zip_archives/linux-x64/artifacts.zip" - ], + "include_paths": ["out/host_debug/zip_archives/linux-x64/artifacts.zip"], "realm": "production" } ``` Description of the fields: -* **name:** - by default the entire build output is uploaded to CAS. - `name` is used to associate the CAS hash to a value that can be referenced - later as a dependency of global tests. Name is also used to select the folder - from within src/out to upload to CAS. e.g if the build generates - src/out/host_debug name must be `host_debug`. -* **base\_path:** - the portion of the path to remove from the full path before - uploading to its final destination. In the example the above the - base\_path **“out/host\_debug/zip\_archives”** will be removed from the - include path **"out/host\_debug/zip\_archives/linux-x64/artifacts.zip"** - before uploading to GCS, e.g. - <bucket>/flutter/<commit>/linux-x64/artifacts.zip. -* **Type:** - the type of storage to use. Currently only **“gcs”** and -**“cas”** are supported. "gcs" uploads artifacts to GCS -and "cas" to archive to CAS service. Cas value is used during development where -we need to inspect the generated artifacts without worrying about location or -cleanups. Gcs is expected for any artifacts being consumed by the flutter tool. -* **Include\_paths:** - a list of strings with the paths to be uploaded to a -given destination. -* **cas_archive** - a boolean value indicating whether the build output will -be archived to CAS or not. The default value is true. -* **realm** - a string value of either `production` or `experimental` -where production means the artifact will be uploaded to the location expected -by the flutter tool and experimental will add an `experimental` prefix to the -path to avoid interfering with production artifacts. - -#### Drone\_dimensions +- **name:** - by default the entire build output is uploaded to CAS. + `name` is used to associate the CAS hash to a value that can be referenced + later as a dependency of global tests. Name is also used to select the folder + from within src/out to upload to CAS. e.g if the build generates + src/out/host_debug name must be `host_debug`. +- **base_path:** - the portion of the path to remove from the full path before + uploading to its final destination. In the example the above the + base_path **“out/host_debug/zip_archives”** will be removed from the + include path **"out/host_debug/zip_archives/linux-x64/artifacts.zip"** + before uploading to GCS, e.g. + <bucket>/flutter/<commit>/linux-x64/artifacts.zip. +- **Type:** - the type of storage to use. Currently only **“gcs”** and + **“cas”** are supported. "gcs" uploads artifacts to GCS + and "cas" to archive to CAS service. Cas value is used during development where + we need to inspect the generated artifacts without worrying about location or + cleanups. Gcs is expected for any artifacts being consumed by the flutter tool. +- **Include_paths:** - a list of strings with the paths to be uploaded to a + given destination. +- **cas_archive** - a boolean value indicating whether the build output will + be archived to CAS or not. The default value is true. +- **realm** - a string value of either `production` or `experimental` + where production means the artifact will be uploaded to the location expected + by the flutter tool and experimental will add an `experimental` prefix to the + path to avoid interfering with production artifacts. + +#### Drone_dimensions A list of strings with key value pairs separated by an equal sign. These dimensions are used to select the bot where the sub-build will be running. @@ -200,7 +196,7 @@ for a key they are separated using `|` (pipe symbol). ] ``` -In the previous example, the build containing this drone\_dimensions component +In the previous example, the build containing this drone_dimensions component will run on a bot with a Linux OS that does not have any devices attached to it. Drone dimensions accept values separates by `|` to specify more than one value @@ -208,7 +204,7 @@ for the dimension. E.g. assuming the pool of bots have Ubuntu and Debian bots a dimension of `"os": "Debian|Ubuntu"` will resolve to use bots running either Debian or Ubuntu. -#### Gclient\_variables +#### Gclient_variables A dictionary with variables passed to gclient during a gclient sync operation. They are usually used to add or remove gclient dependencies. @@ -257,7 +253,7 @@ and “target” which is a list of strings with the Ninja targets to build. ``` In the example above the ninja command will use the configuration for -host\_debug and will build artifacts and embedder targets as described +host_debug and will build artifacts and embedder targets as described by the [flutter/build/archives/BUILD.gn](https://github.com/flutter/engine/blob/main/build/archives/BUILD.gn) file. @@ -275,10 +271,10 @@ configuration. "tests": [ { "language": "python3", - "name": "Host Tests for host_debug_impeller_vulkan", + "name": "Host Tests for host_debug_unopt", "parameters": [ "--variant", - "host_debug_impeller_vulkan", + "host_debug_unopt", "--type", "impeller-vulkan", "--engine-capture-core-dump" @@ -291,29 +287,29 @@ configuration. Description of the fields: -* **language** - the executable used to run the script, e.g. python3, bash. -In general any executable found in the path can be used as language. The -default is empty which means no interpreter will be used to run the script -and it is assumed the script is already an executable with the right -permissions to run in the target platform. -* **name** - the name of the step running the script. -* **parameters** - flags or parameters passed to the script. Parameters -accept magic environment variables(placeholders replaced before executing -the test). Magic environment variables have the following limitations: -only `${FLUTTER_LOGS_DIR}` is currently supported and it needs to be used -alone within the parameter string(e.g. `["${FLUTTER_LOGS_DIR}"]` is OK -but `["path=${FLUTTER_LOGS_DIR}"]` is not). -* **Script** - the path to the script to execute relative to the checkout -directory. -* **contexts** - a list of available contexts to add to the text execution step. -The list of supported contexts can be found [here](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/flutter_deps/api.py#687). As of 06/20/23 two contexts are supported: -"android_virtual_device" and "metric_center_token". +- **language** - the executable used to run the script, e.g. python3, bash. + In general any executable found in the path can be used as language. The + default is empty which means no interpreter will be used to run the script + and it is assumed the script is already an executable with the right + permissions to run in the target platform. +- **name** - the name of the step running the script. +- **parameters** - flags or parameters passed to the script. Parameters + accept magic environment variables(placeholders replaced before executing + the test). Magic environment variables have the following limitations: + only `${FLUTTER_LOGS_DIR}` is currently supported and it needs to be used + alone within the parameter string(e.g. `["${FLUTTER_LOGS_DIR}"]` is OK + but `["path=${FLUTTER_LOGS_DIR}"]` is not). +- **Script** - the path to the script to execute relative to the checkout + directory. +- **contexts** - a list of available contexts to add to the text execution step. + The list of supported contexts can be found [here](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/flutter_deps/api.py#687). As of 06/20/23 two contexts are supported: + "android_virtual_device" and "metric_center_token". The test scripts will run in a deferred context (failing the step only after logs have been uploaded). Tester and builder recipes provide an environment -variable called FLUTTER\_LOGS\_DIR pointing a temporary directory where the +variable called FLUTTER_LOGS_DIR pointing a temporary directory where the test runner can place any logs|artifacts needed to debug issues. At the end -of the test execution the content of FLUTTER\_LOGS\_DIR will be uploaded to +of the test execution the content of FLUTTER_LOGS_DIR will be uploaded to Google Cloud Storage before signaling the pass | fail test state. Contexts are free form python contexts that communicate with the test script @@ -322,7 +318,7 @@ to an [environment variable "token_path"](https://flutter.googlesource.com/recip Note that to keep the recipes generic they don’t know anything about what the test script is doing and it is the responsibility of the test script to -copy the relevant files to the FLUTTER\_LOGS\_DIR directory. +copy the relevant files to the FLUTTER_LOGS_DIR directory. #### Generators @@ -335,18 +331,18 @@ guidelines to make them compatible with the engine build system. The guidelines are as follows: -* Flags receiving paths to resources from multiple sub-builds need to use paths -relative to the checkout (`src/`) directory. If there are global generators in a build -configuration, the engine\_v2 recipes will download the full sub-build archives -to the src/out/<sub-build name> directory. -* Flags receiving paths to output directories must use paths relative to the -src/out folder. This is to be able to reference the artifacts in the global -archives section. -* The script is in charge of generating the final artifact, e.g. if the script -generates multiple files that will be zipped later, then it is the responsibility -of the script to generate the final zip. -* If the generator is producing a Mac/iOS artifact, then it is the responsibility -of the script to embed the signing metadata. +- Flags receiving paths to resources from multiple sub-builds need to use paths + relative to the checkout (`src/`) directory. If there are global generators in a build + configuration, the engine_v2 recipes will download the full sub-build archives + to the src/out/<sub-build name> directory. +- Flags receiving paths to output directories must use paths relative to the + src/out folder. This is to be able to reference the artifacts in the global + archives section. +- The script is in charge of generating the final artifact, e.g. if the script + generates multiple files that will be zipped later, then it is the responsibility + of the script to generate the final zip. +- If the generator is producing a Mac/iOS artifact, then it is the responsibility + of the script to embed the signing metadata. Generators contain a single property “tasks” which is a list of tasks to be performed. @@ -366,27 +362,27 @@ A `task` is a dictionary describing the scripts to be executed. The property's description is as follows: -* **Name** - the name of the step running the script. -* **Parameters** - flags passed to the script. Both input and output paths must -be relative to the checkout directory. -* **Script**, the script path relative to the checkout repository. -* **Language**, the script language executable to run the script. If empty it is assumed to be bash. +- **Name** - the name of the step running the script. +- **Parameters** - flags passed to the script. Both input and output paths must + be relative to the checkout directory. +- **Script**, the script path relative to the checkout repository. +- **Language**, the script language executable to run the script. If empty it is assumed to be bash. ```json { - "name": "Debug-FlutterMacOS.framework", - "parameters": [ - "--dst", - "out/debug", - "--arm64-out-dir", - "out/ios_debug", - "--simulator-x64-out-dir", - "out/ios_debug_sim", - "--simulator-arm64-out-dir", - "out/ios_debug_sim_arm64" - ], - "script": "flutter/sky/tools/create_full_ios_framework.py", - "language": "python3" + "name": "Debug-FlutterMacOS.framework", + "parameters": [ + "--dst", + "out/debug", + "--arm64-out-dir", + "out/ios_debug", + "--simulator-x64-out-dir", + "out/ios_debug_sim", + "--simulator-arm64-out-dir", + "out/ios_debug_sim_arm64" + ], + "script": "flutter/sky/tools/create_full_ios_framework.py", + "language": "python3" } ``` @@ -400,10 +396,10 @@ artifacts generated by an specific engine build. Global tests currently support two different scenarios: -* flutter/flutter tests with [tester](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipes/engine_v2/tester.py) +- flutter/flutter tests with [tester](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipes/engine_v2/tester.py) recipe. This workflow checks out flutter/flutter to run any of the existing sharded tests using the engine artifacts archived to GCS. -* complicated engine tests that require the outputs from multiple subbuilds +- complicated engine tests that require the outputs from multiple subbuilds with [tester_engine](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipes/engine_v2/tester_engine.py). This workflow checks out [flutter/engine] and operates over the dependencies passed to it using cas. @@ -415,31 +411,31 @@ Framework test example: ```json { - "tests": [ - { - "name": "web-tests-1", - "shard": "web_tests", - "subshard": "1", - "test_dependencies": [ - { - "dependency": "chrome_and_driver", - "version": "version:111.0a" - } - ] - } - ] + "tests": [ + { + "name": "web-tests-1", + "shard": "web_tests", + "subshard": "1", + "test_dependencies": [ + { + "dependency": "chrome_and_driver", + "version": "version:111.0a" + } + ] + } + ] } ``` The property's description is as follows: -* **name** the name that will be assigned to the sub-build. -* **shard** the flutter/flutter shard test to run. The supported shard names can be found -on the flutter framework [test.dart](https://github.com/flutter/flutter/blob/master/dev/bots/test.dart#L244). -* **subshard** one of the accepted subshard values for shard. Sub-shards are defined as part -of the shard implementation, please look at the corresponding shard implementation to find the -accepted values. -* **test_dependencies** a list of [dependencies](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/flutter_deps/api.py#75) +- **name** the name that will be assigned to the sub-build. +- **shard** the flutter/flutter shard test to run. The supported shard names can be found + on the flutter framework [test.dart](https://github.com/flutter/flutter/blob/master/dev/bots/test.dart#L244). +- **subshard** one of the accepted subshard values for shard. Sub-shards are defined as part + of the shard implementation, please look at the corresponding shard implementation to find the + accepted values. +- **test_dependencies** a list of [dependencies](https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/flutter_deps/api.py#75) required for the test to run. Engine test example: @@ -448,30 +444,24 @@ Engine test example: { "tests": [ { - "name": "test: lint android_debug_arm64", - "recipe": "engine_v2/tester_engine", - "drone_dimensions": [ - "device_type=none", - "os=Linux" - ], - "dependencies": [ - "host_debug", - "android_debug_arm64" - ], - "tasks": [ - { - "name": "test: lint android_debug_arm64", - "parameters": [ - "--variant", - "android_debug_arm64", - "--lint-all", - "--shard-id=0", - "--shard-variants=host_debug" - ], - "max_attempts": 1, - "script": "flutter/ci/lint.sh" - } - ] + "name": "test: lint android_debug_arm64", + "recipe": "engine_v2/tester_engine", + "drone_dimensions": ["device_type=none", "os=Linux"], + "dependencies": ["host_debug", "android_debug_arm64"], + "tasks": [ + { + "name": "test: lint android_debug_arm64", + "parameters": [ + "--variant", + "android_debug_arm64", + "--lint-all", + "--shard-id=0", + "--shard-variants=host_debug" + ], + "max_attempts": 1, + "script": "flutter/ci/lint.sh" + } + ] } ] } @@ -479,11 +469,11 @@ Engine test example: The property's description is as follows: -* **name** the name to assign to the sub-build. -* **recipe** the recipe name to use if different than tester. -* **drone_dimensions** a list of strings with key values to select the +- **name** the name to assign to the sub-build. +- **recipe** the recipe name to use if different than tester. +- **drone_dimensions** a list of strings with key values to select the bot where the test will run. -* **dependencies** a list of build outputs required +- **dependencies** a list of build outputs required by the test. These build outputs are referenced by the name of build generating the output. This type of dependency is shared using CAS and the contents are mounted in checkout/src/out. E.g. a build configuration @@ -491,31 +481,31 @@ The property's description is as follows: checkout/src/out/host_engine to CAS and a global test with a `host_engine` dependency will mount the content of host engine in the same location of the bot running the test. -* **tasks** a list of dictionaries representing scripts and parameters to run them. +- **tasks** a list of dictionaries representing scripts and parameters to run them. Example task configuration: ```json { - "name": "test: lint android_debug_arm64", - "parameters": [ - "--variant", - "android_debug_arm64", - "--lint-all", - "--shard-id=0", - "--shard-variants=host_debug" - ], - "max_attempts": 1, - "script": "flutter/ci/lint.sh" + "name": "test: lint android_debug_arm64", + "parameters": [ + "--variant", + "android_debug_arm64", + "--lint-all", + "--shard-id=0", + "--shard-variants=host_debug" + ], + "max_attempts": 1, + "script": "flutter/ci/lint.sh" } ``` The property's description is as follows: -* **name** the name assigned to the step running the script. -* **parameters** a list of parameters passed to the script execution. -* **max_attempts** an integer with the maximum number of runs in case of failure. -* **script** the path relative to checkout/src/ to run. +- **name** the name assigned to the step running the script. +- **parameters** a list of parameters passed to the script execution. +- **max_attempts** an integer with the maximum number of runs in case of failure. +- **script** the path relative to checkout/src/ to run. ### Global Generators @@ -568,10 +558,10 @@ and run the generators locally to validate the fixes. CAS client is required to download the sub-build artifacts. To install it in your machine run the following steps: -* `mkdir $HOME/tools` -* Download and unzip CAS binaries from +- `mkdir $HOME/tools` +- Download and unzip CAS binaries from [https://chrome-infra-packages.appspot.com/p/infra/tools/luci/cas] -* Add $HOME/tools to path and your ~/.bashrc +- Add $HOME/tools to path and your ~/.bashrc #### Gclient engine checkout diff --git a/ci/builders/linux_host_engine.json b/ci/builders/linux_host_engine.json index 97f535aa1a0bb..a158eeac578cf 100644 --- a/ci/builders/linux_host_engine.json +++ b/ci/builders/linux_host_engine.json @@ -24,46 +24,6 @@ ] } }, - { - "archives": [], - "drone_dimensions": [ - "device_type=none", - "os=Linux" - ], - "gclient_variables": { - "download_android_deps": false - }, - "gn": [ - "--runtime-mode", - "debug", - "--unoptimized", - "--prebuilt-dart-sdk", - "--target-dir", - "host_debug_impeller_vulkan" - ], - "name": "host_debug_impeller_vulkan", - "ninja": { - "config": "host_debug_impeller_vulkan", - "targets": [ - "flutter", - "flutter/sky/packages" - ] - }, - "tests": [ - { - "language": "python3", - "name": "Host Tests for host_debug_impeller_vulkan", - "parameters": [ - "--variant", - "host_debug_impeller_vulkan", - "--type", - "impeller-vulkan", - "--engine-capture-core-dump" - ], - "script": "flutter/testing/run_tests.py" - } - ] - }, { "archives": [ { diff --git a/impeller/docs/renderdoc_frame_capture.md b/impeller/docs/renderdoc_frame_capture.md index aad8739d52c07..49792b18e2744 100644 --- a/impeller/docs/renderdoc_frame_capture.md +++ b/impeller/docs/renderdoc_frame_capture.md @@ -17,9 +17,10 @@ ninja -C out/host_debug_unopt/ ![Launch App](assets/renderdoc_frame_capture/launch-app.png) 4. Fill out the following parameters, and click launch: - - executable: `$ENGINE_SRC/out/host_debug_impeller_vulkan/impeller_unittests` (expand `ENGINE_SRC`). - - args: `--gtest_filter="*CanDrawRect/Vulkan*" --enable_playground` - - working dir: `$ENGINE_SRC` (expand `ENGINE_SRC`) + + - executable: `$ENGINE_SRC/out/host_debug_unopt/impeller_unittests` (expand `ENGINE_SRC`). + - args: `--gtest_filter="*CanDrawRect/Vulkan*" --enable_playground` + - working dir: `$ENGINE_SRC` (expand `ENGINE_SRC`) 5. For the frame you wish to capture, press `F12`, you will now be able to see the frame capture and inspect the state.