Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions shell/platform/embedder/fixtures/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# `platform/embedder/fixtures`

The files in this directory are golden-file outputss of [`tests`](../tests),
but lack a simple way to be re-generated.

For example, here is what a failure might look like on CI:

```txt
[0;32m[ RUN ] EmbedderTest.VerifyB143464703WithSoftwareBackend
[ERROR:flutter/shell/platform/embedder/tests/embedder_unittests_util.cc(199)] Image did not match expectation.
Expected:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/expectation_verifyb143464703_soft_noxform.png
Got:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/actual_verifyb143464703_soft_noxform.png

../../flutter/shell/platform/embedder/tests/embedder_unittests.cc:1335: Failure
Value of: ImageMatchesFixture("verifyb143464703_soft_noxform.png", rendered_scene)
Actual: false
Expected: true
[ FAILED ] EmbedderTest.VerifyB143464703WithSoftwareBackend (8077 ms)
[----------] 1 test from EmbedderTest (8080 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (8080 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] EmbedderTest.VerifyB143464703WithSoftwareBackend

1 FAILED TEST
[13/296] EmbedderTest.VerifyB143464703WithSoftwareBackend returned/aborted with exit code 1 (8226 ms)
[14/296] EmbedderTest.VerifyB143464703WithSoftwareBackend (8484 ms)
[INFO:test_timeout_listener.cc(76)] Test timeout of 300 seconds per test case will be enforced.
Note: Google Test filter = EmbedderTest.VerifyB143464703WithSoftwareBackend
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EmbedderTest
[ DISABLED ] EmbedderTest.DISABLED_CanLaunchAndShutdownMultipleTimes
[ RUN ] EmbedderTest.VerifyB143464703WithSoftwareBackend
[ERROR:flutter/shell/platform/embedder/tests/embedder_unittests_util.cc(199)] Image did not match expectation.
Expected:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/expectation_verifyb143464703_soft_noxform.png
Got:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/actual_verifyb143464703_soft_noxform.png

../../flutter/shell/platform/embedder/tests/embedder_unittests.cc:1335: Failure
Value of: ImageMatchesFixture("verifyb143464703_soft_noxform.png", rendered_scene)
Actual: false
Expected: true
[ FAILED ] EmbedderTest.VerifyB143464703WithSoftwareBackend (8348 ms)
[----------] 1 test from EmbedderTest (8350 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (8350 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] EmbedderTest.VerifyB143464703WithSoftwareBackend
```

In order to update `verifyb143464703_soft_noxform.png`:

```shell
# The examples below assume:
# $ENGINE = /path/to/engine/src
# $TARGET = /path/to/engine/src/out/{{host_you_want_to_build}}

# 1. Make sure you have built the engine:
$ ninja -j1000 -C $ENGINE/out/$TARGET

# 2. Run the test locally (assuming you have built the engine).
$ $ENGINE/out/$TARGET/embedder_unittests*

# Or, to run just a single test:
$ $ENGINE/out/$TARGET/embedder_unittests --gtest_filter="EmbedderTest.VerifyB143464703WithSoftwareBackend"

# Or, a suite of tests:
$ $ENGINE/out/$TARGET/embedder_unittests --gtest_filter="EmbedderTest.*"

# 3. Now, copy the output to the golden file:
$ cp \
$ENGINE/out/$TARGET/gen/flutter/shell/platform/embedder/assets/expectation_verifyb143464703_soft_noxform.png \
$ENGINE/flutter/shell/platform/embedder/fixtures/verifyb143464703_soft_noxform.png
```

⚠️ **WARNING**: Some of the golden tests do not run on non-Linux OSes, which
means its not currently possible to re-generate them on non-Linux OSes
(<https://github.com/flutter/flutter/issues/53784>). So uh, setup a Linux VM
or find a friend with a Linux machine.