Flutter Engine
The Flutter Engine
platform/embedder/fixtures

The files in this directory are golden-file outputss of tests, but lack a simple way to be re-generated.

For example, here is what a failure might look like on CI:

[0;32m[ RUN ] ␛[mEmbedderTest.VerifyB143464703WithSoftwareBackend
[ERROR:flutter/shell/platform/embedder/tests/embedder_unittests_util.cc(199)] Image did not match expectation.
Expected:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/expectation_verifyb143464703_soft_noxform.png
Got:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/actual_verifyb143464703_soft_noxform.png
../../flutter/shell/platform/embedder/tests/embedder_unittests.cc:1335: Failure
Value of: ImageMatchesFixture("verifyb143464703_soft_noxform.png", rendered_scene)
Actual: false
Expected: true
␛[0;31m[ FAILED ] ␛[mEmbedderTest.VerifyB143464703WithSoftwareBackend (8077 ms)
␛[0;32m[----------] ␛[m1 test from EmbedderTest (8080 ms total)
␛[0;32m[----------] ␛[mGlobal test environment tear-down
␛[0;32m[==========] ␛[m1 test from 1 test suite ran. (8080 ms total)
␛[0;32m[ PASSED ] ␛[m0 tests.
␛[0;31m[ FAILED ] ␛[m1 test, listed below:
␛[0;31m[ FAILED ] ␛[mEmbedderTest.VerifyB143464703WithSoftwareBackend
[13/296] EmbedderTest.VerifyB143464703WithSoftwareBackend returned/aborted with exit code 1 (8226 ms)
[14/296] EmbedderTest.VerifyB143464703WithSoftwareBackend (8484 ms)
[INFO:test_timeout_listener.cc(76)] Test timeout of 300 seconds per test case will be enforced.
␛[0;33mNote: Google Test filter = EmbedderTest.VerifyB143464703WithSoftwareBackend
␛[m␛[0;32m[==========] ␛[mRunning 1 test from 1 test suite.
␛[0;32m[----------] ␛[mGlobal test environment set-up.
␛[0;32m[----------] ␛[m1 test from EmbedderTest
␛[0;33m[ DISABLED ] ␛[mEmbedderTest.DISABLED_CanLaunchAndShutdownMultipleTimes
␛[0;32m[ RUN ] ␛[mEmbedderTest.VerifyB143464703WithSoftwareBackend
[ERROR:flutter/shell/platform/embedder/tests/embedder_unittests_util.cc(199)] Image did not match expectation.
Expected:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/expectation_verifyb143464703_soft_noxform.png
Got:/b/s/w/ir/cache/builder/src/out/host_debug_unopt/gen/flutter/shell/platform/embedder/assets/actual_verifyb143464703_soft_noxform.png
../../flutter/shell/platform/embedder/tests/embedder_unittests.cc:1335: Failure
Value of: ImageMatchesFixture("verifyb143464703_soft_noxform.png", rendered_scene)
Actual: false
Expected: true
␛[0;31m[ FAILED ] ␛[mEmbedderTest.VerifyB143464703WithSoftwareBackend (8348 ms)
␛[0;32m[----------] ␛[m1 test from EmbedderTest (8350 ms total)
␛[0;32m[----------] ␛[mGlobal test environment tear-down
␛[0;32m[==========] ␛[m1 test from 1 test suite ran. (8350 ms total)
␛[0;32m[ PASSED ] ␛[m0 tests.
␛[0;31m[ FAILED ] ␛[m1 test, listed below:
␛[0;31m[ FAILED ] ␛[mEmbedderTest.VerifyB143464703WithSoftwareBackend
static BlurTest tests[]
Definition: BlurTest.cpp:84
#define TEST(S, s, D, expected)
Definition: FitsInTest.cpp:15
#define test(name)
static bool b
struct MyStruct s
def match(bench, filt)
Definition: benchmark.py:23
exit(kErrorExitCode)
static dart::SimpleHashMap * environment
Definition: gen_snapshot.cc:59
static TTSTestCase Failure(const TTSTestCase &original)
testing::EmbedderTest EmbedderTest
bool ImageMatchesFixture(const std::string &fixture_file_name, const sk_sp< SkImage > &scene_image)
DEF_SWITCHES_START aot vmservice shared library Name of the *so containing AOT compiled Dart assets for launching the service isolate vm snapshot The VM snapshot data that will be memory mapped as read only SnapshotAssetPath must be present isolate snapshot The isolate snapshot data that will be memory mapped as read only SnapshotAssetPath must be present cache dir Path to the cache directory This is different from the persistent_cache_path in embedder which is used for Skia shader cache icu native lib Path to the library file that exports the ICU data vm service The hostname IP address on which the Dart VM Service should be served If not defaults to or::depending on whether ipv6 is specified vm service A custom Dart VM Service port The default is to pick a randomly available open port disable vm Disable the Dart VM Service The Dart VM Service is never available in release mode disable vm service Disable mDNS Dart VM Service publication Bind to the IPv6 localhost address for the Dart VM Service Ignored if vm service host is set endless trace Enable an endless trace buffer The default is a ring buffer This is useful when very old events need to viewed For during application launch Memory usage will continue to grow indefinitely however Start app with an specific route defined on the framework flutter assets Path to the Flutter assets directory enable service port Allow the VM service to fallback to automatic port selection if binding to a specified port fails trace Trace early application lifecycle Automatically switches to an endless trace buffer trace skia Filters out all Skia trace event categories except those that are specified in this comma separated list dump skp on shader Automatically dump the skp that triggers new shader compilations This is useful for writing custom ShaderWarmUp to reduce jank By this is not enabled to reduce the overhead purge persistent cache
Definition: switches.h:191
CanvasImage Image
Definition: dart_ui.cc:55
DEF_SWITCHES_START aot vmservice shared library Name of the *so containing AOT compiled Dart assets for launching the service isolate vm snapshot The VM snapshot data that will be memory mapped as read only SnapshotAssetPath must be present isolate snapshot The isolate snapshot data that will be memory mapped as read only SnapshotAssetPath must be present cache dir Path to the cache directory This is different from the persistent_cache_path in embedder which is used for Skia shader cache icu native lib Path to the library file that exports the ICU data vm service The hostname IP address on which the Dart VM Service should be served If not set
Definition: switches.h:76
Definition: gen.py:1
def timeout(deadline, cmd)
SkScalar w
#define ERROR(message)
Definition: elf_loader.cc:260
#define FAILED(hr)

In order to update verifyb143464703_soft_noxform.png:

# The examples below assume:
# $ENGINE = /path/to/engine/src
# $TARGET = /path/to/engine/src/out/{{host_you_want_to_build}}
# 1. Make sure you have built the engine:
$ ninja -j1000 -C $ENGINE/out/$TARGET
# 2. Run the test locally (assuming you have built the engine).
$ $ENGINE/out/$TARGET/embedder_unittests*
# Or, to run just a single test:
$ $ENGINE/out/$TARGET/embedder_unittests --gtest_filter="EmbedderTest.VerifyB143464703WithSoftwareBackend"
# Or, a suite of tests:
$ $ENGINE/out/$TARGET/embedder_unittests --gtest_filter="EmbedderTest.*"
# 3. Now, copy the output to the golden file:
$ cp \
$ENGINE/out/$TARGET/gen/flutter/shell/platform/embedder/assets/expectation_verifyb143464703_soft_noxform.png \
$ENGINE/flutter/shell/platform/embedder/fixtures/verifyb143464703_soft_noxform.png

⚠️ WARNING: Some of the golden tests do not run on non-Linux OSes, which means its not currently possible to re-generate them on non-Linux OSes (https://github.com/flutter/flutter/issues/53784). So uh, setup a Linux VM or find a friend with a Linux machine.