Flutter Engine
The Flutter Engine
|
This is documentation on setting up RBE for building the Flutter engine. It is expected to work for Googlers only on corp Linux, macOS, and Windows hosts, including cloudtop instances.
The first step is to add an entry to the .gclient
file. The entry to add is "use_rbe": True
in the custom_vars
section. It should look like this:
After making this edit, you must be authenticated as a Googler by CIPD so that the RBE configurations can be downloaded from the flutter_internal
CIPD bucket:
After authentication successfully, run gclient sync -D
.
In the engine repo, all RBE builds must be initiated through the et
tool whose entrypoint is the script //flutter/bin/et
. This is so that the local RBE proxy is correctly initialized and shut down around invocations of ninja
.
Before running an RBE build, you must be authenticated with the Google cloud project that owns the RBE worker pool. A gcloud SDK is checked out with the engine repo under e.g. //flutter/buildtools/mac-arm64/gcloud
. (Replace mac-arm64
with the directory name that is correct for your host platform.)
The gcloud
tool in this SDK must be on your path. The tool lives under //flutter/buildtools/mac-arm64/gcloud/bin
, which is the path to add to your PATH
environment variable. Alternatively, you can get the gcloud SDK on your path by installing it on your system by following the instructions at https://cloud.google.com/sdk/docs/install.
On macOS, before running the gcloud
command ensure that python3
is on your path, and does not come from e.g. homebrew. The command which python3
should return /usr/bin/python3
.
If you get an error from bootstrap
about not being able to find Application Default Credentials
you may need to execute the following to create the default credentials:
The builds available to the et
tool are those specified by the build configuration json files under //flutter/ci/builders
. A list of builds suitable for local development can also be printed by running et help build
. The list of all builds can be printed by running et help build --verbose
. Builds in the verbose list prefixed with ci/
or ci\
are exactly the builds run on CI with the same GN flags and Ninja targets.
To run a build, pass the name of the configuration to the et build
command:
If RBE is working correctly, you should see logs like the following:
To disable RBE in a build, pass the flag --no-rbe
to the build
command.
Since LTO is slow and rarely useful in local builds, et
disables it by default in all builds, even when it is specified by a build configuration. To enable it, pass the --lto
flag to the build
command.
Beyond enabling/disabling RBE and LTO, the build
command does not currently support customizing builds at the command line.
If you need custom GN flags or Ninja targets, then this can be achieved by making local edits to the build configuration json files in your checkout. In particular, the file //flutter/ci/builders/local_engine.json
is intended to contain builds that are commonly used for local engine development.
If a configuration does not exist there, and will be needed by multiple people over a long period of time, consider checking in the new configuration so that it will be built on CI, which will keep the RBE cache warm for fast local builds.
On the other hand, if a build is intended to produce artifacts or run tests on CI, then it should not go in local_engine.json
. Instead, it should go in the json file appropriate for its purpose. There are many examples available, so following suit is likely your best bet, but if you are unsure, ask in the hackers-engine
Discord.
RBE is sensitive to what paths in compile commands look like. In particular, all paths in compile commands must be relative paths, and those relative paths must all resolve to files within the engine source tree. In practice, this means that the GN function rebase_path()
should only lack a second parameter when its result won’t be used in a compile or link command. There are unfortunately many examples in the tree where rebase_path()
lacks a second parameter. However to be on the safe side a good rule of thumb is: do not add new usages of the single parameter rebase_path()
.
RBE builds can be slow for a few different reasons. The most common reason is likely to be that the remote caches are cold. When the caches are warm, a compile step consists only of downloading the compiled TU from the cache. When the caches are cold, the remote workers must run the compilation commands, which takes more time. If the worker pool is overloaded, compile commands may run locally instead, which will also be slower.
RBE builds can also be slow if your network connection is bandwidth constrained. Anecdotally, even with a warm cache, I have noticed slow builds from home due to RBE saturating my low-tier Comcast Business connection.
For developers on a macOS host device, ensure that you're using the same version of Xcode as is in use on the bots. The value of "Build version" returned by xcodebuild -version
should match the sdk_version
value set in ci/builders/local_engine.json
for the build you're running.
For Googlers on a corp macOS device, both RBE and non-RBE builds can be slow due to various background and monitoring processes running. See here for how to disable some of them. You should also disable Spotlight scanning of the engine source directory as described here.
When RBE builds are slow, non-RBE builds may be faster, especially incremental builds. You can disable remote builds without invalidating your existing build by setting the environment variable RBE_exec_strategy=local
.
[!WARNING] Since
et
will start and stop the local RBE proxy while performing a build, the following command will only work when a build is running.
The status of the local RBE proxy can be queried with the following command
It will give output describing the number of actions completed and in progress, and the number of remote executions, local executions, and remote cache hits.
For example:
The logs for RBE live under the system /tmp
folder in the files /tmp/reproxy. {INFO,WARNING,ERROR}
and /tmp/bootstrap.{INFO,WARNING,ERROR}
.
In CI runs, the RBE logs are also available by following the links as in the screenshot below under teardown remote execution > collect rbe logs
:
These logs can be useful to highlight failures in RBE’s dependency analysis. The dependency analysis can fail for a variety of reasons, but a common one during development is likely to be that the source file is really malformed somehow. This can be debugged by doing a local build with RBE turned off.