Google Git
Sign in
cos / mirrors / cros / chromiumos / bazel / refs/heads/stabilize-15633.69.B
commit2312faa3bc16920c620ab123f079eadfcfb5622d[log] [tgz]
authorShuhei Takahashi <nya@chromium.org>Mon Sep 11 20:10:28 2023 +0900
committerChromeos LUCI <chromeos-scoped@luci-project-accounts.iam.gserviceaccount.com>Mon Oct 02 00:50:51 2023 +0000
tree1e61677fb52c44c50bbf609693800524d6e8e2cd
parentbc3929819d46e66497003a62f3744829eaedd9f6 [diff]
Reland(2): "Enable the fast package installation method by default"

This is a reland of commit aad18e188e6c315e79c3cd9f77b3821c5619da87

b/300130101 should have been fixed by earlier changes in the stack.

Original change's description:
> Reland "Enable the fast package installation method by default"
>
> This is a reland of commit 408001e5e8016218bd89ca439b410f5ea0310ebb
>
> We've fixed b/300130101.
>
> Original change's description:
> > Enable the fast package installation method by default
> >
> > BUG=b:280673549
> > TEST=BOARD=amd64-generic bazel build @portage//sys-apps/attr
> >
> > Change-Id: I78498c72e5c02a4d5a0821fef0a1fba1313c7630
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/bazel/+/4855872
> > Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>
> > Commit-Queue: Shuhei Takahashi <nya@chromium.org>
> > Tested-by: Shuhei Takahashi <nya@chromium.org>
>
> BUG=b:280673549
> BUG=b:300130101
>
> Change-Id: I6e7a65c418ecc10100204c46d3b0e56a48311eda
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/bazel/+/4878897
> Auto-Submit: Shuhei Takahashi <nya@chromium.org>
> Tested-by: Shuhei Takahashi <nya@chromium.org>
> Commit-Queue: Ryo Hashimoto <hashimoto@chromium.org>
> Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>

BUG=b:280673549
BUG=b:300130101

Cq-Depend: chromium:4896616
Cq-Depend: chromium:4898078
Change-Id: Iab8d9ca06d9211fcf803ce339c3a995dc72c04be
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/bazel/+/4896618
Tested-by: Shuhei Takahashi <nya@chromium.org>
Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>
Commit-Queue: Shuhei Takahashi <nya@chromium.org>
2 files changed
tree: 1e61677fb52c44c50bbf609693800524d6e8e2cd
  1. bash/
  2. bazelrcs/
  3. build_defs/
  4. cc/
  5. cros_pkg/
  6. go/
  7. images/
  8. module_extensions/
  9. platforms/
  10. portage/
  11. python/
  12. repo_defs/
  13. rust/
  14. transitions/
  15. workspace_root/
  16. .gitignore
  17. BUILD.bazel
  18. design.md
  19. DIR_METADATA
  20. go.mod
  21. go.sum
  22. OWNERS
  23. OWNERS.clover
  24. PRESUBMIT.cfg
  25. README.md
README.md

ChromeOS Bazelification

This repository provides the implementation to build ChromeOS with Bazel.

Checking out

For building ChromeOS with Bazel, use the following repo command to check out with a few additional repositories.

$ mkdir ~/chromiumos
$ cd ~/chromiumos
$ repo init -u https://chrome-internal.googlesource.com/chromeos/manifest-internal -g default,bazel -b main
$ repo sync -c -j 4
$ cd src

Unless otherwise specified, examples in this doc assume that your current directory is ~/chromiumos/src.

Building packages

Now you're ready to start building. To build a single Portage package, e.g. sys-apps/attr:

$ BOARD=amd64-generic bazel build @portage//sys-apps/attr

To build all packages included in the ChromeOS base image:

$ BOARD=amd64-generic bazel build @portage//virtual/target-os:package_set
Please make sure that which bazel prints a path under your depot_tools checkout. The wrapper script provided by depot_tools performs additional tasks besides running the real bazel executable.

Inside CrOS SDK chroot

Inside CrOS SDK chroot (i.e. the build environment you enter with cros_sdk command), you should be able to run the same command except that you need to use /mnt/host/source/chromite/bin/bazel instead of bazel.

For example:

$ BOARD=amd64-generic /mnt/host/source/chromite/bin/bazel build @portage//sys-apps/attr

You can also run build_packages --bazel --board=$BOARD to run build_packages with Bazel.

Building images

We have the following targets to build images:

  • //bazel/images:chromiumos_minimal_image: Minimal image that contains sys-apps/baselayout and sys-kernel/chromeos-kernel only.
  • //bazel/images:chromiumos_base_image: Base image.
  • //bazel/images:chromiumos_dev_image: Dev image.
  • //bazel/images:chromiumos_test_image: Test image.
For historical reasons, the output file name of the dev image is chromiumos_image.bin, not chromiumos_dev_image.bin.

As of June 2023, we primarily test our builds for amd64-generic and arm64-generic. Please file bugs if images don't build for these two boards. Other boards may or may not work (yet).

Building a ChromeOS image takes several hours. Most packages build in a few minutes, but there are several known heavy packages, such as chromeos-base/chromeos-chrome that takes 2-3 hours. You can inject prebuilt binary packages to bypass building those packages. See Injecting prebuilt binary packages for more details.

After building an image, you can use cros_vm command available in CrOS SDK to run a VM locally. Make sure to copy an image out from bazel-bin as it's not writable by default.

$ cp bazel-bin/bazel/images/chromiumos_base_image.bin /tmp/
$ chmod +w /tmp/chromiumos_base_image.bin
$ chromite/bin/cros_vm --start --board=amd64-generic --image-path /tmp/chromiumos_base_image.bin

You can use VNC viewer to view the VM.

$ vncviewer localhost:5900

You can also use cros_vm command to stop the VM.

$ chromite/bin/cros_vm --stop

Testing your change

The run_tests.sh script runs currently available tests:

$ portage/tools/run_tests.sh

Optionally, you can skip running some tests by specifying some of the following environment variables when running run_tests.sh: SKIP_CARGO_TESTS=1, SKIP_BAZEL_TESTS=1, SKIP_PORTAGE_TESTS=1.

Directory structure

  • portage/ ... for building Portage packages (aka Alchemy)
    • bin/ ... executables
    • common/ ... common Rust/Go libraries
    • build_defs/ ... build rule definitions in Starlark
    • repo_defs/ ... additional repository definitions
      • prebuilts/ ... defines prebuilt binaries
    • sdk/ ... defines the base SDK
    • tools/ ... misc small tools for development
  • images/ ... defines ChromeOS image targets
  • workspace_root/ ... contains various files to be symlinked to the workspace root, including WORKSPACE.bazel and BUILD.bazel

Misc Memo

Debugging a failing package

TODO: Fix the ability to get the build working directory. The method described here is no longer working.

If a package is failing to build, it‘s sometimes useful to view the package’s work directory. To do this run:

bazel build --sandbox_debug //your/ebuild

In the build output you will see a cd into the execroot:

cd /home/rrangel/.cache/bazel/_bazel_rrangel/ca19c0757f7accdebe9bbcbd2cb0838e/sandbox/linux-sandbox/842/execroot/__main__

This directory will contain a directory called build_package.*. It contains all the artifacts that were generated while building the package.

Build logs can be found in:

scratch/diff/build/arm64-generic/tmp/portage/logs/

The package work dir can be found in:

scratch/diff/build/<board>/tmp/portage/<category>/<package>-<version>

Debugging an ephemeral CrOS chroot

Sometimes you want to enter an ephemeral CrOS chroot where a package build is failing to inspect the environment interactively.

To enter an ephemeral CrOS chroot, run the following command:

$ BOARD=arm64-generic bazel run @portage//sys-apps/attr:debug -- --login=after

This command will give you an interactive shell after building a package. You can also specify other values to --login to choose the timing to enter an interactive console:

  • --login=before: before building the package
  • --login=after: after building the package
  • --login=after-fail: after failing to build the package

Injecting prebuilt binary packages

In the case your work is blocked by some package build failures, you can workaround them by injecting prebuilt binary packages via command line flags.

For every ebuild target under @portage//internal/packages/..., an associated string flag target is defined. You can set a gs:// URL of a prebuilt binary package to inject it.

For example, to inject a prebuilt binary packages for chromeos-chrome, you can set this option:

--@portage//internal/packages/stage1/target/board/chromiumos/chromeos-base/chromeos-chrome:114.0.5715.0_rc-r2_prebuilt=gs://chromeos-prebuilt/board/amd64-generic/postsubmit-R114-15427.0.0-49533-8783437624917045025/packages/chromeos-base/chromeos-chrome-114.0.5715.0_rc-r2.tbz2

You can run generate_chrome_prebuilt_config.py to generate the prebuilt config for the current version of chromeos-chrome.

% BOARD=amd64-generic portage/tools/generate_chrome_prebuilt_config.py

We have several named config groupings in prebuilts.bazelrc that define typical options to inject prebuilts. You can specify --config to use them.

  • --config=prebuilts/arm64-generic: Injects prebuilt binary packages needed to build arm64-generic images.

Extracting binary packages

In case you need to extract the contents of a binary package so you can easily inspect it, you can use the xpak split CLI.

bazel run //bazel/portage/bin/xpak:xpak -- split --extract libffi-3.1-r8.tbz2 libusb-0-r2.tbz2

Running tests on every local commit

If you'd like to run the tests every time you commit, add the following. You can skip it with git commit --no-verify.

cd ~/chromiumos/src/bazel
ln -s ../../../../../src/bazel/portage/tools/run_tests.sh .git/hooks/pre-commit