commit | d8cd744eeb2a44db3c89e0579c96738b86ac854c | [log] [tgz] |
---|---|---|
author | Douglas Anderson <dianders@chromium.org> | Thu Jul 30 11:17:34 2020 -0700 |
committer | Commit Bot <commit-bot@chromium.org> | Sun Aug 02 20:41:35 2020 +0000 |
tree | 8a088155dcafa353fcd2dfe4342c99c154d61868 | |
parent | 87eb95e94faeafcfe180ac27a5b6d641d38eece9 [diff] |
installer: Speed install with big root fs and little RAM by chunking When we run chromeos-install we install a single kernel and rootfs from the source to two different partitions on the destination. On systems with enough RAM, we can observe writing the second partition finishes much faster than writing the first partition. That's because the system buffer cache contains the contents after the first copy and doesn't need to read the source again. On systems where the rootfs is bigger than the available buffer cache (RAM), the second write takes just as long as the first write (since both have to read from the source in this case). Specifically disk caches typically throw out the oldest data. In our case, that's exactly the worst possible thing to do and we end up getting a 0% cache hit rate for the 2nd read of the source. We can fix this by breaking the install of the root filesystem into chunks. We'll install chunk 1 twice, then chunk 2 twice, etc. Because each chunk is smaller then it will be in cache for the second write. For now we arbitrarily choose 4 chunks. Why? * Right now the root fs just barely doesn't fit, so even splitting into 4 chunks should give us a bit of buffer. * Splitting into more chunks seems to give slightly worse performance on the handful of tests I did (maybe slightly worse readahead perf because we keep starting and stopping or some other per-transfer overhead?). I tested 5 times at each size and took the average: - 1 chunk => 2 chunk: almost no difference. - 2 chunk => 4 chunk: took an extra 3-4 seconds compared to 1 chunk - 4 chunk => 8 chunk: took an extra 10 seconds compared to 1 chunk At the moment we only support chunking when using "dd" to install. If someone tries to chunk to a partition that's written with UBI or NAND we won't fail but we'll write it all during the first chunk and writing the rest of the chunks will be no-ops. BUG=chromium:1106818 TEST=chromeos-install Change-Id: I9046cc562613b5928382aa0511ad27f05b591d15 Disallow-Recycled-Builds: kukui-arc-r-cq Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform2/+/2330304 Commit-Queue: Douglas Anderson <dianders@chromium.org> Tested-by: Douglas Anderson <dianders@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Auto-Submit: Douglas Anderson <dianders@chromium.org>
This repo holds (most) of the custom code that makes up the Chromium OS platform. That largely covers daemons, programs, and libraries that were written specifically for Chromium OS.
We moved from multiple separate repos in platform/ to a single repo in platform2/ for a number of reasons:
While most projects were merged, not all of them were. Some projects were standalone already (such as vboot), or never got around to being folded in (such as imageloader). Some day those extra projects might get merged in.
Similarly, some projects that were merged in, were then merged back out. This was due to the evolution of the Brillo project and collaboration with Android. That means the AOSP repos are the upstream and Chromium OS carries copies.
Project | Description |
---|---|
arc | Tools/deamons/init-scripts to run ARC |
attestation | Daemon and client for managing remote attestation |
authpolicy | Daemon for integrating with Microsoft Active Directory (AD) domains |
avtest_label_detect | Test tool for OCRing device labels |
biod | Biometrics daemon |
bluetooth | Bluetooth Service and tools |
bootstat | Tools for tracking points in the overall boot process (for metrics) |
buffet | Daemon for reacting to cloud messages |
camera | Chrome OS Camera daemon |
cfm-dfu-notification | CFM specific library for DFU notifications |
chaps | PKCS #11 implementation for TPM 1 devices |
chromeos-common-script | Shared scripts for partitions and basic disk information |
chromeos-config | CrOS unified build runtime config manager |
chromeos-dbus-bindings | Simplifies the implementation of D-Bus daemons and proxies |
chromeos-nvt-tcon-updater | Library for integrating the Novatek TCON firmware updater into a CrOS device |
codelab | Codelab exercise |
common-mk | Common build & test logic for platform2 projects |
crash-reporter | The system crash handler & reporter |
cros-disks | Daemon for mounting removable media (e.g. USB sticks and SD cards) |
cros-fuzz | Support crate for fuzzing rust code |
crosdns | Hostname resolution service for Chrome OS |
crosh | The Chromium OS shell |
croslog | The log manipulation command |
cryptohome | Daemon and tools for managing encrypted /home and /var directories |
cups_proxy | Daemon for proxying CUPS printing request |
debugd | Centralized debug daemon for random tools |
dev-install | Tools & settings for managing the developer environment on the device |
diagnostics | Device telemetry and diagnostics daemons |
disk_updater | Utility for updating root disk firmware (e.g. SSDs and eMMC) |
dlcservice | Downloadable Content (DLC) Service daemon |
easy-unlock | Daemon for handling Easy Unlock requests (e.g. unlocking Chromebooks with an Android device) |
feedback | Daemon for headless systems that want to gather feedback (normally Chrome manages it) |
fitpicker | |
foomatic_shell | Simple shell used by the foomatic-rip package |
glib-bridge | library for libchrome-glib message loop interoperation |
goldfishd | Android Emulator Daemon |
hammerd | Firmware updater utility for hammer hardware |
hardware_verifier | Hardware verifier tool |
hermes | Chrome OS LPA implementation for eSIM hardware support |
hwsec-test-utils | Hwsec-related test-only features |
iioservice | Daemon and libraries that provide sensor data to all processes |
image-burner | Daemon for writing disk images (e.g. recovery) to USB sticks & SD cards |
imageloader | Daemon for mounting signed disk images |
init | CrOS common startup init scripts and boot time helpers |
installer | CrOS installer utility (for AU/recovery/etc...) |
ippusb_bridge | HTTP proxy to IPP-enabled printers |
ippusb_manager | “Service” for ipp-over-usb printing |
kerberos | Daemon for managing Kerberos tickets |
libbrillo | Common platform utility library |
libchromeos-rs | Common platform utility library for Rust |
libchromeos-ui | |
libcontainer | |
libhwsec | Library for the utility functions of all TPM related daemons |
libipp | Library for building and parsing IPP (Internet Printing Protocol) frames |
libmems | Utility library to configure, manage and retrieve events from IIO sensors |
libpasswordprovider | Password Provider library for securely managing credentials with system services |
libtpmcrypto | Library for AES256-GCM encryption with TPM sealed keys |
login_manager | Session manager for handling the life cycle of the main session (e.g. Chrome) |
lorgnette | Daemon for managing attached USB scanners via SANE |
media_perception | Media perception service for select platforms |
memd | Daemon that logs memory-related data and events |
mems_setup | Boot-time initializer tool for sensors |
metrics | Client side user metrics collection |
midis | MIDI service |
mist | Modem USB Interface Switching Tool |
ml | Machine learning service |
ml_benchmark | ML performance benchmark for Chrome OS |
modem-utilities | |
modemfwd | Daemon for managing modem firmware updaters |
mtpd | Daemon for handling Media Transfer Protocol (MTP) with devices (e.g. phones) |
nnapi | Implementation of the Android Neural Networks API |
ocr | Optical Character Recognition (OCR) service for Chrome OS |
oobe_config | Utilities for saving and restoring OOBE config state |
p2p | Service for sharing files between CrOS devices (e.g. updates) |
patchpanel | Platform networking daemons |
permission_broker | |
policy_proto | Build file to compile policy proto file |
policy_utils | Tools and related library to set or override device policies |
power_manager | Userspace power management daemon and associated tools |
print_tools | Various tools related to the native printing system |
regions | |
run_oci | Minimalistic container runtime |
runtime_probe | Runtime probe tool for ChromeOS |
salsa | Touchpad experimentation framework |
screenshot | Tiny command to take a screenshot |
sealed_storage | Library for sealing data to device identity and state |
secure-wipe | Secure disk wipe |
secure_erase_file | Helper tools for securely erasing files from storage (e.g. keys and PII data) |
sepolicy | SELinux policy for Chrome OS |
shill | Chrome OS Connection Manager |
sirenia | Minimalistic init written in Rust |
smbfs | FUSE-based filesystem for accessing Samba / Windows networking shares |
smbprovider | Daemon for connecting Samba / Windows networking shares to the Files.app |
smogcheck | Developer library for working with raw I2C devices |
st_flash | |
storage_info | Helper shell functions for retrieving disk information) |
system-proxy | Daemon for web proxy authentication support on Chrome OS |
system_api | Headers and .proto files etc. to be shared with chromium |
thd | Thermal daemon to help keep systems running cool |
timberslide | Tool for working with EC crashes for reporting purposes |
touch_firmware_calibration | |
touch_keyboard | Utilities for a touch based virtual keyboard |
tpm2-simulator | A software TPM 2.0 implementation (for testing/debugging) |
tpm_manager | Daemon and client for managing TPM setup and operations |
tpm_softclear_utils | Utilities that soft-clear TPM (for testing only) |
trace_events | A framework for adding trace events to your Rust code. |
trim | Service to manage filesystem trim operations in the background |
trunks | Middleware and resource manager for interfacing with TPM 2.0 hardware |
typecd | System daemon to keep track of USB Type C state |
u2fd | U2FHID emulation daemon for systems with secure elements (not TPMs) |
ureadahead-diff | Tool to calculate difference between 2 ureadahead packs |
usb_bouncer | Tools for managing USBGuard white-lists and configuration on Chrome OS |
userfeedback | Various utilities to gather extended data for user feedback reports |
userspace_touchpad | |
virtual_file_provider | |
vm_tools | Utilities for Virtual Machine (VM) orchestration |
vpn-manager | Chrome OS Native L2TP/IPSec VPN Daemon |
webserver | Small web server with D-Bus client backends |
wifi-testbed | Tools for creating a WiFi testbed image |
These projects can be found here: https://chromium.googlesource.com/aosp/platform/