Running detachable base fw tests

FAFT is currently not supported for detachable base tests because it does not support two ECs on the same device. Tracking bug. Because of this, the firmware tests for detachable bases were written as scripts that need to be manually run on the DUT. Setting up and running the tests takes approximately 1-2 hours and keyboard EC firmware updates are fairly infrequent (once a year on average pre-FSI, and ~once post-FSI). (Test plan)

Prepare Host and DUT

Tests already come installed in the DUT ChromeOS test image at ‘/usr/local/bin/hammertests’. Before you start, ensure that the test folder exists in the test image.

A list of current devices with detachable base mappings is as follows:

ON HOST

1. Create a chroot from ToT checkout.

2. Obtain the images that will be used in the tests. Obtain the base ec image from GoldenEye/ BCS/ APFE. [Usually the dev will have supplied the image in the testing request bug or indicated where to download it from, because these tests are typically run before RO is locked in the firmware branch].

3. Select build # you want to test.

4. Select board name.

5. Download the ‘firmware_from_source.tar’ file and unzip.

   Change directory into the unzipped folder then cd into the base ec firmware folder (e.g. staff).

   Inside the base ec firmware folder, copy ec.bin into the chroot created in step 1. You can rename this file if you wish to indicate that it’s base firmware e.g. staff_ec.bin.

6. Flash the keyboard EC with the image obtained in step 2, following the steps below:

   a. Expose the servo header in the base keyboard by partially opening the back and/ or cutting out the covering to expose the servo header. Note that for some devices (tablets), removing the entire back cover will result in the keyboard not working as the pogo pins will detach.

   b. Use servo v4 + micro to flash the base firmware. Ensure that the tablet is not powering the base and is completely detached when flashing the firmware using the commands below. Run these commands inside chroot to flash the base ec firmware (this assumes you’re in the top-level directory of the tot checkout and inside chroot)

   E.g. for staff:

   Start servod

     sudo servod -b hammer
   

   Power the base:

     dut-control spi1_vref:pp3300 spi1_buf_en:on spi1_buf_on_flex_en:on
   

   Flash the base with ec image (refer to device: base mapping at the start of this document):

     src/platform/ec/util/flash_ec --board=hammer --image=staff_ec.bin.
   

   You should see the flashing completing successfully.

   Power off the base:

     dut-control spi1_vref:off spi1_buf_en:off spi1_buf_on_flex_en:off
   

   c. Reconnect the base to the tablet and check that keyboard/ trackpad works i.e. type a key and see output on the display, move the trackpad and see output on the display.

7. Generate the images needed for tests, from the base ec image obtained in step 5 following the steps below:

   a. Inside chroot: Run the shell script gen_test_images.sh that generates the necessary images, supplying ec.bin as the first argument for the BOARD name (where board is the name of the detachable base) and the IMAGE name as the second argument e.g. for staff:

   ./src/third_party/autotest/files/server/cros/faft/gen_test_images.sh staff staff_ec.bin
   

   The above script needs to run inside chroot of the DUT host

   Ensure that the ec.bin image used above is the same image running on the base ec of the DUT. If they are different, RW verification will fail during testing.

   You should then see the following 17 images created:

   1. EC_RW.bin
   2. staff_corrupt_first_byte.bin
   3. staff.dev.hex
   4. staff.dev.rb9
   5. key.vbprik2
   6. staff_corrupt_first_byte.bin.hex
   7. staff.dev.rb0
   8. staff.dev.rb9.hex
   9. key.vbpubk2
   10. staff_corrupt_last_byte.bin
   11. staff.dev.rb0.hex
   12. staff.bin
   13. staff_corrupt_last_byte.bin.hex
   14. staff.dev.rb1
   15. staff.bin.hex
   16. staff.dev
   17. staff.dev.rb1.hex

   There is one image file that must be manually included in the images folder. For first-time firmware qualification e.g. of a new keyboard, include a copy of the firmware and name it ‘BASE_NAME_older.bin’ where BASE_NAME is the keyboard name e.g. staff_older.bin. For revised firmware qualifications i.e. qualifying new version of RO/RW but not the first RO/RW, include an image of the older firmware and also name it ‘BASE_NAME_older.bin’. The image 'BASE_NAME_older.bin'is used to test that RW updates can happen with the new firmware. In the case of first time firmware, the tests still check that RW region is updatable even though the older image is the same as the newer image. b. Copy the new images folder in your chroot into the DUT from the host via ssh (ensure ssh test keys are added to the shell window first):

   scp -r images root@dut.ip.address:/usr/local/bin/hammertests.
   

ON DUT

Flash the DUT with a test ChromeOS image. The test image already has the base tests in the folder '/usr/local/bin/hammertests’. You should see the following tests.

1. flash_wrong_address.py

2. rb_protection.py

3. rb_rw_protected.py

4. ro_boot_valid_rw.py

5. ro_stay_ro.py

6. ro_update_rw.py

7. rw_no_update_ro.py

8. transfer_touchpad_works.py

9. verify_pairing.py

You should also see the following control files

1. hammertests_control.py

2. hammertests_control_rb.py

3. hammertests_control_tp.py

Disable verified boot on DUT so that hammerd does not make it’s own decisions. The tests disable hammerd but it will be enabled on reboot. We need it to be disabled persistently by renaming the ‘hammerd’ file to ‘hammerd.out’ so that it doesn’t execute. Run the following commands on the DUT console:

/usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 2
/usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 4

Reboot the DUT then rename hammerd

mv /usr/bin/hammerd /usr/bin/hammerd.out

TEST EXECUTION

To run the tests, simply run each of the control files in the DUT from the /usr/local/bin/hammertests directory. You can do this directly on the DUT or ssh via the host. E.g.

./hammertests_control.py

Each control file runs a number of tests, and creates log folders corresponding to each of the tests outlined above. Some tests have several iterations.

Image that should be flashed on DUT for hammertests_control.py: $BASE_NAME_ec.bin (locked) This locked image should be supplied by the dev who requests the testing and attached in the testing request bug.

Image that should be flashed on DUT for hammertests_control_rb.py: $BASE_NAME.dev.rb1 (rollback version = 1) This image is in the ‘images’ folder which is generated by the script gen_test_images.sh

Image that should be flashed on DUT for hammertests_control_tp.py: $BASE_NAME_ec.bin (unlocked) This unlocked image should be supplied by the dev who requests the testing and attached in the testing request bug.

After running all the tests, attach the log files to the request bug.

Manual Tests

RO cannot update RW with dev signed image:

• Flash the image staff.dev generated by ./genimages_staff.sh and verify on the ec console that RW update fails due to failed signature verification.

Keyboard:

Run command evtest on DUT and check that each key works. (https://docs.google.com/document/d/1tA8QknZ37TBrZelneOWtzHtPNGbC5aDDel6PeOV5tZQ/edit#heading=h.vagnagl3dytt).

Keyboard backlight works

• Can control backlight using:

  echo $VALUE > /sys/class/leds/hammer::kbd_backlight/brightness
  

• Or via powerd:

  backlight_dbus_tool --increase_keyboard
  

Keyboard backlight not advertised on SKUs where backlight not available

Implemented here: https://issuetracker.google.com/67722756.

TODO(drinkcat): give system path to keyboard backlight to distinguish devices that have backlight support from those that don't

Trackpad:

Touchpad works

Needed:

• Base running RW version b under test with trackpad FW v2

Steps:

• Manual testing

• Check that touchpad works

• 1 to 5 finger swipes

• Click using touchpad works

I2C passthrough interface does not work

(to be active only when WP is off, for Elan usage)

• WP on, cannot update the touchpad FW over I2C

  touchpad_updater -f FW_IMAGE
  

• WP off (through servo or physically remove the screw), can update the touchpad FW over I2C:

  touchpad_updater -f FW_IMAGE
  

Trackpad dimensions are correct

No errors of this kind in EC console:

[1.430530 reset rv 0 buf=0000]

[1.432640 max=3206/1832 width=124/124 adj=0 dpi=800/800]

[1.432948 *** TP mismatch!]

[1.433585 elan_tp_init:0]

See https://issuetracker.google.com/67982128 for example.

We only really need to test for version update when we release new TP firmware