This document contains instructions for Brillo/Android developers interested in running automated integration tests at their desk.
The autotest repository, the test_droid tool, and all tests are checked out in both AOSP and internal Android trees at external/autotest. You need a copy of the autotest source code and an adb binary in your PATH to run tests.
Once you have a local copy of the autotest source, you can easily run tests against a DUT connected directly to your workstation via a USB cable. Please note your first time running test_droid it will download and install a number of required packages locally into your autotest checkout.
Run site_utils/test_droid.py from your autotest checkout to launch a test against a given DUT:
$ ./site_utils/test_droid.py <Test Name>
For example, to run the brillo_WhitelistedGtests test:
$ ./site_utils/test_droid.py brillo_WhitelistedGtests
test_droid can run multiple tests at once:
$ ./site_utils/test_droid.py brillo_WhitelistedGtests brillo_KernelVersionTest
As well as test suites:
$ ./site_utils/test_droid.py suite:brillo-bvt
If you have more than one device connected, you'll have to specify its serial number. First, look it up:
$ adb devices * daemon started successfully * List of devices attached 7d52318 device
Then use it when running:
$ ./site_utils/test_droid.py -s 7d52318 brillo_WhitelistedGtests
Autotest now supports the concept of testbeds, which are multiple devices being controlled by a single test. test_droid supports running these tests by specifying a comma separated list of serials as the test device:
$ adb devices List of devices attached emulator-5554 device 7d52318 device $ ./site_utils/test_droid.py -s emulator-5554,7d52318 testbed_DummyTest
test_droid can run tests against devices connected to a remote server. This requires passwordless SSH access from the workstation to the remote server. If no username is specified, test_droid will try the root and adb users. If using the adb user, make sure it has passwordless sudo rights to run the adb and fastboot commands. You can specify a different user in the remote host name (the same passwordless requirement applies).
The easiest way to set this up is to use the Chrome OS testing keys. Add to your SSH config an entry that looks like the following:
HostName <Remote Server IP or Hostname> Port 9222 User root CheckHostIP no StrictHostKeyChecking no IdentityFile ~/.ssh/testing_rsa Protocol 2
To run the test:
$ ./site_utils/test_droid.py \
-r <Remote Server IP or Hostname> \
<Test Name>
$ ./site_utils/test_droid.py \
-r <User>@<Remote Server IP or Hostname> \
<Test Name>
$ ./site_utils/test_droid.py -r 100.96.48.119 suite:brillo-bvt
Currently Autotest in AOSP is read-only, so you cannot use repo upload to upload code changes. If you do edit or add a new test, make a commit and upload it to https://chromium-review.googlesource.com.
Due to the way Android does its automerging, the AOSP mirror of Autotest cannot be uploaded directly back to the Chromium upstream because the upstream does not contain the merge commits. It will also reject the automerge committer ID. To work around this, if you need to upload a commit to Autotest, you first have to fetch and check out the upstream branch before making your changes:
$ git remote add -t master cros \
https://chromium.googlesource.com/chromiumos/third_party/autotest
$ git fetch cros
$ git checkout cros/master
$ git checkout -b <local branch name>
$ git branch --set-upstream-to=cros/master
Be sure to run pylint on every file you touch:
$ ./utils/run_pylint.py <file name>
Run autotest unittests (which usually requires external packages):
$ utils/build_externals.py # install any missing os packages with apt-get $ utils/unittest_suite.py
Then upload your commit for review:
$ git push cros <local branch name>:refs/for/master
Testing on Brillo/Android (and test_droid by extension) is currently limited to server-side tests, which run on an autotest server and control a Brillo/Android DUT (device under test) via remote command execution. In the context of test_droid, the test logic is running on your development machine, and controlling devices with commands executed via adb.
test_droid does not support the autoupdate end-to-end test. For instructions on how to run this test please refer to the Running Brillo/Android Autoupdate End-to-End Test doc.