recover_duts: ping SSH clients to check connectivity

recover_duts' check_ethernet.hook tests for network connectivity by
pinging the system's default gateway on any device that starts with
"eth". This is obviously a problem when the default route does not use
such a device, as it can happen during some autotests (like
power_LoadTest) that use the cros/backchannel.py script to switch their
default route to wireless.

What we really want to know is if the autotest server is still reachable
and can control the device. Autotest uses an SSH connection to control
its client, so this patch changes recover_duts to search for an active
SSH connection and ping that remote client first. This will work with
backchannel tests as it specifically ensures to keep routes to subnets
that have active SSH connections on the wired device. If no active SSH
connection is detected, the hook will fall back to the old method.

BUG=chromium-os:36454
TEST=Submit this patch and rejoice as all the power_LoadTest runs in the
lab magically start to work again for the first time in months.

Change-Id: I0066ccebe7a2f77744f2104555160c08feb7c108
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39036
Reviewed-by: Chris Sosa <sosa@chromium.org>
1 file changed