check_ethernet: promote rescan_usb_hubs to recovery_method

Today, we may run rescan_usb_hubs a number of times just because it's
part of the find_ethernet_interfaces() function. This isn't really a
good idea for cases that aren't recoverable -- it wastes a bunch of time
and caues confusing "bouncy" behavior. We can improve this by sharing
the 'recovery_method' logic, which
(a) tries the recovery action
(b) checks whether that action did anything useful
(c) if so, retry connectivity diagnostics; if not, try the next method

This pattern matches what we want for rescan_usb_hubs anyway: just try
it once, if there were no operational links; don't re-try it several
times as an implicit part of looking for devices.

This eliminates any logic in find_ethernet_devices(), so we just alias
with search_devices().

A few additions to rescan_usb_hubs() along the way:
 * re-check whether there are any devices (search_devices()) after
   running, to determine our "success"; poll over 15 seconds -- in
   addition to kernel hub enumeration, it can take some time for
   usb_guard to authorize the device
 * factor out some delays until after we iterate over all hubs
 * add a sanity check in case the "${usbhub}/... globbing doesn't match

TEST=manually-induced failures (e.g., delete interface), see how script

Change-Id: I118db269002ebba9267446bfbbdba7b3bf6e938e
Tested-by: Brian Norris <>
Commit-Ready: Brian Norris <>
Legacy-Commit-Queue: Commit Bot <>
Reviewed-by: Grant Grundler <>
1 file changed