recover_duts: Increase connection timeout to 15 minutes
Autoserv's SSH parameters allow a connection timeout of up to 12 minutes
before marking a test as failed. Some tests like power_SuspendStress rely
on being able to disrupt connectivity for that long, so we need to make
sure that recover_duts doesn't kill a device during them if it happens
to run at just the wrong time. This patch increases the total tolerated
timeout in check_ethernet.hook to 15 minutes to prevent that problem.
(No correctly working test should ever disrupt the network for longer
than that since the SSH timeout would regularly cause them to fail.)
BUG=chromium:334951
TEST=None
Change-Id: I2a15cd937667c3ca3716397c68c1d72a2802a33a
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/181851
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
diff --git a/recover_duts/hooks/check_ethernet.hook b/recover_duts/hooks/check_ethernet.hook
index ff676cb..94b4146 100755
--- a/recover_duts/hooks/check_ethernet.hook
+++ b/recover_duts/hooks/check_ethernet.hook
@@ -103,12 +103,15 @@
}
main() {
- # Attempt to ping our controlling autotest server over ethernet.
- for i in {0..9}; do
+ # Attempt to ping our controlling autotest server over ethernet. We guarantee
+ # a minimum of 12 minutes network timeout tolerance for tests that disrupt
+ # connectivity with the SSH connection from the autotest server. This timeout
+ # is 15 minutes to make sure it can never fail before that SSH session does.
+ for i in {0..29}; do
if ping_controlling_server; then
return 0
fi
- sleep 5
+ sleep 30
done
# We can't reach our controlling server through any ethernet devices.
