mmm_donut: If tasks die while munching, die immediately

Recently I noticed that the OOM killer sometimes kills all of our
children while in the munching stage but before memory fills up. That
leaves the parent process waiting in a loop. Check for death of the
children to fix that.

BUG=None
TEST=mmm_donut.py on kukui-kodama after 1000 in min_ttl_ms

Change-Id: Ib0b5fab1414c21b50cece7e7b90c1cb4790aff3e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/microbenchmarks/+/4482047
Reviewed-by: Vovo Yang <vovoy@chromium.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
Commit-Queue: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Brian Geffon <bgeffon@chromium.org>
diff --git a/mmm_donut.py b/mmm_donut.py
index e0c6edb..7d5c73c 100755
--- a/mmm_donut.py
+++ b/mmm_donut.py
@@ -498,6 +498,11 @@
                 continue
             task_num = done_queue.get()
             cmd_queues[task_num].put('munch')
+
+            for task_num, task in enumerate(tasks):
+                if not task.is_alive():
+                    raise WorkerDeadError(task_num)
+
         _end_stage('munch', tasks, done_queue, cmd_queues)
 
         print('Tasting for %d seconds; taste %d MB at a time.' %