CHROMIUM: scheduler: log packages that take a long time to build

If a package gets stuck or takes a long time, it can be hard to see
or notice as the output from portage just stops as it waits.  Lets
add a periodic check that logs a "Still building" message for every
package that takes more than 30min to complete, and log it every
30min after that.

So if a package takes less than 30min, there is no difference with
today's output.  But if it takes longer, we'll get one message at
every 30min interval.

Even further, if a package takes more than 2hr to build, we'll
start dumping the current pstree listing for that package.  Since
this can be a bit verbose, we hold it only for slow packages.
The slowest we have is Chrome and that should normally finish in
under 2hrs with Goma enabled on bots.

BUG=chromium:1095661
TEST=CQ passes
TEST=looked at output behavior with manual lower timeouts

Change-Id: I930f8065a506c7fe4f4de003e20d1335203505f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/portage_tool/+/2337614
Reviewed-by: Chris McDonald <cjmcdonald@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
2 files changed