commit 7ee1e5dc3841f8e521b45607bd07567fb9f7863f
author zhizhouy <zhizhouy@google.com> Tue Mar 24 17:11:45 2020 -0700
committer Commit Bot <commit-bot@chromium.org> Thu Mar 26 03:46:38 2020 +0000
tree bdc0f9f87e7b208634f6db9747a2aab432d2bd56
parent d511f2bc35d35f203ab500d93d57c328b50529e5

crosperf: raise error at exit when benchmarks fail to run

Crosperf always returns 0 no matter benchmarks fail or experiment
interrupted in the middle. Thus we cannot tell if a run succeeded or not
with the exit status of it and it makes our nightly test failures hard
to find.

In this patch, I changed the behavior of crosperf return value:
1) Crosperf will not generate any report or send email if terminated or
all benchmarks fail. It raises RuntimeError stating all benchmarks
failing in the end.
2) Crosperf will generate report if benchmarks (but not all) fail, and
will raise RuntimeError stating benchmarks partially failing.
3) Crosperf will also copy results json files to local results directory
for further info.

BUG=chromium:1063703
TEST=Passed all unittests, tested with different failure situations.

Change-Id: I998bad51cd7301b9451645d22e8734963bc01aed
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/2119231
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Zhizhou Yang <zhizhouy@google.com>
Tested-by: Zhizhou Yang <zhizhouy@google.com>
Auto-Submit: Zhizhou Yang <zhizhouy@google.com>