Disable platform_ToolchainOptions for consistency with tip of tree.
The platform_ToolchainOptions test is supposed to only be run as part of
the regression suite, rather than as part of every run.
The full explanation for this from Jim Hebert is included below.
Speaking as the (security) guy who initially proposed to kick it out of
suite_Smoke: I supported removing it from Smoke because the test is so
prone to false positives that effectively adding any new dep closed the
tree (or rev'ing any existing dep enough that a binary appeared at a
new filename, e.g. libs with version numbers in their filename). I'm
not familiar with the canaries but I assume this is another tree-closing
situation.
The decision to put it INTO Smoke(/canary) in the first place was based
on the incorrect understanding that this was a high-value (it is) AND
low-false-fail (it is not) test. The test for quite a while had a
false-pass bug in it, which was masking the true run rate of
false-fails we have now been plagued by.
The severity of the reaction (tree closure - we want it opened asap!)
and the high probability of false-fail were combining, IMHO, to people
blindly throwing anything and everything onto the ever-growing whitelist
just to get the tree open again, and then never following up to investigate
what they'd just whitelisted. That's actually worse than not having the
test at all, because it's making no positive impact on the product, and
wasting developer time constantly hacking it.
I'd rather see this test go red (and not close the tree), which gives us
time to actually stop and consider whether this is a false positive or
perhaps the package is explicitly avoiding some of the compiler flags.
We accomplish that, today, with platform_ToolchainOptions having moved
from bvt/Smoke back to the regression suite. This test has not been
banished entirely.
BUG=chromium-os:26413, chromium-os:27820
TEST=Tested on tip of tree.
Change-Id: If775372c1ece32dd6fc49206249673cbe33f20f6
Reviewed-on: https://gerrit.chromium.org/gerrit/15899
(cherry picked from commit c03b55c8bd597a4ef45cb655a9d73b0992ae20b1)
Reviewed-on: https://gerrit.chromium.org/gerrit/18182
Reviewed-by: Jim Hebert <jimhebert@chromium.org>
Tested-by: David James <davidjames@chromium.org>
1 file changed
