tree 2390928fc7d5f5fc2b86a6349be27b565f4ed625
parent 526a5f8ffa6609e9ab9546c1b53acf6237394944
author Mike Frysinger <vapier@chromium.org> 1568409999 -0400
committer chrome-bot <chrome-bot@chromium.org> 1568955068 -0700

pre-upload: overhaul how we handle UTF-8 from commands

We've been working on bytes in most places from command output.  This
largely just works because people use UTF-8 or the binary content gets
ignored.  Since we want to operate on strings everywhere, change the
run command helper to always return strings rather than bytes.  We had
a single place that was dealing with possible UTF-8 errors, so move it
to this common place for all to benefit.

For now we convert invalid UTF-8 content to U+FFFD characters.  Seems
unlikely anyone will notice as none of our tools operate on binary
data, and if they do, it's directly.  If we get to the point where we
have a tool that wants bytes in/bytes out, we can expand the API.

BUG=chromium:1003955
TEST=unittests pass w/python2 & python3

Change-Id: Iedd7c179e0b0be6d426aad0edb88117f2f472b95
Reviewed-on: https://chromium-review.googlesource.com/1803860
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Will Bradley <wbbradley@chromium.org>