devserver: support for serving remotely hosted payloads

This feature is aimed to allow one instance of a devserver to respond to
an update check with a link to a payload that's hosted (statically
staged) on another devserver. Specifically, we plan on using it in the
test lab to spawn test-private devserver instances that will direct DUTs
to obtain their payloads from a central devserver. To use this feature,
the test-private instance should be invoked with the following switches:

 --archive_dir=static/  (avoids payload generation, value insignificant)
 --urlbase=http://<central-devserver-hostname>:<port>
 --payload=static/<path>/<to>/<payload>/<file>
 --remote_payload  (triggers proper handling of the remote payload)

Note that the --payload value is optional, and is only used if the
update check issued by the client does not contain a payload URL, for
example: .../update/static/<path>/<to>/<payload>/<file>

* Adds a new option --remote_payload for triggering special handling of
  remotely hosted payload files. This is hard to infer based on existing
  options (such as --urlbase) and disambiguates the behavior.

* Added functionality for retrieving necessary attributes of the remote
  payload file, such as size and hashes, which need to be send back to
  the DUT in the update response.

* The payload file name is assumed to be a devserver constant
  (update.gz).

* The remote payload URL is assumed to have a /static prefix to it. This
  invariant must be preserved by both the client (request) and the
  backend devserver (provisioning).

* Added unit test to cover remote payload logic.

BUG=chromium-os:33762
TEST=Devserver responds with remote payload data; passes unit tests.

Change-Id: Ief34bbd18d9046460f2b2a7a6c88b465d65426e8
Reviewed-on: https://gerrit.chromium.org/gerrit/34499
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
3 files changed