This page explains how to publish a trained ML model so that the ML Service can make it available at runtime for inference. Design doc for model publishing: go/cros-ml-service-models.
Currently the ML Service supports TensorFlow Lite models. Each model needs to be a single file. The file format is a TFLite flat buffers (extension .tflite).
You can convert a TF model into a TFLite model using the TensorFlow Lite Converter (Toco) tool.
Google-specific: For resources about the overall Chrome OS ML model training & deployment lifecycle across google3, Chromium, and Chrome OS, see go/ml-abc.
You need to be aware what (minimum) version of the TFLite runtime your model requires.
TODO(amoylan): Show an example of how to verify this using Toco.
The following table shows the version of ML Service's TFLite runtime for each CrOS major version:
| CrOS version | TFLite version |
|---|---|
| M71 (and prior) | (none) |
| M72 | 1.9.0 |
| M73 | 1.9.0 |
| M74 | 1.9.0 |
| M75 | 1.9.0 |
| M76 | 1.9.0 |
| M77 | 1.9.0 |
| M78 | 1.14.0 |
| M79 | 1.14.0 |
| M80 | 1.14.0 |
| M81 | 1.14.0 |
Two methods will be supported to publish an ML model:
To know more about why we have two methods and the pros and cons of each, read the section below.
Once you have a model ready that you want to publish, upload it to https://storage.googleapis.com/chromeos-localmirror/distfiles/. The web interface is https://pantheon.corp.google.com/storage/browser/chromeos-localmirror/distfiles. The mirror is accessible to all Googlers with prod access.
There is no directory structure under distfiles/, so the filename needs to be unique enough to avoid conflicts. This is the required convention for filenames of models:
mlservice-model-<feature_name_and_variant>-<timestamp_and_version>.<ext>
mlservice-model is a fixed prefix<feature_name_and_variant> should indicate the feature that this model is built for and any variant of the model if applicable<timestamp_and_version> is used to differentiate different versions of the same model. The preferred format is 20180725. To disambiguate further, the actual time (hour, minutes, etc) can be added, or you can add a version string like -v2.An example of filename is: mlservice-model-tab_discarder_quantized-20180507-v2.tflite.
After you upload the file, make it publicly visible by selecting Edit Permissions and adding a ‘Reader’ permission for a Group named ‘allUsers’.
Files in the ChromeOS file mirror should never be deleted. You simply add newer models as you need them, but leave the previous ones in the mirror, even if you don't plan to use them ever again.
Once your model is uploaded to the ChromeOS file mirror, the next step is to create a CL. This CL will have changes to the ML Service ebuild, which installs the model(s) into rootfs. The ebuild is located at /chromiumos/overlays/chromiumos-overlay/chromeos-base/ml/ml-9999.ebuild. Simply add your models in the system_models variable: they are installed in the src_install() function in the same file. The install location in the ChromeOS system is /opt/google/chrome/ml_models.
Then you need to update the Manifest file. In the ebuild path above, run ebuild <ebuild_file> manifest. A new entry for the new model will be added into Manifest file. This is the checksum to ensure the model file is downloaded successfully.
See CL/1125701 (and relative fix in CL/1140020) as an example.
There are three places to update here:
See this CL for an example of all the above.
See the logging section for more details.
If you have a new, probably better version of a model, these are the recommended steps to swap it in:
If the model you are replacing hasn't launched yet, you can of course skip the Finch rollout. Optionally, you can also take this shortcut (with caveats):
Detailed instructions for this section are pending the final implementation of OOB model updates. See go/cros-model-update for the design. ETA 2020-Q1.
We provide two methods of model deployment, each with their own pros and cons.
Method #1, installing inside rootfs, has the benefit of making the model available right after the system boots for the first time. No updates and thus no network connection is required. Besides, it was better suited as a starting point to create a working prototype of the ML Service in the early stages of development.
Method #1 has one big limit: storage space on rootfs. Since we want to avoid increasing the size of rootfs, we reserve this method for those models that are either very small (less than 30KB) or that are system critical and need to be available before the first successful update of the device.
A second property of Method #1 is that it ties model versions to the platform version, i.e. updated every 6 weeks after passing through dev and Beta versions.
Method #2 is the solution to the limitations of Method #1. Large models can be installed this way, because we have fewer restrictions on the size increase of the stateful partition (within reason); and the component updater approach allows the rollout of models (controlled by e.g. Finch) in between platform releases.
The disadvantage of Method #2 is that the model cannot be used until the component updater has fetched the component, i.e. it won't be available out-of-box. For models that need to be present when the device is first used or otherwise always be available, a combination of Methods #1 and #2 will be needed.
Contact chrome-knowledge-eng@ to consult about the right approach.
There are 9 “request performance metric” histograms and 3 “request event” enums histograms that model owner MUST config to be logged on go/uma-.
The name format of the 9 “request performance metric” histograms is,
MachineLearningService.MetricModelName.RequestEventName.ResourceName
MetricModelName is defined by specifying |metrics_model_name| in model_metadata.cc.
There are 3 different RequestEventName corresponding to 3 types of requests (e.g. from Chrome):
For each request, there are three ResourceName corresponding to three computer resources:
For the enum histograms, the name format is,
MachineLearningService.MetricModelName.RequestEventName.Event
where MetricModelName and RequestEventName are same as defined above.
There are two tasks to enable the logging:
<histogram_suffixes name="MachineLearningServiceModels" separator="."></>. You can see the suffixes of “SmartDimModel” or “TestModel” for examples. Please note that, after modifying histograms.xml, one has to use pretty_print.py and validate_format.py to validate the format of histograms.xml before uploading your CL. Both of the scripts locate in the same folder of histograms.xml and can be directly run without arguments. pretty_print.py can also help you justify the format.In reviewing the CL of modifying histograms.xml, you need to specify one reviewer from the owners of histograms.xml. The owners can be found at metrics-owner.