new_variant: add end-to-end test

Add an end-to-end test for new_variant.py to verify that creating
firmware for a new variant still works.

BUG=b:167305316
TEST=See testdata/README.md for full details. Short version:
`cd testdata` then pick a reference board to test:
`./new_variant_fulltest.sh hatch`
`./new_variant_fulltest.sh puff`
`./new_variant_fulltest.sh volteer`
`./new_variant_fulltest.sh trembyle`
`./new_variant_fulltest.sh dalboz`
`./new_variant_fulltest.sh waddledee`
`./new_variant_fulltest.sh waddledoo`
Watch the output to see the build succeed and then clean up all of
the generated CLs. Run `repo status` to see that there aren't any
modified files hanging around.

Change-Id: I33157f41f398c0dc60a453584de71b538dd9d37c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/dev-util/+/2393470
Tested-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Rob Barnes <robbarnes@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Paul Fagerburg <pfagerburg@chromium.org>
diff --git a/contrib/variant/testdata/README.md b/contrib/variant/testdata/README.md
index 4b61a40..26a759a 100644
--- a/contrib/variant/testdata/README.md
+++ b/contrib/variant/testdata/README.md
@@ -72,3 +72,105 @@
 DEBUG:root:Add to commit cb_config Cq-Depend: chromium:1641906
 (cr) $ rm ~/.new_variant.yaml
 ```
+
+End-to-End Test
+===============
+`testdata/e2e.sh` is an end-to-end test of `new_variant.py`. The script takes
+the name of a reference board as a parameter, and creates a new variant of
+that reference board. The script ensures that `new_variant.py` will not
+upload the CLs for the new variant to gerrit, so the script can be run
+multiple times to test functionality.
+
+The supported reference boards are:
+* hatch
+* puff
+* volteer
+* trembyle (zork)
+* dalboz (zork)
+* waddledee (dedede)
+* waddledoo (dedede)
+
+For example, to test that creating a variant of the Waddledee reference board
+still works,
+
+```
+cd /mnt/host/source/src/platform/dev/contrib/variant/testdata
+./e2e.sh waddledee
+```
+
+When the build finishes, all of the branches and commits for the new variant
+will be `repo abandon`ed. The output at the end of a successful test looks
+something like this:
+
+```
+[Lots of build messages not included here]
+
+>>> Using system located in ROOT tree /build/dedede/
+
+>>> No outdated packages were found on your system.
+DEBUG:root:process returns 0
+DEBUG:root:Symlink /build/dedede/etc/portage/package.mask/cros-workon already exists. Don't recreate it.
+DEBUG:root:Symlink /build/dedede/etc/portage/package.unmask/cros-workon already exists. Don't recreate it.
+DEBUG:root:Symlink /build/dedede/etc/portage/package.keywords/cros-workon already exists. Don't recreate it.
+INFO:root:Stopped working on 'chromeos-base/chromeos-config-bsp-dedede-private chromeos-base/chromeos-ec sys-boot/coreboot-private-files-baseboard-dedede sys-boot/intel-jslfsp sys-boot/coreboot sys-boot/libpayload chromeos-base/vboot_reference sys-boot/depthcharge' for 'dedede'
+DEBUG:root:build_path = "/build/dedede/firmware"
+INFO:root:Running step abort
+DEBUG:root:Processing step cb_variant
+INFO:root:Abandoning branch coreboot_kingitchy_20200904 in directory /mnt/host/source/src/third_party/coreboot
+DEBUG:root:Run ['repo', 'abandon', 'coreboot_kingitchy_20200904', '.']
+DEBUG:root:cwd = /mnt/host/source/src/third_party/coreboot
+Abandoned branches:
+coreboot_kingitchy_20200904| src/third_party/coreboot
+
+DEBUG:root:process returns 0
+DEBUG:root:Processing step cb_config
+INFO:root:Abandoning branch create_kingitchy_20200904 in directory /mnt/host/source/src/third_party/chromiumos-overlay
+DEBUG:root:Run ['repo', 'abandon', 'create_kingitchy_20200904', '.']
+DEBUG:root:cwd = /mnt/host/source/src/third_party/chromiumos-overlay
+Abandoned branches:
+create_kingitchy_20200904| src/third_party/chromiumos-overlay
+
+DEBUG:root:process returns 0
+DEBUG:root:Processing step commit_fit
+INFO:root:Abandoning branch create_kingitchy_20200904 in directory /mnt/host/source/src/private-overlays/baseboard-dedede-private
+DEBUG:root:Run ['repo', 'abandon', 'create_kingitchy_20200904', '.']
+DEBUG:root:cwd = /mnt/host/source/src/private-overlays/baseboard-dedede-private
+Abandoned branches:
+create_kingitchy_20200904| src/private-overlays/baseboard-dedede-private
+
+DEBUG:root:process returns 0
+DEBUG:root:Processing step ec_image
+INFO:root:Abandoning branch create_kingitchy_20200904 in directory /mnt/host/source/src/platform/ec
+DEBUG:root:Run ['repo', 'abandon', 'create_kingitchy_20200904', '.']
+DEBUG:root:cwd = /mnt/host/source/src/platform/ec
+Abandoned branches:
+create_kingitchy_20200904| src/platform/ec
+
+DEBUG:root:process returns 0
+INFO:root:Running step clean_up
+/mnt/host/source/src/platform/dev/contrib/variant /mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/private-overlays/overlay-dedede-private/chromeos-base/chromeos-config-bsp-dedede-private /mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/project/dedede /mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/private-overlays/baseboard-dedede-private/sys-boot/coreboot-private-files-baseboard-dedede/asset_generation/outputs /mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+/mnt/host/source/src/platform/dev/contrib/variant ~/trunk/src/platform/dev/contrib/variant
+(cr) $
+```
+
+For boards that require a fitimage, `e2e.sh` will create a fake fitimage for
+the new variant by copying the reference board's fitimage. Obviously this
+won't be bootable, but since the purpose is just to ensure that the build
+works, this is OK, and prevents the tester from having to create the fitimage
+outside the chroot (by running `gen_fit_image.sh`) and then restarting
+`new_variant.py`.
+
+Similarly, for boards that use the project configuration repositories (all
+of the boards this test supports except for Hatch), `e2e.sh` creates a
+configuration directory that will suffice for building, but it is not an
+actual repo.
+
+When `e2e.sh` is done, it will clean up the temporary files it created and
+revert the changes it made to checked-in files. If the build fails, the user
+will have to clean it up for now; improved error handling is planned.
diff --git a/contrib/variant/testdata/modify_step_list.sed b/contrib/variant/testdata/modify_step_list.sed
new file mode 100644
index 0000000..417b907
--- /dev/null
+++ b/contrib/variant/testdata/modify_step_list.sed
@@ -0,0 +1,5 @@
+# Replace all of the steps between EMERGE and CLEAN_UP with an ABORT so
+# that generated CLs will not be uploaded to gerrit.
+s/(step_names.EMERGE,)\n.*\n(.*)(step_names.CLEAN_UP)/\1\n\2step_names.ABORT,\n\2\3/
+# Remove the FW_BUILD_CONFIG step.
+s/step_names.FW_BUILD_CONFIG,//
\ No newline at end of file
diff --git a/contrib/variant/testdata/new_variant_fulltest.sh b/contrib/variant/testdata/new_variant_fulltest.sh
new file mode 100755
index 0000000..f902a62
--- /dev/null
+++ b/contrib/variant/testdata/new_variant_fulltest.sh
@@ -0,0 +1,209 @@
+#!/bin/bash
+# Copyright 2020 The Chromium OS Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+# End-to-end test of creating firmware for a new variant of a reference board
+VERSION="1.0.0"
+SCRIPT=$(basename -- "${0}")
+set -e
+
+export LC_ALL=C
+
+if [[ ! -e /etc/cros_chroot_version ]]; then
+  echo "This script must be run inside the chroot."
+  exit 1
+fi
+
+if [[ "$#" -lt 1 ]]; then
+  echo "Usage: ${SCRIPT} reference_name"
+  echo "e.g. ${SCRIPT} hatch | puff | volteer | waddledee | waddledoo | trembyle | dalboz"
+  echo "End-to-end test to create a new variant of a reference board"
+  echo "Script version ${VERSION}"
+  exit 1
+fi
+
+# ${var,,} converts to all lowercase.
+REFERENCE="${1,,}"
+
+# Set variables depending on the reference board.
+#
+# All boards:
+#
+# BASE - the name of the baseboard.
+# NEW - the name of the new variant.
+#
+# Boards using Boxtser config only:
+#
+# CONFIG_DIR - the directory for the project configuration files, if needed
+#   for the baseboard.
+# OVERLAY_DIR - the directory for the chromeos-config overlay ebuild, if it
+#   needs to be modified for this baseboard so that the new variant will build.
+# EBUILD - the name of the chromeos-config overlay ebuild, if needed.
+#
+# Intel-based reference boards only:
+#
+# FITIMAGE_OUTPUTS_DIR - the directory where gen_fit_image.sh will place
+#   the fitimage files.
+# FITIMAGE_FILES_DIR - the directory where commit_fitimage.sh moves (or copies)
+#   the fitimage that gen_fit_image.sh created. This is where the reference
+#   board's fitimage lives, and we will copy that fitimage and related files
+#   to the new variant's name and place the copy in ${FITIMAGE_OUTPUTS_DIR}.
+#
+case "${REFERENCE}" in
+  hatch)
+    BASE=hatch
+    NEW=tiamat
+    FITIMAGE_OUTPUTS_DIR=/mnt/host/source/src/private-overlays/baseboard-hatch-private/sys-boot/coreboot-private-files-hatch/asset_generation/outputs
+    FITIMAGE_FILES_DIR=/mnt/host/source/src/private-overlays/baseboard-hatch-private/sys-boot/coreboot-private-files-hatch/files
+    ;;
+
+  puff)
+    BASE=puff
+    NEW=tiamat
+    CONFIG_DIR=/mnt/host/source/src/project/puff
+    OVERLAY_DIR=/mnt/host/source/src/private-overlays/overlay-puff-private/chromeos-base/chromeos-config-bsp-puff-private
+    EBUILD=chromeos-config-bsp-puff-private-9999.ebuild
+    FITIMAGE_OUTPUTS_DIR=/mnt/host/source/src/private-overlays/baseboard-puff-private/sys-boot/coreboot-private-files-puff/asset_generation/outputs
+    FITIMAGE_FILES_DIR=/mnt/host/source/src/private-overlays/baseboard-puff-private/sys-boot/coreboot-private-files-puff/files
+    ;;
+
+  volteer)
+    BASE=volteer
+    NEW=gnastygnorc
+    CONFIG_DIR=/mnt/host/source/src/project/volteer
+    OVERLAY_DIR=/mnt/host/source/src/private-overlays/overlay-volteer-private/chromeos-base/chromeos-config-bsp-volteer-private
+    EBUILD=chromeos-config-bsp-volteer-private-9999.ebuild
+    FITIMAGE_OUTPUTS_DIR=/mnt/host/source/src/private-overlays/baseboard-volteer-private/sys-boot/coreboot-private-files-baseboard-volteer/asset_generation/outputs
+    FITIMAGE_FILES_DIR=/mnt/host/source/src/private-overlays/baseboard-volteer-private/sys-boot/coreboot-private-files-baseboard-volteer/files
+    ;;
+
+  waddledee|waddledoo)
+    BASE=dedede
+    NEW=kingitchy
+    CONFIG_DIR=/mnt/host/source/src/project/dedede
+    OVERLAY_DIR=/mnt/host/source/src/private-overlays/overlay-dedede-private/chromeos-base/chromeos-config-bsp-dedede-private
+    EBUILD=chromeos-config-bsp-dedede-private-9999.ebuild
+    FITIMAGE_OUTPUTS_DIR=/mnt/host/source/src/private-overlays/baseboard-dedede-private/sys-boot/coreboot-private-files-baseboard-dedede/asset_generation/outputs
+    FITIMAGE_FILES_DIR=/mnt/host/source/src/private-overlays/baseboard-dedede-private/sys-boot/coreboot-private-files-baseboard-dedede/files/blobs
+    ;;
+
+  trembyle|dalboz)
+    BASE=zork
+    NEW=grue
+    CONFIG_DIR=/mnt/host/source/src/project/zork
+    OVERLAY_DIR=/mnt/host/source/src/private-overlays/overlay-zork-private/chromeos-base/chromeos-config-bsp-zork-private
+    EBUILD=chromeos-config-bsp-zork-private-9999.ebuild
+    ;;
+
+  *)
+    echo Unsupported reference board "${REFERENCE}"
+    exit 1
+    ;;
+esac
+
+# ${var^^} converts to all uppercase.
+NEW_UPPER="${NEW^^}"
+
+VARIANT_DIR=/mnt/host/source/src/platform/dev/contrib/variant
+pushd "${VARIANT_DIR}"
+
+# When exiting for any reason, restore files under version control to their
+# unmodified state and remove any new files that were created outside of
+# version control.
+# Not all of these steps may have happened yet, but we can silently ignore
+# any errors with git trying to restore a file that hasn't changed, or `rm`
+# trying to remove files that don't exist.
+cleanup() {
+  # Undo changes to the control file.
+  pushd "${VARIANT_DIR}"
+  git restore "${REFERENCE}.py"
+  popd
+  # If we have an ebuild, undo any changes.
+  if [[ ! -z ${OVERLAY_DIR+x} ]] ; then
+    pushd "${OVERLAY_DIR}"
+    git restore "${EBUILD}"
+    popd
+  fi
+  # If we have a Boxster config dir, remove it
+  if [[ ! -z ${CONFIG_DIR+x} ]] ; then
+    pushd "${CONFIG_DIR}"
+    rm -Rf "${NEW}"
+    popd
+  fi
+  # If we have a fitimage, remove any files we created to fake out the
+  # fitimage for the new variant.
+  if [[ ! -z ${FITIMAGE_OUTPUTS_DIR+x} ]] ; then
+    pushd "${FITIMAGE_OUTPUTS_DIR}"
+    rm -f "fitimage-${NEW}.bin" "fitimage-${NEW}-versions.txt"
+    # Clean up the extra Volteer fitimage files, too.
+    if [[ "${REFERENCE}" == "volteer" ]] ; then
+      rm -f "fit-${NEW}.log"
+      popd
+      pushd "${FITIMAGE_FILES_DIR}/blobs"
+      rm -f "csme-${NEW}.bin" "descriptor-${NEW}.bin"
+    fi
+    popd
+  fi
+}
+trap 'cleanup' EXIT
+
+# Make sure we don't upload any CLs that are generated.
+sed -i -z -E -f testdata/modify_step_list.sed "${REFERENCE}.py"
+
+# Add the new variant to the overlay ebuild, if defined.
+if [[ ! -z ${OVERLAY_DIR+x} ]] ; then
+  pushd "${OVERLAY_DIR}"
+  sed -i -E -e "s/PROJECTS=\(/PROJECTS=\(\n\t\"${NEW}\"/" "${EBUILD}"
+  popd
+fi
+
+# Create the project configuration repo, if defined.
+if [[ ! -z ${CONFIG_DIR+x} ]] ; then
+  mkdir -p "${CONFIG_DIR}/${NEW}"
+  pushd /mnt/host/source/src/config
+  sbin/gen_project  /mnt/host/source/src/config "${BASE}" "/mnt/host/source/src/program/${BASE}/" "${NEW}" "${CONFIG_DIR}/${NEW}"
+  popd
+  # Because this isn't actually a git repo synced from the server, we can't
+  # do any `git` or `repo` commands inside it, which means we can't run
+  # fw_build_config.sh to make the changes we need. Instead just apply the
+  # changes manually.
+  pushd "${CONFIG_DIR}/${NEW}"
+  # Apply FW_BUILD_CONFIG to new project and build the config
+  sed -i -e "s/_FW_BUILD_CONFIG = None/_FW_BUILD_CONFIG = program.firmware_build_config(_${NEW_UPPER})/" config.star
+  ./config.star
+  popd
+fi
+
+# If we have a fitimage, make a copy of the reference board's fitimage under
+# the new variant's name so that we don't have to generate the fitimage outside
+# the chroot.
+if [[ ! -z ${FITIMAGE_OUTPUTS_DIR+x} ]] ; then
+  pushd "${FITIMAGE_OUTPUTS_DIR}"
+  cp "${FITIMAGE_FILES_DIR}/fitimage-${REFERENCE}.bin" "fitimage-${NEW}.bin"
+  cp "${FITIMAGE_FILES_DIR}/fitimage-${REFERENCE}-versions.txt" "fitimage-${NEW}-versions.txt"
+  # Volteer requires some extra files; the FIT log is named after the
+  # variant, and there are two other blobs that are customized to the
+  # variant and have names to reflect it.
+  if [[ "${REFERENCE}" == "volteer" ]] ; then
+    cp "fit-${REFERENCE}.log" "fit-${NEW}.log"
+    popd
+    pushd "${FITIMAGE_FILES_DIR}/blobs"
+    cp "csme-${REFERENCE}.bin" "csme-${NEW}.bin"
+    cp "descriptor-${REFERENCE}.bin" "descriptor-${NEW}.bin"
+  fi
+  popd
+fi
+
+
+# Now create the new variant.
+./new_variant.py --board="${REFERENCE}" --variant="${NEW}" --verbose
+# TODO(b/167305316) capture output of new_variant.py.
+
+# TODO(b/167305316) parse output and/or check [-e ~/.new_variant.yaml] to
+#   determine success/failure.
+
+# If new_variant didn't clean up after itself, then --abort.
+if [[ ! "${HOME}/.new_variant.yaml" ]] ; then
+  ./new_variant.py --abort --verbose
+fi