commit 2b9990703a7cea3aeea16808c3da6795698d4c80
author Robert Kolchmeyer <rkolchmeyer@google.com> Mon Aug 28 17:23:22 2023 -0700
committer Robert Kolchmeyer <rkolchmeyer@google.com> Mon Sep 11 17:11:46 2023 +0000
tree c32027a035aa968265a33a8ff6d3c0faa42c4ba2
parent 593ac3305297951103a5fb1b2b278f59f81b88e1

Break the cos-customizer dependency on snapshot.debian.org

snapshot.debian.org is unreliable, and its unreliability sometimes
blocks cos-customizer presubmits and releases for days. It also makes
blocking on cos-customizer integration tests impractical. We need to
break the dependency on snapshot.debian.org.

Unfortunately, the dependency on snapshot.debian.org is pretty tightly
integrated into the cos-customizer build, which is based on bazel and
the distroless package manager. The distroless package manager has since
been deprecated, and the old software is immutable (it depends on an
opaque executable in a GCS bucket that we don't own). To move away from
snapshot.debian.org, we need to move away from the bazel+distroless
build system for cos-customizer.

cos-customizer unfortunately depended on bazel+distroless at a deep
level. The provisioner program, which is used by cos-customizer to
execute preloading logic on the preload VM, bundles a few helper
programs and container images with it. We used bazel+distroless to also
build these bundled container images, and we used bazel+`go embed` to
bundle these other artifacts with the provisioner program.

The most natural alternative for building cos-customizer is a
Dockerfile, since that's what we use for our other container images and
it's a pretty widespread tool for building container images. This change
switches the cos-customizer container build to a Dockerfile from bazel.
Go programs are still compiled with bazel, but the container images are
built with a Dockerfile.

In order to separate the container image build into a different build
system from the Go program builds, I needed to change the way
cos-customizer behaves at runtime. In particular, executable data
dependencies could no longer reasonably be packaged with `go embed`
technology, and instead needed to be passed in to these programs at
runtime.

The following are cos-customizer data dependencies that needed
to be extracted out:
- The cidata file system image
- the scratch file system image

The following are provisioner data dependencies that needed to be
extracted out:
- The docker-credential-gcr executable
- The veritysetup container image
- The handle_disk_layout executable

The Dockerfile orchestrates the build of all these components and
arranges them in the right way so that startup.yaml can correctly pass
their locations in at runtime.

I also needed to update some tests to test against a newer version of
COS - I moved the tests from M69 to M93. The docker version on M69 is
now sufficiently old that it can no longer pull
docker.io/library/ubuntu:latest. As far as I can tell this is completely
independent of the cos-customizer changes, but perhaps the issue was
exposed since these changes make ./run_tests.sh much more reliable.

I tried to split this CL up into smaller CLs, but snapshot.debian.org
unreliability made testing intermediate CLs pretty hard. The tests can
only pass on the full set of changes.

BUG=b/293196753
TEST=./run_tests, `bazel test -- ... -//src/pkg/tools/...`

Change-Id: I0f1c3a329cfca38479c9d58745d251d2fe0778ce
Reviewed-on: https://cos-review.googlesource.com/c/cos/tools/+/56387
Cloud-Build: GCB Service account <228075978874@cloudbuild.gserviceaccount.com>
Tested-by: Robert Kolchmeyer <rkolchmeyer@google.com>
Reviewed-by: Nandhini Rengaraj <nrengaraj@google.com>