install_pkgs uses named volumes to work with DIND. #1277
Conversation
Previously, config_stripper would store the full TAR file and gzipped TAR file in memory to compute the SHA256 hash. After this change, the stripped TAR file is stored in a temporary location on disk and then renamed into its final location. Furthermore, the SHA256 hash of the gzipped TAR file is computed incrementally as the gzipping progresses.
Previously, install_pkgs would mount files into the target image directly (with '-v src:dst'). This unfortunately doesn't work with docker-in-docker (DIND) setups where the mount is done from the perspective of the host which does not share a filesystem with the docker client. To work with such setups, install_pkgs now creates a named volume and copies files into the named volume. The named volume containing the install script and TAR file is then mounted into the target container.
ralimi
requested review from
alex1545,
nlopezgi and
smukherj1
as code owners
|
/assign @smukherj1 |
|
This PR is producing multiple failures on buildkite presubmit. For example: Please check the errors out and make sure you can build all these targets / run these tests locally with your changes. |
Not all images have a command, so we need to specify one.
|
Thanks for pointing that out. I've re-run the test under tests/docker/package_managers that showed that problem, and I've adjusted the code to fix that. There are quite a few tests under this repository that simply fail for me, and they look to be for a number of reasons that are unrelated to this change (insufficient permissions for a project in GCR, join_layers gets killed because it runs out of RAM while loading the image). |
|
Because of those failures, it's somewhat hard to interpret the results for a run of the full repository. Let me know if I'm missing something? |
|
/gcbrun - buildkite CI is having some infra problems. Running GCB tests anyway to check PR. |
|
/gcbrun |
| @@ -3,9 +3,8 @@ | |||
| # It expects to be volume-mounted inside a docker image, in /tmp along with the | |||
| # installables.tar. | |||
| set -ex | |||
| pushd /tmp | |||
| pushd /tmp/pkginstall | |||
| mkdir -p $(dirname $tmpdir/%{installables_tar}) | ||
| cp -L $(pwd)/%{installables_tar} $tmpdir/%{installables_tar} | ||
| cp -L $(pwd)/%{installer_script} $tmpdir/installer.sh | ||
| # Temporarily create a container sowe can mount the named volume |
|
No problem at all - thanks for the review! Fixed up the docs. |
|
/gcbrun |
|
/gcbrun |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: nlopezgi, ralimi The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Previously, install_pkgs would mount files into
the target image directly (with '-v src:dst').
This unfortunately doesn't work with
docker-in-docker (DIND) setups where the mount
is done from the perspective of the host which
does not share a filesystem with the docker client.
To work with such setups, install_pkgs now creates
a named volume and copies files into the named
volume. The named volume containing the install
script and TAR file is then mounted into the
target container.