Build docker image for elasticsearch snapshot

[liza-mae]

Relates to #70021. Build a docker image and post in it in the archives from the Kibana ES snapshots job.

[liza-mae]

Spoke to @tylersmalley about his review comments, we were unable to get those to work as we expected, but I had already tried different approaches and had slack threads with both Maxime and Rory, +Elasticsearch team to try to match the output of release manager docker image, without a clear answer, so I just did it using docker save CLI.

Main slack thread: https://elastic.slack.com/archives/C0D8ST60Y/p1593206321450800

Adding @pugnascotia as a reviewer to guide us.

[liza-mae]

@pugnascotia, for context we aren't pushing the image to a docker registry, we just are going to post the docker image .tar.gz with the archives the Kibana team is putting in GCP bucket with other distributables similar to release manager. Although if it is better/faster to post it to a docker registry I can set that up too.

[liza-mae]

I am thinking it might be best to push the image to a docker registry, thoughts?

[tylersmalley]

The docker registry would create other issues and another place to manage these images, let's stick to the archives.

[liza-mae]

FYI.. release manager does both and it would be needed if we expand testing to ECK. Just thinking ahead. We can stick with the archives for now.

[pugnascotia]

Some context:

While the ES build does support exporting Docker images to disk, right now it only happens when you call a particular target, because it's only used for a very specific test case. It also only exports the elasticsearch:test image tag, which probably isn't very helpful.

I believe release-manager does its own exporting to disk due to the Tekton pipeline that it uses for pushing images to a registry.

I'm wondering if we can push this exporting logic into the ES build itself, for both Kibana and Infra to use, but that will involve a bit of fiddling around. In the meantime, if you're only interested in the x86 non-OSS image, you could just run:

./gradlew distribution:docker:buildDockerImage

...then do the save as you already have it, although you could gzip the exports images directory to their final locations:

docker images "docker.elastic.co/elasticsearch/elasticsearch" --format "{{.Tag}}" | xargs -n1 bash -c \
  'docker save docker.elastic.co/elasticsearch/elasticsearch:${0} | gzip > ${destination}/elasticsearch-${0}-docker-image.tar.gz'

[pugnascotia]

For interest, here's where the release manager does its docker save:

https://github.com/elastic/infra/blob/master/cd/release/release-manager/buildSrc/src/main/groovy/org/elastic/gradle/StackProject.groovy#L965

[liza-mae]

Thanks @pugnascotia! It is good to know I was on the right track :)

[liza-mae]

@elasticmachine merge upstream

[liza-mae]

@tylersmalley @brianseeders can I get this reviewed please?

[brianseeders]

Can you also change node(workers.label('s')) { on line 28 to node(workers.label('l')) {?

We need a larger machine.. with all of the docker stuff, the small worker is running out of space.

[brianseeders]

Tested on my sandbox, looks good now.