Shouldn't we recommend to run the latest version of production instea…

[philippkahr]

When I run the stack version e.g. 8.3.3, this will be built upon the release of 8.3.3. I don't think that it will be rebuild everytime an integration receives an update and is compatible with 8.3.3? https://github.com/elastic/package-storage#release-distribution at least that is how I understand this.

Customers that need to host their own EPR should rather host the production version and run some automatic job to update the package registry frequently to receive the latest updates to the integrations. <-- We should also mention that the production registry is frequently updated.

[mergify]

This pull request does not have a backport label. Could you fix it @philippkahr? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-/d./d is the label to automatically backport to the /d./d branch. /d is the digit
  NOTE: backport-skip has been added to this pull request.

[apmmachine]

A documentation preview will be available soon:

• 📚 HTML diff
• 📘 Fleet and Elastic Agent Guide
• 📙 Observability Guide

[dedemorton]

Good point. @bmorelli25 I'm curious to get your input here. With the integration packages releasing on their own schedule, I think using the production version makes a lot of sense. WDTY?

[bmorelli25]

I honestly don't know. Maybe @jsoriano can comment as to why we recommend the current version instead of production.

[philippkahr]

There was an internal discussion with @jlind23 who also recommended to run the stack version. Would also be very curious to know why that is the case.

[jlind23]

@bmorelli25 what do you mean by:

why we recommend the current version instead of production.

When the registry docker image is built, it takes all integrations available in the production branch and copy/paste them in this docker image. Every integrations added after the docker image build will not be included.

This problem doesn't exist with the public registry as the registry APIs do always rely on the production branch content.

[philippkahr]

Ok, then let me rephrase the use case / question.

As a customer I run in an airgapped environment. Which version of the EPR am I supposed to run? Should it be always aligned to the stack version?

Ask: when I deploy a 7.17 cluster, should I use 7.17 of the EPR, or is it better to use 8.3.3?

Furthermore, if I am not allowed / recommended to run the production label of the container, I will not profit from any update to the integrations until a new ES release is there and a new container with 8.x is built. That means I get the same release cycle for integration fixes and updates as I had with *beats. In addition to that, it could lead to confusion when a user is looking up on docs.elastic.co and sees that an update for an integration is available, yet, they don't see it in their stack.

Due to both of this "issues" I decided to open up this PR to discuss this properly.

[jsoriano]

I remember this was more a product decision, but there are some good reasons to recommend the use of tagged releases:

• They are more similar to the state of the registry during the testing phases of each Stack release. With more control on what is released in these images we could introduce additional validation steps if necessary at some moment.
• We accept that users in air-gapped environments are not going to have the same experience as users of the public registry. In that sense, and regarding updates, they are not going to receive the latest fixes on integrations, till a new image is released. Notice that this is still not worse that current cadence for fixes on any other software we release.
• Tagged versions are now a full snapshot of production images, with all packages ever released. We are refactoring the package registry to use a cloud storage instead of including all packages in a docker image. Eventually there won't be anything as a production image containing all packages.
• As the number of packages grow, it won't be practical to distribute the whole set of them, we will need to curate a selection of packages, and they will be included on this tagged images. We may consider releasing different "flavors" in the future with different selections of packages.
• We need to have a release process and a version schema for these selections of packages, and at the moment we are using the company-wide release process for this. This could be reconsidered though, to have daily releases or something like this.

Summarizing, even when it is true that current tagged images are quite close to the production images, this is likely going to change soon. Users can still use newer images if they want newer packages, without needing to update the rest of the stack.

As a side note, we are currently experimenting with a proxy mode (elastic/package-registry#770). In deployments where a local registry has access to the public one, this can allow mixed scenarios, where the local registry contains a set of packages, but it can connect to the public registry to get updates or access to additional packages.

ccing @mukeshelastic

[jsoriano]

As a customer I run in an airgapped environment. Which version of the EPR am I supposed to run? Should it be always aligned to the stack version?

We recommend to use the same as the stack version, but you can use any of the available options. Maybe we should clarify in the docs that the recommendation is about being aligned to the stack version.

Furthermore, if I am not allowed / recommended to run the production label of the container, I will not profit from any update to the integrations until a new ES release is there and a new container with 8.x is built. That means I get the same release cycle for integration fixes and updates as I had with *beats. In addition to that, it could lead to confusion when a user is looking up on docs.elastic.co and sees that an update for an integration is available, yet, they don't see it in their stack.

Maybe we should clarify this in the air-gapped docs, that they are not going to have updates till they update their registry.

[philippkahr]

Perfect. I think @dedemorton we should focus on getting this feedback (#2063 (comment)) back into the airgapped docs.

In the meantime I close this PR, as stack aligned is the version we want to run.