Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce declarative plugin management #77544

Merged
merged 117 commits into from
Nov 15, 2021

Conversation

pugnascotia
Copy link
Contributor

@pugnascotia pugnascotia commented Sep 10, 2021

Closes #70219.

Introduce a declarative way for the Elasticsearch server to manage plugins, which reads the elasticsearch-plugins.yml file and works which plugins need to be added and / or removed to match the configuration. Also make it possible to configure a proxy in the config file, instead of through the environment.

Most of the work of adding and removing is still done in the InstallPluginAction and RemovePluginAction classes, so the behaviour should be the same as with the install and remove commands. However, these commands will now abort if the above config file exists. The intent is to make it harder for the configuration to drift.

This new method only applies to docker distribution types at the moment.

Since this syncing mechanism declarative, rather than imperative, the Cloud-specific plugin wrapper script is no longer required. Instead, an environment variable informs InstallPluginAction to install plugins from an archive directory instead of downloading them, where possible.

Also fix how the cloud-ess image depends on the cloud image.
@pugnascotia pugnascotia added >feature :Core/Infra/Plugins Plugin API and infrastructure v8.0.0 :Delivery/Cloud Cloud-specific packaging and deployment labels Sep 10, 2021
@pugnascotia
Copy link
Contributor Author

@mark-vieira I've added support for testing plugin installation on all Docker flavours, using the analysis-icu plugin. Can you check what I did with the task / configuration / artifact wiring, please?

I also tried to re-enable PluginCliTests, which I believe you last disabled, but something isn't working and I disabled them again.

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Yeah, I need to go back and fix the plugin tests as a result of the example plugin refactoring.

@pugnascotia
Copy link
Contributor Author

I talked to Cloud Delivery and it doesn't look like the wrapper script is being used, so we're good to merge.

@pugnascotia
Copy link
Contributor Author

Oh, and I will add docs but I think that'll require some collaboration with the Docs folks, so I'll do that separately.

@pugnascotia pugnascotia merged commit 3018e52 into elastic:master Nov 15, 2021
@pugnascotia pugnascotia deleted the 70219-declarative-plugins branch November 15, 2021 14:41
pugnascotia added a commit to pugnascotia/elasticsearch that referenced this pull request Nov 15, 2021
Closes elastic#70219.

Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.

Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.

This new method only applies to `docker` distribution types at the
moment.

Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
elasticsearchmachine pushed a commit that referenced this pull request Nov 15, 2021
Closes #70219.

Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.

Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.

This new method only applies to `docker` distribution types at the
moment.

Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
@mark-vieira
Copy link
Contributor

I think we missed backporting this to 8.0 although we did to 7.16.

pugnascotia added a commit that referenced this pull request Nov 16, 2021
Closes #70219.

Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.

Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.

This new method only applies to `docker` distribution types at the
moment.

Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
@pugnascotia
Copy link
Contributor Author

Sorry - backported to 8.0 in 6d54e6f.

pugnascotia added a commit to pugnascotia/elasticsearch that referenced this pull request Nov 16, 2021
pugnascotia added a commit that referenced this pull request Nov 17, 2021
pugnascotia added a commit that referenced this pull request Nov 17, 2021
pugnascotia added a commit that referenced this pull request Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Plugins Plugin API and infrastructure :Delivery/Cloud Cloud-specific packaging and deployment >feature Team:Core/Infra Meta label for core/infra team Team:Delivery Meta label for Delivery team v8.0.0-rc1 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide a declarative way to manage plugins
5 participants