Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Elasticsearch ILM Polices #2454
Add support for Elasticsearch ILM Polices #2454
Changes from 1 commit
bfb2e87
6acf5da
17a8e1c
06896ec
5f0dac7
f24f9fa
09ba770
fe5938f
f02553e
1af12df
0f5bb18
8d3c483
e5fda11
016f626
f524da3
0076eaf
653b861
526f214
8a51584
fcbf676
a3e8f0e
8e0b750
e2a0fee
3f31372
aacf4c9
77a8a17
4755013
667a511
3696256
4b044bc
877175c
f57c970
c514c91
183af3c
d1e9dc4
4c8ad45
602b943
00b7daa
b980e87
5b80343
dc1b855
76256d6
1a888aa
9bec1a4
01654f9
ea4e71c
23f53dd
ef16c6a
0f0cc90
d4895d7
6864653
6f8bfb1
d85d954
c0f3e93
09e4e9d
a3b465b
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't want to add a new set of index templates.
Could we instead add the
lifecycle
configuration to our existing index templates?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pavolloffay - I tried that initially, When jaeger comes up, it is also trying to create the index templates ( even with --es.use-alias enabled) and if the template name is same - it is overriding it. Hence had to use new set of templates with higher priority (order = 1).
Ideally, if --es-use-alias is enabled - I believe template creation should also be left to external component (esRollover) along with alias creation. There might be a valid reason why it is not - I am not sure.
Right now, both jaeger and esRollover try to create the templates. As Jaeger starts after esRollover is run, its overriding the templates created by esRollover.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if the templates are overridden it should not be an issue since the definition does not change it just references the name of ILM.
--es.create-index-templates
can be used to disable template creationThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lifecycle or ILM is inside settings of index template -
Default template has settings without lifecycle. Hence when we start jaeger - it overrides the settings of the template ( unless --es.create-index-templates is used to disable it). That means we are saying user has to use --es.use-alias with --es.create-index-templates always - if ILM is needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get it now, I want to avoid having multiple variations of the index templates.
Can be the lifecycle be configured but a PUT (change) request?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes - completely agree. Ideal thing would be to disable creation of templates from jaeger end whenever we have --es.use-aliases.
Unfortunately we have to run init before starting jaeger. And index-templates must be created before 1st index is created. So, right now, I see no way but to have another set of templates for ILM with different name and higher order - so that when index templates are applied - ILM template takes precedence.
Either this or we somehow mandate to always use --es.use-alias with --es.create-index-templates set to false.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pavolloffay, I think I know why this was not done. As of now, If we enable use-aliases and start jaeger with out running esRollover init first --> Jaeger is creating the default index templates and two initial indices with names jaeger-span-write & jaeger-span-read, so that it would continue to work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pavolloffay - As suggested - changes made to code to avoid using multiple variations of index templates. Please go through the latest commit
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pavolloffay Any feedback on this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pavolloffay / @jaegertracing/elasticsearch - Let me know, if you have any thoughts on the approach