-
Notifications
You must be signed in to change notification settings - Fork 166
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove de_dot functionality from plugin #320
Comments
@jcantrill It makes sense that de_dot functionality would not be part of this plugin by default. In your message though you mentioned referring users to alternate plugins, do you happen to know of a replacement for this functionality? Did this already get documented somewhere? |
I do not know of a specific plugin though one could use record_transformer https://github.com/repeatedly/fluent-plugin-record-modifier#ruby-code-trick-for-complex-logic |
We recently upgraded to a version of this plugin were the de_dot feature has been removed and we encountered tons of mapping conflicts in elasticsearch/opensearch due to the removal. I'm wondering how you have solved this, or you never encounter such mapping problems(I know it links to fluent-bit) for annotations or labels in your platform? |
You might either find a plugin, write a plugin, or use this technique https://github.com/repeatedly/fluent-plugin-record-modifier#ruby-code-trick-for-complex-logic
Mapping collisions from my experience are either related to trying to push different data types to the same field which is not supported by Elasticsearch. There is a way around this by declaring fields not indexable but that may or may not be useful for you. We solved the type issue by modifying the es templates to be nested The other issue described here is how Elasticsearch treats dotted keys: is it the field name or the path the the field. My recollection is there originally was a bug that was fixed (es2.x) and that later you could tell ES via a runtime switch how to treat dotted fields. It may have also been the nested template change also resolved this issue; I don't recall. |
Hi, thanks for the reply. |
This functionality is a hold over from the original implementation that did a log of tangential things. Remove this functionality and refer users to alternate plugins
The text was updated successfully, but these errors were encountered: