fix: support long texts for labels in ElasticsearchDocumentStore

[anakin87]

Related Issues

• fixes write_labels crashes for ElasticsearchDocumentStore for labels with long context #2621

Proposed Changes:

As deeply explained in #2621, in label index, I changed field types of answer and document from flattened to nested, aligning ElasticsearchDocumentStore with OpensearchDocumentStore. This way, the text in these fields is tokenized correctly and is no longer considered one immense term.

How did you test it?

Added new related test

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added tests that demonstrate the correct behavior of the change
• I've used the conventional commit convention for my PR title
• I documented my code
• I ran pre-commit hooks and fixed any issue

[anakin87]

@vblagoje I just merged main into this branch.
Any thoughts about this PR?

[vblagoje]

@anakin87, my apologies - I somehow missed these notifications. I am not an expert in ElasticSearch, let me ask who can understand this PR much better than I do.

[ZanSara]

@anakin87 just to make sure: does this represent a schema break? For example: if you write some documents in ES with the code in main, then switch to this branch and try to read them (or vice-versa) does it works or it throws exceptions?

I'm asking because if it throws exceptions this should be marked as a breaking change and we should warn users about this detail when migrating (and potentially offer some instructions on how to access their documents again)

[anakin87]

@ZanSara great question!

Digging into this, I remembered that the method _create_label_index is used only during index creation
(who would have thought? 😃 ).
And once the index is created, as it is expected, the schema doesn't change.

I performed some tests:

• write labels using the main and read them using this branch
  ✔️
• write labels using the main and write more labels using this branch
  same bug with long text (as the schema doesn't change)
• write labels using this branch and read them using the main
  ✔️
• write labels using this branch and write more labels using the main
  ✔️ no bug, since the schema doesn't change

So it doesn't seem like a breaking change.
It must be said that if the user keeps using an index with the old schema, the bug won't disappear.

[ZanSara]

@anakin87 great! I think it's good to merge then 👍

[ZanSara]

🚀

[anakin87]

Thank you.
But now suddenly a test has started to fail. 😃
I'll try to understand why...

[ZanSara]

The stacktrace of that test is quite funny I have to say 😂

[anakin87]

@ZanSara finally ready to go!