Fixed performance bug. Using a list where a set is needed.

[baregawi]

Proposed changes:

• change a list to a set to make "dedupe" linear time instead of quadratic time.

Status (please check what you already did):

• First draft (up for discussions & feedback)
• Final code
• Added tests
• Updated documentation

[baregawi]

@ZanSara could you approve the workflow if you're around? Also, there is only 3 lines of code change if you could code review it. This would seem to slow down write_documents in base document store significantly.

[ZanSara]

Yep looks good! As long as the pipeline is green I'll approve this 👍

[ZanSara]

Thank you so much for the help by the way! You're doing a great job lately! 😊

[baregawi]

It is my pleasure! It makes a lot more sense to build something like this together then everyone having their own, lower quality version of it lol. Thank Deepset for starting this!

I think my focus will be performance and batching at the moment since those seem to be bottlenecks to experimentation. Let me know if there is anywhere else I should focus on.