feat: introduce proposal design process

[masci]

Related Issues

• fixes n/a

Proposed Changes:

• Introduce a formal process to detail substantial changes to Haystack
• Migrate one old similar doc (ADR approach) to the new process

Notes for the reviewer

ADR feels limited for our use case:

• it only captures a decision and the reasons behind that.
• it's written AFTER something has already been decided and/or implemented

A proposal on the other side:

• it is written BEFORE even adding code
• it contains plenty of details about why it is needed but more importantly how it would be implemented

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added tests that demonstrate the correct behavior of the change
• I've used the conventional commit convention for my PR title
• I documented my code
• I ran pre-commit hooks and fixed any issue

[vblagoje]

LGTM, looking forward to using it! @masci I can open the RFC template in logseq and work on it just like any other markdown document I already write in logseq. This is awesome!

[julian-risch]

One remark from my side is about the internal use of ADRs in other teams (link to private repo). It would be good if we could keep the RFC process and the ADRs aligned as closely as possible. Let's not have different processes across teams if it's avoidable. As the RFC process will also influence other teams, I would suggest asking them for feedback as well. Getting diverse opinions. 🙂
@masci Maybe you could add a brief explanation why to replace ADRs with RFCs in the PR description? And if you would like to get more feedback tag a few people from other teams? Otherwise the PR looks good to me. 👍

[bogdankostic]

I agree with Julian that we should explain why we are replacing ADRs with RFCs, other than that, this looks really good to me :)

[masci]

@julian-risch @bogdankostic ADR and RFC are not the same! Also, as much as I love consistency, different teams require different tools, specially our team that shares the project with a community of 100+ external contributors.

Also worth mentioning ADR were never really adopted in Haystack (there was only one) so we're not really "replacing" an old process, think about this PR as introducing one.

[vblagoje]

What do we need to do to integrate this PR? I am excited to issue the first RFC, which I am sure @ZanSara wants to improve with her ideas for multimodality and more. Let's kick off this process @masci

[masci]

@vblagoje waiting for @tholor to provide feedback on the initiative at large.

In the meantime, can you open an issue using the same template in this PR, without mentioning the RFC process so we can start discussing your proposal?

[tholor]

@masci @vblagoje Yep, would like to wait for some feedback from the team survey this week, before officially launching this process and sharing it with the community.

Of course, this shouldn't stop you from using the template here already to share and discuss a proposal @vblagoje.

[tholor]

Good to go from my side!

Sorry for holding this one up, but I felt it was worth understanding what we want to improve with it and what the biggest pain points are right now. The team survey + conversations definitely helped with that.

So, let's experiment with it and see if helps us to improve documentation, speed and clarity of decisions. I could also see that we do a few small iterations on the exact format / process after we made some learnings.

[vblagoje]

LGTM