Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AnswerToSpeech #2584

Merged
merged 74 commits into from
Jun 15, 2022
Merged

AnswerToSpeech #2584

merged 74 commits into from
Jun 15, 2022

Conversation

ZanSara
Copy link
Contributor

@ZanSara ZanSara commented May 20, 2022

Proposed changes:

  • Add two new primitives, AudioAnswer and GeneratedAudioAnswer, that contains references to audio files into their answer and context fields.
  • Add a new node called AnswerToSpeech that can convert regular Answers into a GeneratedAudioAnswer.

Open questions:

  • What is your opinion on the new primitives? Is this a direction we would like to take to support multimodal input-output in Haystack?
    • Note, this is a stub: if we like hierarchical primitives, I will do some refinement and come up with a clean hierarchy.
  • Shall I add a short tutorial on how to use this node?

Status (please check what you already did):

@ZanSara ZanSara added type:feature New feature or request topic:audio labels May 20, 2022
@ZanSara ZanSara marked this pull request as ready for review May 25, 2022 14:17
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very promising! I left a few comments that we could discuss and I would like to see a small example pipeline/tutorial. Before we can merge this we would need to add docstrings and also test cases so there is still some work to do but so far it looks great! 👍

haystack/schema.py Outdated Show resolved Hide resolved
haystack/schema.py Outdated Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Outdated Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Outdated Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Outdated Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Show resolved Hide resolved
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍 This is still a bit experimental I'd say and our documentation could mention that. It will be good for us to see how the community uses this feature. We could even ask for feedback on Haystack slack explicitly. Maybe a community member wants to showcase a use case, who knows? Let's make it as simple as possible for the community to try this out. I left a few comments regarding doc strings and I'll add the needs-documentation to make @agnieszka-m aware of this upcoming new pipeline node.

haystack/nodes/audio/_text_to_speech.py Show resolved Hide resolved
haystack/nodes/audio/_text_to_speech.py Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Outdated Show resolved Hide resolved
haystack/nodes/audio/document_to_speech.py Outdated Show resolved Hide resolved
haystack/nodes/audio/answer_to_speech.py Outdated Show resolved Hide resolved
haystack/nodes/audio/_text_to_speech.py Outdated Show resolved Hide resolved
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@ZanSara ZanSara merged commit 584e046 into master Jun 15, 2022
@ZanSara ZanSara deleted the text2speech branch June 15, 2022 08:13
@ZanSara ZanSara mentioned this pull request Jul 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic:audio type:feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants