RFC: Streaming Inference / Application

[mthrok]

Torchaudio team is looking for a way to support streaming applications. We are trying to define the problem space and scope the challenge we tackle. For this purpose, we would like to learn your thoughts and experience in streaming applications. If you have a thought, please let us know by leaving a comment.

Questions include, but not limited to

• How are you feeding your input stream to your system?
  • WebSocket? FFMpeg + STDIN? gRPC?
• What technology stack do you use?
  • Kaldi? ONNX? TorchScript? TensorRT? DeepStream?
• What are the pain points in your application lifecycle?
  • Development?
  • Deployment?
  • Maintenance?
• What type of application do you run?
  • Speech recognition?
  • Audio enhancement?
  • noise reduction
  • audio event detection
• What kind of device is your application running on?
  • Web server?
  • Desktop system?
  • Mobile device?
  • Embedded system?

[tongjinle123]

1、grpc
2、onnx，torchscript
3、deployment
4、speech recognition
5、～

[mthrok]

@tongjinle123 Thanks for the comment. Is your production environment Python or C++?

[hbredin]

As promised in #1442, here are my potential needs for pyannote.audio.
I don't have much experience in this area so bear with me if my answers fall flat.

* How are you feeding your input stream to your system?

I'd love to be able to expand this streamlit demo with the actual microphone of the end user. I guess with something like streamlit-webrtc.

* What technology stack do you use?
  * Kaldi? ONNX? TorchScript? TensorRT? DeepStream?

None of these for now... Just regular PyTorch as pyannote.audio is designed for research purposes, not actual production.

* What type of application do you run?

• speaker diarization
• audio event detection

* What kind of device is your application running on?

• Web server