Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a function to process audio track #1

Open
LogWell opened this issue Sep 24, 2024 · 1 comment
Open

add a function to process audio track #1

LogWell opened this issue Sep 24, 2024 · 1 comment

Comments

@LogWell
Copy link

LogWell commented Sep 24, 2024

Thank you for the sample code.

I want to do some simple transformations on audio and video.

There are sample codes for video operations: https://github.com/aiortc/aiortc/blob/main/examples/server/server.py#L22

I don't know how to implement the process function in AudioProcessingTrack(just for some simple operations)
in this file: https://github.com/aiortc/aiortc/blob/main/examples/webcam/webcam.py#L83

【I tried something similar to the video, but the code doesn't work】

Will you do me a favor!

server.py.txt
index.html.txt
client.js.txt

@LogWell
Copy link
Author

LogWell commented Sep 24, 2024

return frame is OK, but others are NG!

class AudioProcessingTrack(MediaStreamTrack):
    kind = "audio"

    def __init__(self, track):
        super().__init__()  
        self.track = track

    async def recv(self):
        frame = await self.track.recv()
        new_frame = await self.process(frame)
        return new_frame

    async def process(self, frame):
        # return frame  #! OK

        gain = 1.5
        for p in frame.planes:
            samples = np.frombuffer(p.to_bytes(), dtype=np.int16)
            samples = np.clip(samples * gain, -32768, 32767)
            p.update(samples.tobytes())
        
        new_frame = AudioFrame(format=frame.format, layout=frame.layout, samples=frame.samples)
        new_frame.pts = frame.pts
        new_frame.sample_rate = frame.sample_rate
        new_frame.time_base = frame.time_base
        return new_frame

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant