Hi, are there any resources on how i might leverage Metaflow to build and scale servers for live speech to text inferencing using whisper for 1000+ concurrent users ?

@User any thoughts here?

Hey @freezing-controller-39043... this seems super interesting. Can you share more details of what you're trying to do? This blog post discussed using Metafow for transcribing using Whisper. Might be useful. But, the scale is certainly different. Also, how long and how big are the audio file/s. Whisper can run on GPUs or even CPUs (but it will take more time).

Thanks for getting back @square-wire-39606 and @proud-eye-90172. > Can you share more details of what you're trying to do? @proud-eye-90172, I'm building adalat ai, we're a non profit legal-tech company that's building ai solutions for the judicial system in India to end judicial delays and improve access to justice. Our flagship product is a legal speech to text transcription tool. For which we have fine-tuned whisper like encoder-decoder models. > This blog post discussed using Metafow for transcribing using Whisper. Might be useful. Thanks i've gone through it, our usecase is slightly different. We want "live" speech to text, not over REST, but websocket. So that the transcription is real time as a user speaks. > Also, how long and how big are the audio file/s. These are typically conversations that go on for about an hour, but as you can imagine, the size of audio file is not the biggest problem here, right now, as we're building live transcripion.