I don't know who I'm posting this for, but you may find it interesting. I've been adding subtitles to @interpeer_talks videos (not all finished yet), and here are some notes.
Whisper has an annoying API in that it'll extract speech from all the video before returning it, which means if your process fails at any time before completion, you won't get partial text. Who designs an API like that in this century?
