Thanks to another user i got home assistant notifications with TTS running.
So everything can stay muted and i can let the phone talk via home assistant. Neat part is that this works even with both our phones lost and i can just ask my voice assistant where my phone is and it starts speaking