so they could be edited out of the episode and then the user could also download said episode where ads are cut out of the final audio file.
This is the part that might be problematic and I can see being part of a civil suit (I am not a lawyer). Depending on how you collect and store the episodes (which you may not actually have to do to achieve your goal, but is the easiest solution) you would likely run afoul of "distribution" precedents in the US that may result in a judgement against you.
But even if you didn't actually break the law, the media lobbies globally are well known for filing huge numbers of lawsuits over anything that even looks a little like it might be costing them money. Defending yourself at all is hard time-consuming and often expensive. It's not something I would recommend going into casually.
https://torrentfreak.com/category/lawsuits/ is a great site for learning about the current lawsuits from a tech perspective, and has helped me out many times over the last decade. It's one of the gems of the internet, in my opinion.