You definitely would have legal issues redistributing the ad-free version.
Sponsor block works partly because it simply automates something the user is already allowed to do - it’s legally very safe. No modification or distribution of the source file is necessary, only some metadata.
It’s an approach that works against the one-off sponsorships read by the actual performers, but isn’t effective against ads dynamically inserted by the download server.
One option could be to crowdsource a database of signatures of audio ads, Shazam style. This could then be used by software controlled by the user (c.f. SB browser extension) to detect the ads and skip them, or have the software cut the ads out of files the user had legitimately downloaded, regardless of which podcast or where the ads appear.
Sponsorships by the actual content producers could then be handled in the same way as SB: check the podcast ID and total track length is right (to ensure no ads were missed) then flag and skip certain timestamps.