Ah, but if a president can do whatever they want without being held legally responsible, then they can punish someone regardless of whether or not they’re legally responsible for what they did.
That comes with a fun loophole - if the courts decide that presidents are immune to all legal repercussions, then Biden can just illegally arrest Trump and throw him in prison for no reason, and suffer no legal repercussions for it.