Batteries have an insulated separator between the positive and negative sides. They design the battery with a particular maximum voltage in mind, so they engineer it with a separator that is always a higher resistance. Thus the electrons will only be able to make the jump when a circuit with lower resistance is formed.