The problem with that approach is that the resulting AI doesn't contain any identifiable "copies" of the material that was used to train it. No copying, no copyright. The AI model is not a legally recognizable derivative work.
That's a HUGE assumption you've made, and certainly not something that has been tested in court, let alone found to be true.
In the context of existing legal precedent, there's an argument to be made that the resulting model is itself a derivative work of the copyright-protected works, even if it does not literally contain an identifiable copy, as it is a derivative of the work in the common meaning of the term.
If the future output of the model that happens to sound very similar to the original voice actor counts as a copyright violation, then human sound-alikes and impersonators would also be in violation and things become a huge mess.
A key distinction here is that a human brain is not a work, and in that sense, a human brain learning things is not a derivative work.