I think a big current issue on nvidia at least is that cuda/drivers do not elegantly allow multitasking. Games/video can tank performance of AI, and AI tanks performance of games/video, even if the card has capacity to run them both at the same time. I expect the solution is to solve this at a driver/OS/CUDA level and then just keep improving card specs each generation.
We may see a greater share of GPU silicon go to AI oriented hardware too in time, depending on the application.