I finally worked out out a way how to use #Dashy with both local access, and via VPN.
Dashy allows to create subpages, and so right now I have a main page with links pointing locally (192.168.0.XXX), and a subpage with links pointing at the VPN IP of my homelab. This way I can easily switch between dashboards.
This has been bugging me for some time, I am so happy to find a clean solution.