And, circling back to ports, you can make firewall rules that prevent devices from talking across VLANs on certain ports. Your Nintendo Switch doesn’t need SSH access to your KNX server, to re-use your previous example, so you block your console’s VLAN from being able to talk to your server VLAN at all.
The best way to do it is to block literally everything between VLANs, and then only allow the ports you know you need for the functionality you want.