I’m in the first bit of the Depths they send you to, where you have to light 2 of those big roots and talk to Robbie. He then tells me to go back up but the map doesn’t allow me to fast travel.
I read online about a 4th shrine on Sky Island that you have to teleport to before jumping down but I never knew about that so it seems the game just decided I could play on without that feature.