I've been thinking about it lately, and I even wanted to change it quickly, but it turns out it's not easy, and I wouldn't want to make such a big change hastily. It will have to wait a bit because I have a queue of critical tasks, and more external matters are diverting my attention. I've also decided to make the placement of the boost/fav entirely configurable by the instance admin.
Honestly, I love the current state, I'm very used to it, and I would like to keep it. At least on my smaller instance. I also have plans for a short interactive tutorial that will briefly explain how the most important features on the website work.
However, I completely understand the arguments that some change is needed. But making it configurable won't take much more time than making it fixed. Downvotes won't be federated.