That's not really a culture I want to be fostered here on kbin.social. Kbin instances are part of the wider fediverse, and there's nothing about someone on kbin.social that makes their vote more important than someone else's vote. If Kbin had a reaction system (like I mentioned in another comment) instead of downvotes, then not federating downvotes would make more sense, as specific "this is spam" or "this doesn't contribute to the discussion" reactions are more useful than just a down arrow. But the system we have right now is extremely unintuitive to new users and promotes a culture of "Eh, downvotes from people on other instances don't really matter. What's important is what people who are arbitrarily on my instance think."
If you really want to know which votes are local and which ones aren't, it'd be much better to either
have an indicator next to each user in the activity tab showing whether they're local or not, like what KES does with threads & comments, or
have something you can click that lists the numbers of local votes and remote votes separately.