Not quite. What it means is that people on my instance don't have the same view as people on other instances. When we visit a community or look at a post, we see it as if downvotes don't exist, wherever those downvotes come from. You seem them with downvotes included.
How you see my post doesn't bother me. How I see other posts is what I'm interested in, and for me, downvotes don't influence ranking