It’s curious, I have a similar story but with different countries, and the reactions are VERY different. I was born in Argentina, but my family emigrated to Brazil when I was 13 years old. I speak fluent Portuguese but obviously have an accent that people can’t quite place, but once it’s pointed out they notice it. Yet the vast majority of my interactions about it are something similar to:
Where are you from?
I was born in Argentina, but lived in Brazil over 16 years
Ah, so you’re mostly Brazilian then
And I think that that says a lot about Brazilians and how they’re very welcoming and friendly. Unfortunately the British don’t seem to be the same way, at least from your experience, maybe people in larger cities are more used to immigrants so they would see you as mostly British or something.
As for the voting, for me at least the only way was to become a citizen, most countries allow you to ask for citizenship if you’ve been living legally long enough so you probably qualify. Just bear in mind that some countries ask you to abandon your other citizenships when you do so, so not sure if that’s your case and if it’s worth it just to be able to vote.