Numerous experiences in my life have proven to me that this is partly accurate and op has grounds to be worried.
I cannot count the times i was lost in thought in a public space and someone interrupted me with “the fuck you looking at?” or “is something wrong?” type of questions.
People notice. You notice. Sometimes it’s ignored, but enough times you need to unscramble your thoughts to answer a “the fuck you looking at?” out of nowhere.
Having a phone is the modern excuse. Just stare at the screen or pretend you’re scrolling.
Yet, having a low self-esteem will make one anxious in a public setting quite unreasonably. And while hard to achieve all the time, an answer is in fact to stop caring about what other people think. It does work.