It's a lot harder to follow your dreams in life than not. People say following dreams is lazy, they say it shows that you don't take life seriously or that you live in your own head and are selfish.
This is completely incorrect, people who follow their dreams are inspiring and brave and it usually takes a huge amount of work to achieve their dreams, lazy is not the right word to describe those who follow their dreams at all... continued in comments
It's likely that doing things that the judgemental people want you to do will not make you happy, so you need to make a choice just for you. It's not easy, it's definitely not lazy but it can absolutely be worth it.