To not care is based on an act of fear.
Everyone is entitled to their own opinion. You can accept it as truth or not. But regardless, you should always accept their opinion but stay neutral.
Anyone who has an opinion of you, whether positive or negative, is actually offering you their time and you should be so honored because time is very valuable. Think about it. If they really didn’t care to have an opinion of you, you would not waste any of their time. But the fact that anyone has any opinion of you is giving you something very valuable. And it goes the other way, any time you respond, you give them a piece of your time. Staying neutral is far more less time given than anger, wouldn’t you agree?
So just accept opinions as truth or not, stay neutral towards them, thank them for the time they have given you and then move along. When you do this, you are actually standing up to an act of fear being brought upon you, not running away from it. Cheers!