Interesting fact: humans are the only animals which follow gaze or even a pointed finger. What do I mean? When you point over there to your dog, he will look at your finger, not where you're pointing. Humans are the only animals that will look at where you're pointing/looking. What does this tell us about humans? We have social cues programmed into the very fabric of our being.
When you're at a party or in a large group and suddenly you see a big part of the group's eyes light up, they smile, and they're all looking behind you - what do you do? You turn around, and you probably see the popular, fun guy, and/or the really hot girl has just walked in the room.
Our perception of the world is deeply tied into how the rest of our social community reacts to it. Check out this funny video:
http://www.youtube.com/watch?v=Xo1R17Gq6dg
Especially the very last prank they do. There is NO danger, but just because enough people react AS IF there's danger, the victim of the prank goes into self defence mode.
Thus, it's only natural to have our view of OURSELVES based, at least somewhat, on other people's perception of us. If we didn't care at all, we'd be sociopaths. The problem is, a lot of people care TOO MUCH, and that's when problems can arise. One of the best little catchphrases I saw was in someone's sig on this forum. It read "If I don't like something about myself, I'll change it. If you don't like something about me, then fuck you." And I think that's basically it. Try your best to improve aspects of your personality and life, but when it comes down to it, don't let other people allow you to feel negative about yourself.