There must be more to this and a way around this paradox other than a vow of silence.. But I can't put my finger on it..
There is a simple solution it is to not worry about what other people may think of you. Take it in to consideration but don't take it to heart. When a person who sees something as egotistical when it isn't there it is their issue not yours.
It is not what is being said but how and why it is being said that it the big deal.
It really isn't a paradox when you think it through.