Only those who truly care about you will let you speak your mind freely with them. Those who don’t, are not friends, even if they are members of your own family. And therefore even these are not family, either. In both cases these individuals are merely companions or compatriots or coworkers and co-collaborators.
What you are witnessing in such instances is the conditioned behavior among these individuals to preserve the integrity of the group over the individual. But this is wisdom that has largely been perverted because one cannot care for a forest without caring to some extent for the individual creatures within it.
What is actually at work is the biological tendency to preserve sameness and shun the “other.” There are, after all, forests that have been so successful in breeding one kind of tree that all other species of plant and shrub have been rendered extinct. Tribalism and group survival is all that is at work, nothing more and nothing less.
So you see, you must make a choice about the game you want to play. Be a collectivist and remain silent about what makes you an inconvenience to those that wish to shun those different to them. Or be an individualist and speak freely about what makes you different, even if it rubs against social norms. The choice is yours.