To me it's wrong when you judge somebody for what they are and can't control - gender, skin color, etc. But it's OK for actions or choices - racists, misogynists, etc.
But lately I've been finding just how blurry that "line" is, and it's looking more like a nebulous continuum. For example, take a positive/negative pair like ambitious vs. lazy, or rational vs. impulsive. It's not clear to me that people choose to be ambitious or rational, it seems closer to the truth that people have personality traits or preferences that predispose them to those behaviors.
Take an Elon Musk for example. He works very hard, and is constantly pushing the limits. But is that because he's a lazy, unmotivated person who has chosen to work hard even though he hates it? Or is he driven by achievement and loves the manic pace? I'd bet 95% the latter, maybe 100%.
How many assholes act that way because they enjoy hurting other people, while most of us dislike it? It's not like I want to be an asshole and I deny myself constantly so I'm a nice person...I just don't like to be an asshole.
Some people plan and save for the future, I'm definitely in that category. Others live for the moment, and trust the future will somehow just work out. One is virtuous foresight, the other is failing the marshmallow test. But I don't plan for the future against my instinct to live for the moment; I feel uneasy when I haven't thought things through or planned it out. It's not a virtuous decision, it's just a dislike of having uncertainty hanging over me.
So to the extent that many of these behaviors are not choices, but preferences, is it justifiable to look down on others for them? Take the "stupid" discussion, or more accurately ignorance. Some people, like me, love learning, probably because we're good at it and it's rewarding. Others dislike it, perhaps because they get much less reward for the effort, and the process may be unpleasant. So are we justified in looking down on them or punishing them for their ignorance?
This is a bit abstract, but I hope the examples have made it reasonably clear.