The real bias is ‘hotness’
Professors often hate student evaluations because they don’t believe they measure teaching quality. But they don’t have to hate evaluations for another reason that has been debunked: bias against women.
Two psychology researchers, one male and one female, scraped ratings from a million professors on RateMyProfessors.com, which is an “acceptable proxy” for evaluations because previous research has shown these reviews by students have a high correlation with in-class evaluations.
They found that not only were differences negligible between rated men and women whose gender could be “unambiguously” identified, but that high “hotness” ratings on RateMyProfessors correlate “extremely closely with positive rankings.”
Pascal Wallisch, clinical assistant psychology professor at New York University and head of its Fox lab, and Julie Cachia, who has taught courses on advanced psychological statistics and cognitive neuroscience, take a skeptical view of earlier research claiming to show bias against women in student evaluations.
They write in Slate that these claims are important because they have been used to argue in favor of ditching student evaluations entirely, and yet they are, methodologically speaking, a joke:
The available research is complicated and varies in terms of quality as well as findings—some studies report a bias against women, but some note that female instructors receive slightly better ratings.
For instance, a recent study purported to show that if students believe that their instructor is female—regardless of the actual gender of the instructor—they rate the instructor statistically significantly lower in terms of fairness, praise, promptness, and overall rating. But this study was severely underpowered, with a sample size of 20 per group.
It was also based on an online class – still a minority of college classes – and stereotypes have been shown to be stronger “if there is more uncertainty”:
If there is less uncertainty, as in an in-class setting, the effects of stereotypes could be expected to be much diminished. Indeed, there is usually no significant gender-based difference in student evaluations for in-class (not online) settings.
Another study, inexplicably published by Slate, literally used “an anecdote” for its entire sample size – one male and one female instructor in an online course:
How do the authors know that it was gender that the students focused on and not—say—teaching style? People are very different to begin with, so any experiment in psychology needs a large sample size to make sure that the signal one is looking for (in this case gender differences) is not drowned out by what is pejoratively called noise.
https://twitter.com/clayroutledge/status/986408714965016576
In the research by Wallisch and Cachia, they found “extremely subtle” gender and stereotype effects that are probably offset by “gender-related confounds.”
Male professors now have more experience on average than female professors, which correlates with higher ratings, and women are “overrepresented in the tails of the distribution” – more “truly amazing” and “absolutely terrible” reviews:
We want to emphasize how surprised and shocked we were when we saw this. That’s because in virtually all other domains anyone has looked—income, general life outcomes (e.g., having a powerful position vs. being homeless or incarcerated), and even number of offspring—men are overrepresented in the tails of these distributions, with women being overrepresented in the middle.
Their explanation for the tails discrepancy is that women are both benefiting and suffering from gender expectations:
As women have entered this profession relatively recently (in large numbers), it is reasonable to posit that stereotypes against them manifest as lowered expectations overall. The rest is basic psychology. If a female professor is objectively good, beating the low expectations, she will be perceived as amazing—people will think that she overcame all the gender-based obstacles that were put in her path and came out ahead anyway, and she will end up on the far-right tail of the distribution. If a female professor is objectively bad, underperforming even the low expectations, she will be perceived as truly awful, students might suspect that she was hired just because she is a woman, and she will end up on the far-left tail of the distribution.
The biggest correlation in their research is from the “pepper” that RateMyProfessors awards to faculty with a high hotness rating, and “and in our sample, it is as strong in men as it is in women”:
As this is purely correlational, we can’t discern if attractive professors get good ratings or if professors who get good ratings are perceived as attractive. There are even other possibilities: Maybe professors who are good at their job are more confident than others and are thus perceived as more attractive. Maybe students “award” a pepper as a reward for a job well done. We can’t discern these possibilities here, but we don’t have to. We just wanted to illustrate what a strong effect looks like, and the effect of perceived physical attractiveness on student ratings is strong, whereas gender effects are not.
MORE: PC critic accuses university of using classroom joke to fire him
IMAGE: Ruslan Guzov/Shutterstock
Please join the conversation about our stories on Facebook, Twitter, Instagram, Reddit, MeWe, Rumble, Gab, Minds and Gettr.