Biases in Research

I’m not impressed by ideology. Perhaps this is because I’ve invented several of my own.
Solomon Short (David Gerrold)

It’s strange how you can get involved in some topics. I’m currently working on a mobile application to improve critical thinking reading obese people. The aim is to reduce stigmatization of obese individuals, i.e., to fight negative discrimination.

Given that I like to know what I am dealing with (if only to understand comments from slightly overweight colleagues like “Your research makes me depressive.”), I looked around for possible ways I was (negatively) discriminated against. Difficult, given that I am white, male, straight, sane (more or less), middle-class, etc. But, there was something … strangely enough, being a man is heavily (negatively) discriminated against.

Frankly, it was a bit like sitting in a theater or opera, looking at the stage and suddenly realizing that this was only one stage, and not even the main stage. That there is a whole new story playing behind you. And once you realize it, you see how this (negative) discrimination works in a lot of areas.

For example, take the Generation 35 Plus report. Ostensibly, the report was about the situation of post-docs in Germany. However, the introduction by Prof. Dr. Johanna Wanka (minister for education and research) started with bemoaning the difference between more women achieving the general qualification for university entrance/graduating more frequently from universities, yet still being underrepresented in the higher echelons of business companies.

Note that the focus here was on women being underrepresented in board of directors and the like (doesn’t this usually take some decades after graduation?), not on men being underrepresented when it comes to qualifications for university or finishing universities. That seems to be irrelevant.

Very strange in a report about the problems of highly qualified people in industry and science.

Even worse, the report made assertions about gender differences and the success of assistance measures for women. The problem? The whole study did interview only 10 men and 10 women. Given they created three categories the number of interviews is too small to do any statistical analysis to determine gender differences between the categories. Even non-parametric statistics cannot by applied here.

It just does not make any sense.

Personally, I think scientists have a responsibility. Their assertions should not be mere guesswork or in the service of ideologies. No matter what a scientist — male or female — might feel, the question is what the data supports, not what some people want to achieve in society. And in this case, the data did not support anything.

How could such a report be even published?

But okay, it’s not a peer-reviewed article. It’s not like other scientists did say that this report was any good.

Only, there are also these kinds of papers that made it into journals despite being severely flawed.

Take, for example, the article “Deviant Behavior in Computer-Mediated Communication: Development and Validation of a Measure of Cybersexual Harassment” in the “Journal of Computer-Mediated Communication”. The author did do three questionnaire studies and asked samples (60%, 60%, and 32% of which were women) questions about the occurrence of cyber-sexual harassment.

The problem?

Among the questions were items asking if the respondent did ever:

6. Send an e-mail entitled “25 Reasons why Beer is Better than Women” to your coworkers.
7. Post the following comment on a work-related bulletin board “am I the only one with the balls to stand up”.
13. Send e-mails to your co-workers that joke that women are inferior to men.

and whether this person did agree that

18. It is safe to express prejudice against women online.

These items are distinctly worded to cover only cyber-sexual harassment of men against women. This is in line with the introduction/theory of that article talking about  “how women would be treated”, “misogynist behaviors that are insulting, hostile, or degrading towards women”, “women are in danger of new forms of harassment”, “allowing for the harassment of women”, etc. — following the usual stereotype of women as sole victims and men as sole perpetrators.

The worst part — this is a questionnaire that might be used to determine gender differences in cyber-sexual harassment — while having an intrinsic bias against cyber-sexual harassment by women towards men.

The questions could have easily been worded to cover both genders, and even actions like:

Send an e-mail entitled “25 Reasons why Beer is Better than Women” to your coworkers.

could have been phrased as whether the person has ever sent an eMail that might be viewed as derogatory by the other gender, or included the female alternative like “32 Reasons Why Cookie Dough is Better than Men” or the like.

But no such luck.

Personally, I think this scale is completely worthless — it is not gender neutral. No way of knowing whether differences between men and women occur due to actual differences in discrimination, or because 4 items were only applicable to cyber-sexual harassment of men against women.

And — strangely enough — these are not the only examples. As a scientist, I find this behavior atrocious. And I’ve started to ask for clarification. I’ve contacted both (corresponding) authors, got a reply in the second case, but not a very satisfactory one. When I asked more specifically:

While reading your article, I got the impression that the goal was to develop an universally applicable questionnaire. After all, you asked for *actions* of sexual harassment and in your sample, 60%, 60% and 32% were women. If women are excluded because they cannot be/are highly unlikely to be perpetrators (a suggestion to which I do not agree), it would have made no sense to use women in the sample.
In the same vein, I am also a little skeptical about using court cases as reason for items that are only applicable to one form of harassment (male perpetrators, female victims). Norms might influence how cases progress — e.g., it might be considered “unmanly” to bring a case of sexual harassment to court (e.g., because the norm is to welcome female attention, even if the person is not attractive and obnoxious, or because of a perceived fear to appear homosexual if the perpetrator was a man [actually, I meant “if the perpetrator was a woman and the man rejects it]). If a questionnaire mirrors these norms, it will miss a lot and can only confirm what is already ‘known’.
I think it’s much better to ask for concrete actions in a neutral way, leaving gender out of the questions. This bypasses interpretations (e.g., men who do not see certain actions as harassment, even if they are the *victim* of it) and allows for a fair assessment of sexual harassment, independent of gender. After all, sexual harassment can happen in all forms — a) men as perpetrators, women as victims, b) men as perpetrators, men as victims, c) women as perpetrators, men as victims, and d) women as perpetrators, women as victims.
And only an unbiased questionnaire can bring light into the (empirical) question how frequent online sexual harassment is and about the distribution of perpetrators and victims across gender and sexual identity.

there were no further replies.

I mean, seriously, what’s the deal here? Are we like Eastern Germany/the USSR a couple of decades ago, faking research to promote state ideology? Don’t we know already where that road leads us?

I find it deeply unnerving that such a ‘research’ is even published.


Update: 2013-11-26: Corrected some errors and improved clarity.