Deeply Misguiding — Problems with Surveys on Inequality

[The children in a summer camp are just starting their life-saver swim course]
“I’ll be the victim.”
“All your life.”
Amanda and Wednesday in “Addams Family Values” (1993)

Surveys on “Gender Inequality” are often shocking — but for reasons that have nothing to do with actual gender inequality in society. Many surveys are conducted to “confirm” the prejudices of the people conducting them. Thing is … the answers you get in a survey depend strongly on the questions you ask and many “shocking” answers are the result of biased questions — and a consequence of poor study planning, followed by misguided implementation and interpretation.

Given this invitation:

question_girlguiding

and a couple of tweets I made regarding the invitation from the #girlsattitudes survey:

answer_girlguiding

I decided to go beyond 5 * 140 characters and write down why I think that many studies/surveys on gender inequality are deeply flawed.

To avoid sounding overly negative — which just seems to be the issue with activism these days, doesn’t it? — I have written down some recommendations to conduct better studies. I start with some general issues, then — to give a concrete example — I focus on the #girlsattitudes survey (report and — more importantly — the data file here). With the girlguiding survey, I’m focusing mostly on the questions asked — as they determine the answers.

However, these issues are applicable to most reports in the media about studies that supposedly show the inequality between the genders in society. They can be used to plan and conduct better studies and to criticize existing studies.

#0 Be Open About Your Conflict Of Interests, your biases, and actively try to work against them

Why do you want to conduct the survey? What do you expect to gain from it? If you have something at stake, e.g., you provide guidance for girls and you want girls to need this guidance, or if you want to head an initiative for more protection for female journalists, say so.

In any scientific publication, possible conflicts of interest must be stated, because no human being (not even a hard-core scientist) is free from biases and stating possible conflicts of interests makes readers aware of them and allows them to check more efficiently for possible biases.

Speaking of biases, it is nice if you “passionately believe that girls have the potential to be leaders in all walks of life, to be a powerful force for good and to be inspiring advocates for change [but they need guidance to realize their potential that we can provide]” (girlguiding). But don’t be a fool and assume that this strong belief does not influence the way to ask questions. It’s a strong bias that skews your perspective and you need to be aware of this bias — if you want to attain results that allow you to see more then you previously did. Intercultural, interdisciplinary, and international cooperation is often needed to become aware of this biases in ones perspective. It’s too deeply ingrained to become aware of it yourself.

It seems counter-intuitive to conduct a study where you (also) try to disconfirm what you strongly believe in, but this is necessary to guard yourself against your biases. Ask honest questions that can actually disconfirm your world-view.

Coming up with good questions of this kind and avoiding group-think is hard. Frequently this requires interaction with people who see the world differently, people you might normally avoid.

This also means that it is a really bad idea to conduct a survey or study as part of a funding effort. Sure, the results you get with a biased study might give you ammunition to aim for (more) funding, but the thing is — empirical studies/surveys can be easily evaluated for their merit. It’s what scientists do for free (peer-review isn’t paid). And if you do a biased study/survey — even if the issue is really existing and has grave consequences — scientists will identify the flaws and righteously call bullshit on your study. And you have wasted a chance to make life better.

#1 If You Want To Find Out About Negative Sexual Discrimination of Girls/Women, You Need to Look at Boys/Men Too

Inequality means differences between two groups, here male and female. But it is not enough to show that girls/women face negative sexual discrimination, you have to show that this is unique for girls/women. Whether it’s negative sexual discrimination in school or at work — you can collect thousands of examples — unless you look at the other side it does not mean a thing.

You might just be complaining about the rain on your birthday while the other group is being water-boarded all the time. You need an explicit, correctly assessed comparison standard.

For example, girls in schools are harassed — okay, and female journalists might get a lot of threats — but do you really think that it’s only girls who are harassed in school or that male journalists are untouchable and beyond reproach? Or is it just that boys/men just do not talk about it?

It’s hard to see, but most negative sexual discrimination against men is invisible because it’s expected and seen as social norm. Just take “Women and children first.” — no arguing about children, but “women … first” is hardly equality.

By the way, if you are feeling disgust right now, look at the table below which shows some of the differences in the instances (not the categories) of negative sexual discrimination:

Category Girls/Women Boys/Men
Judged on Appearance Money, Position in the pack, Possessions, etc.
Suck at Math Languages
Seen as Too Emotional Emotional Cripple (“emotional range of a teaspoon”)
Occupied too much with Gossip, Appearance Sex (“they think with their dicks”)
Getting called a slut, a bitch a faggot, a coward, a creep, a pussy
damages reputation promiscuity, enjoying sex showing any weakness, esp. fear, pain, crying; also asking for help
uncomfortable clothes high heels neckties
desired figure slim tall, strong

It’s nice that “a woman should be judged on her ability not her appearance” (#girlsattitudes), but while being judged on ability is something that is implicitly asserted as something men enjoy, it’s not ability itself men are judged on. It’s the very specific ability to be of use to others.

Men are supposed to be providers and disposable human shields — their lives are not seen as important as a woman’s.

Sure, concrete expressions of negative sexual discrimination are unique for girls/women — “they are judged by different standards to their male counterparts” (#girlsattitudes). For example, the view that if you are a girl, you suck at math. But the categories usually are not.

Boys and men are not judged on ‘what they really are’, but on another similarly artificial and equally unfair standard. It’s a different cage, maybe even a bigger cage, but it’s still a cage. But it’s hard to see unless you actually live as a man.

So, you have to ask the right questions, otherwise it’s like asking men how often they wear a bra or a skirt and use the low numbers as “proof” that men don’t wear clothing thus we can stop making clothes for them.

And — perhaps more importantly — you have to ask these questions in a way that boys/men are willing to answer them. Because one instance of negative sexual discrimination against boys/men is that boys/men are not supposed to show any weaknesses. Think that is a boon? Think again. This means that we usually do not see negative sexual discrimination of boys/men, because boys/men keep silent about it out of fear of appearing weak (resulting in a loss of status), and because society ignores and avoid boys/men who complain. They are considered as sub-par (i.e., they are discriminate against) and this is excused by calling them creeps, boring, weak, cowards, etc.

So most boys/men learn not to show that they are hurt, sad, etc. and thus their emotions are invisible or seen as irrelevant. But this does not mean that these emotions don’t exist or do not matter.

So, even if you are only interested in girls’/women’s suffering or disadvantages, you nevertheless have to assess boys’/men’s suffering or disadvantages as well. And you have to do so intelligently. You have to understand the kinds of negative sexual discrimination boys/men face and how to get honest responses.

And sorry, but a group of women discussing the issue is not enough. You need men, real men, for the job. Not “real men” in the sense of “I can cut myself and do not bleed” kind of sense, but real men who know what it is to live as a man in the current environment. And yes, you might not like MRAs and the like, but they are your opposites whose input you need to include to land in the middle of the distribution.

#2 Go Beyond Disadvantages and Have a Look at the Advantages Too

Discrimination incurs disadvantages and advantages. But while the focus is usually on the disadvantages (for girls/women), don’t neglect the advantages. They are part of the equation too and needed to get a complete picture.

For example, it’s easier for a woman to get help, punishments are usually lighter, and while there is prejudice regarding math, so is regarding languages — only this one is positive.

It’s one thing to complain that women make only 40% of graduates in STEM fields, but asking for higher graduation numbers neglects the fact that about 60% of all graduates are female. Isn’t the overall disparity also an issue?

The problem with assertions like “equality for girls” is that equality requires both sides to be equal — in advantages and disadvantages. A directed approach to deal only with the (perceived) disadvantages of one side leads only to a greater overall disparity. “Equality for girls” reads as “we want girls to be equal in areas where they appear to be disadvantaged, while we want to ignore areas where they have advantages”. That’s hardly equality. It sounds more like Animal Farm: “All animals are created equal, but some animals are more equal than others.” (“Animal Farm” by George Orwell)

Issues #1 and #2 point to a 2×2 table of men vs. women and advantages vs. disadvantages — where you have to look at every cell to come to valid conclusions about inequality (+xy, -xy, +xx, -xx). Even if you think that some cells are empty, you need to address them explicitly.

#3 Don’t Assume That Boys/Men Are the Only Perpetrators

Don’t assume that girls/women are wonderful, because they are not. They are flawed human beings — just like boys and men. And in many instances, they are as much part of the problem as suffering from it.

If “girls feel that they experience high levels of sexism in their everyday lives – at school, online and in the media” be open about who actually makes them feel this way.

Violence by women/girls to other women/girls might not be as overt, as physical, or as direct as violence by men/boys to other men/boys and women/girls, but it does exist. Mostly as relational violence. And it’s frequently more dangerous, because it’s harder to see and insanely cruel. Just read a book like “Odd Girl Out” (by Rachel Simmons) — I mean, seriously, little angels my ass.

So ask how much of the negative sexual discrimination of women is done by other women. How much of the comments about appearance (body, clothes) are done by women, how much of the slut shaming?

After all, many women slut-shame to control sex, which gives them power, esp. control over the relationship. “Sluts”, i.e., women who like sex for its own sake, diminish that power by “giving it away for free”.

This is a potentially devastating problem for organizations providing “safe spaces” for women (“safe, non-judgemental environment where girls can explore the issues they care about while having lots of fun, enjoying new experiences and learning vital skills”). By ignoring that women can be part of the problem a “girl space” can become a toxic environment. The worst enemy is the one that is part our yourself.

In the same vein, how much of the negative sexual discrimination is done by women to men? How many boys/men are called creeps due to social missteps or actual psychological disorders? How are men judged who refuse to slave away and doing overtime to be “providers”? How are men judged who show fear, who cry?

I mean — seriously — if women would just suddenly stop slut-shaming each other and sanction boys/men for it, a huge part of negative sexual discrimination against girls/women would vanish in an instant. But as long as women are each others worst enemy, that future remains a pipe dream.

So, giving the emphasis many hard-core feminists place on women’s impotence, frequently, when it comes to surveys about inequality, the question of how much of that negative sexual discrimination and harassment is relational aggression by other girls/women is completely ignored.

#4 What are the differences really like?

It’s easy to speak of “boys/men” (or also of “girls/women”) as one homogeneous group. But differences usually do not mean that both groups are completely separated, e.g., all boys are better in math than all girls, but usually there is a huge overlap. Have a look at the actual distribution and provide this distribution. Otherwise you oversimplify and mislead. The mean is nice and all, but in many instances it is a highly misleading statistic.

#5 Differentiate within your own group and within the “outgroup”

In reports on inequality, the comparison standard (see #1) is often implicit. Boys or Men “don’t have to deal with this shit”. But boys/men are not a homogeneous group (see also #4). There is a lot of variation.

Just think back to your time at school and the groups of boys at school. There are the creeps, the retards, the geeks, the regular kids, the popular kids, and the really cool kids.

Don’t make the mistake of using the “really cool kids” with insanely high popularity as (implicit) comparison standard. Likewise, if you compare the life of the average woman with the life of a man, don’t use a mash-up of Brad Pitt’s looks, Rockefellers’ money, and Kennedy’s influence as comparison. Because, well, thanks for the “confidence”, but this shows an abysmal lack of empathy and understanding.

Think also of the homeless dude on the street or the social inept (“creep”) that gets shot down in every social encounter — they make the other end of this long continuum.

Sometimes it seems that being a man automatically gets you all the advantages and all the best places. It does not, but it appears this way as the spotlight is on those who are successful. Other men are invisible or are actively avoided.

So be clear about your comparison standards — be clear about what you think is ‘normal’ or ‘desired’. Or what men have to face. And talk to men beforehand to make sure that your views are accurate — for all men, not only those at the extreme positive end of the distribution.

#6 If Possible, Use Objective Double-Blind Measures

Given that there is a lot of subjective bias in evaluating behavior, if in any way possible, use objective, double-blind measures. If it comes, for example, to evaluating threats (e.g., against male vs. female journalists), separate the description from the gender of the victim. Otherwise you risk that people will discard some threats because “men can deal with it” or judge it differently (e.g., as a joke).

If possible, conduct a psychological experiment (a real one, not a “psychological” “experiment”). For example, if you want to show that girls get worse grades for the same performance then show it. Take written exams of girls and boys and give them teachers to grade. Tell half of the teachers the truth about the gender of the author, lie to the other half (i.e., some teachers correctly think that the exam was written by a boy, others think it was written by a girl when it was actually written by a boy). Then compare the grades for the same exams depending on the assumed gender of the author. If there is a gender bias, this way it will become obvious. Do this for different subjects to control for positive sexual discrimination of girls/women.

#7 Provide Clear Definitions of the Terms You Use

It’s astonishing how much debate, conflict, and stress results from different definitions. It’s one of the first rules for a real discussion — clarify your terms. Otherwise you risk agreeing while you hold vastly different opinions (because you use the same term for different things) and disagreeing while you hold the same opinions (because you use the same term differently). Language is not exact or absolute, it’s negotiated or agreed upon.

So if you ask for “sexual harassment”, provide a clear and unambiguous definition of the term. Otherwise your results can mean anything and nothing.

If possible, use narrow definitions, not broad ones. If “sexual harassment” can mean anything from “he looked at my boobs” to “a stranger raped me”, it is not very useful. Sure, you can get high numbers of incidents, but any scientist worth being called “a scientist” will call bullshit of your study.

And don’t assume a certain interpretation of ambiguous questions just because it gets you funding and pays your bills — ask specifically. You might get lower numbers in the extreme cases, but you will get a more realistic picture leading to more helpful interventions.

#8 Allow Respondents to Answer on the Whole Range of Options

Efficiency is beautiful. It shows that the person asking the question has thought about them. So you might be tempted to ask, e.g., whether “Men and women have the same chance of succeeding in their job” or whether “Employers prefer to employ men than women”. If you are dealing with negative sexual discrimination against women, this seems logical and efficient.

But it constrains the scale.

The actual range goes from: “Employers prefer to employ women than men” over “Men and women have the same chance of succeeding in their chosen job” [BTW, what has ‘succeeding‘ to do with employment?] to “Employers prefer to employ men than women”.

If you constrain the answers to negative sexual discrimination to women from zero to full, and do not take into account that some employers might like to hire women than men, you are severely biasing your results.

And if you think that this does not happen, I can provide a counter-example. When I was applying for a job when I was a student, the HR person (a woman) told me that they usually hire women for this job, “because women work differently, you know, more conscientiously”. So don’t tell me “a difference” automatically means that “women are discriminated against”, because it does not.

#9 Differentiate between life-time prevalence and actual scope of the problem

Many studies that ask questions about sexual harassment ask about life-time prevalence. That’s nice, but it does not tell you anything about the scope of the problem.

If you have one sociopathic individual in every school of a thousand+ students, you can get a very high life-time prevalence if that one sociopath is very busy. But these numbers completely mislead you regarding the best course of action.

Even if you only focus on boys/men as the problem (see #3), the question is not how many girls/women have encountered harassment, but — if only boys/men are a problem (a big if) — how many boys/men actually are a problem. In a population of 100 girls, if one man harasses 80 girls, this leads to the same “encountered” values than if 80 men harass one different girl each. But the implications are vastly different.

Strangely enough, questions like “Of the boys/men I know (including family, friends, etc.) x of y boy/men harass girls/women (or I would like to avoid due to harassment, etc.)” are not asked.

The danger here is making harassment seem normal for boys/men. It unintentionally provides these sociopathic individuals an easy way out to avoid responsibility (“I normally don’t do this, but you know how boys/men are.”) instead of getting them the treatment (or confinement) they (or society) needs.

#10 Avoid Speculative Questions

Feminism has extremely active PR. There are numbers thrown around and world-views conveyed that are … impressive (not accurate, mind you, but the way they are conveyed, yup, impressive).

But I think that a survey should capture how the respondents themselves feel, not how they ought to feel.

So don’t ask for perceived inequality, ask for encountered inequality.

For hard-core feminists the first question is more important. They want to find out how well they did convey their message that women are disadvantaged victims. But if you are really interested in actual equality, that is not the interesting question. If you are interested in the actual living conditions then ask for what the respondents actually encountered.

#11 Don’t Weaken Your Sample

As a certain infamous survey found out, you need a representative sample. The people you ask must represent the sample of the population you want to generalize to.

Want to make assertions about the living conditions of your country — don’t ask the rich only. Otherwise you might come to the conclusion that your country is living in milk and honey.

That seems like a no-brainer, but strangely enough, many surveys on inequality have no qualms about weakening the representativeness of their sample. They do so by allowing for a disconnect between “I have encountered X” and “I know someone who has encountered X”.

It might sound very tempting to use these “I know someone who has encountered X” numbers as indicators of actual negative sexual discrimination, but it’s inherently flawed. If you have a representative sample, then these people represent the whole population. Like in the “if the world were a village of 100 people” example each person represents a lot of people in the population. But this also means that each person must represent him- or herself, not serve as proxy to affected individuals.

Just think about all the people you know — you might know someone who, for example, suffers from cancer. If you are asked about the prevalence of cancer in society and you would allow for “I am not affected, but I know someone who is affected” as valid incident of “person affected” you could easily end up with 90 to 100% “affected”. Not very realistic — no matter how much you want these results.

#12 Be very aware of “I know someone how”

In the same vein as #11, be very wary of people asserting that “they know someone who …”. First of all, it’s impossible to verify. Or do you really think these people “who know someone” willingly provide name and addresses? And what would you think of them if they did? Second of all, it’s extremely condescending. Just imagine someone — even your best friend — walking up to you and saying: “Hey, I know what your problem is, it’s ….”. Would you really be happy about it? And would it be accurate? After all, things that seem easy from the outside are one hell of a lot more complicated from the inside.

Ask people about themselves — that’s difficult enough.

#13 Don’t ask people what they have not encountered

Personally, I think if a real catastrophe would hit, I would shine. I would be brave, I would make the right decisions, I would be impressive.

But is this a realistic assessment?

Probably not.

People have a hard time imagining how they would really act in situations they have not encountered. We all want to act courageous, beautifully, and without mistakes — but reality is much more … complicated.

So before you ask any questions about how people feel in a certain situation, ask whether they have actually encountered this situation and compare the reactions of those who have encountered the situation and those who only imagined it. Likely, there will be a difference.

#14 Avoid Leading Questions

Many issues might seem clear to proponents of a certain world-view. For example, for some feminists, the wage gap is a reality. Others think it is not as large once you take overtime and other factors into account.

Thing is — don’t imply certain assumptions when you are asking questions. Don’t assert certain “facts” when you are asking questions.

If you are asking a question, ask a question. If you are stating an assumption (or even a “fact”), don’t combine it with a question, but provide concrete data instead. A question like “I worry about the pay gap between men and women” asserts that there is a wage gap and makes anyone feel stupid who negates that question — not matter whether the wage gap actually exists or actually is that large as you make it appear to be.

Ask honest questions. State what you think the wage gap is (controlled for overtime, job choice, historical developments and the like), ask them whether they think this assessment is correct, and then ask whether they worry about it.

Likewise when it comes to “rape or sexual assault” (do you really think this can be grouped together, that there is no difference between women who encountered ‘sexual assault’ — whatever that means — and rape) — don’t assert your interpretation of the world as fact.

For example, a question like “When you read about the high proportion of women who have experienced rape or sexual assault, it almost makes it seem like a normal occurrence – and that we shouldn’t fuss about it” is not only a bad question because it combines two answers into one (“normal occurrence” and “shouldn’t fuss about”), but it also asserts a certain ‘fact’ — and does so quite ambiguously. What exactly means ‘high’ in this regard (1 in a million? 1 in a thousand? 1 in a hundred?)? And is this supported by real data?

#15 Ask Only One Question at a Time

Suppose someone asks you: “Are you a human being and do you want to give me 100 bucks?” — what would your answer be? Yes? Yes, but? No? No, but?

Never ask more than one question at a time. It just does not work. Some people will interpret the question one way, others another. Just take a look at the question from #13: “Sexual safety – When you read about the high proportion of women who have experienced rape or sexual assault, it almost makes it seem like a normal occurrence – and that we shouldn’t fuss about it”. What is the question here — that it is a normal occurrence? Or that we shouldn’t fuss about it? Hard-core feminists would probably say ‘yes’ to the former and say ‘hell no’ to the later — but what would their actual answer be?

Personally, my counter-question would be:

  1. What do you consider as “rape or sexual assault”?
  2. Why do you think that “sexual assault” (how do you define it?) is equal to rape (how do you define it?)?
  3. Where do you get your data to make the interpretation that this is “high”?
  4. Why do you think that “it seem like a normal occurrence”? Is it really larger than 50%?
  5. Why do you think that just because something is considered as normal means we should accept it? How could anything change if you assume this to be true?

So, just don’t do it. Avoid leading questions (#13) and ask only one question at a time. Otherwise you might feel free to interpret the results any way you like it, but all those who read your interpretation will feel ‘free’ to call bullshit on your interpretation — because no matter whether you actually are right or wrong, you interpretation certainly is bullshit.

#16 Offer an “I don’t care” or “I don’t think about this issue this way” option

Want to force people to take a side — just don’t offer them a way out. Just give them an even number of options — for or against, but no middle ground.

Sure, some people will stop answering your survey, but those you can report on will either be for or against. And with the right questions you can get them to answer the way you want.

For example, if you want to make “Pictures that ‘shame’ celebrities for not looking perfect” an issue, just don’t offer any middle ground. People can either admit that this makes them “anxious about [their] own appearance” or not. But they cannot state “I sooo don’t care about it”.

And there are ways to make both extreme positions an issue.

#17 If You Ask People of Different Ages, Know the Difference Between Cross-Sectional Data and Longitudinal Data

Suppose in 2013, you ask younger students and older students about a certain issue, e.g., whether they have tried a diet after hearing about a celebrity using it. This is cross-sectional data. You are asking different cohorts (age-groups) at one point in time about their experience.

Does this mean that you can treat it as a prediction for the future?

No.

Just imagine if you ask for life-time prevalence of sexual harassment (however it is defined, see #7) and you get 13% in the younger sample and 10% in the older sample. If you have encountered it in your lifetime you have encountered it, there is no way is suddenly disappears. So this interpretation is clearly wrong. It seems obvious to assert that it becomes a greater problem, however. After all, 13% of the younger sample say so and only 10% in the older sample, so it increases in frequency, doesn’t it?

Nope, this is likewise wrong.

Why?

There are a lot of possible differences that could have happened in the meantime, e.g. different interpretations of the question (if you did not define your terms, see #7).

If you are asking the same question to different age groups you are doing a snapshot of how different people see the world today. It does not allow you to predict the future unless you assume that the overall conditions do not change in the future — something that (I think) is impossible to hold (look, ten years ago, no-one imagined Twitter to be a thing). So just don’t make predictions with cross-sectional data, just don’t do it.

#18 Don’t aggregate interval data into dichotomous data

Suppose someone asks you: “On a scale of 1 to 10, now much do you think that the current administration is doing a good job.” and you think that it does not do a perfect job, it can improve, but it is not bad either, so you go for a 5.

Five is more or less medium, isn’t it?

Now imagine that you read a report saying that most people think that the government does a bad job.

You think “Well, that’s harsh, but at least my vote shows a different picture.” — until you realize that the researchers have collapsed all votes on the left side of the scale, i.e., of 1, 2, 3, 4, and 5 into “does a bad job” and the votes on the right side, i.e., 6, 7, 8, 9, 10 into does a good job.

Hey, why are you surprised — your value was on the bad side, not on the good side. Who cares how close to the center it was — when push comes to shove, you were on the negative side.

That’s actually how many surveys on inequality translate once they get to mainstream media. Any ambiguous results have long been streamlined and a “5 out of 10” becomes “against” while a “6 out of 10” becomes a “in favor of …”. In a world of black and white, anything ambiguous becomes streamlined as “for” or “against”.

While this might simplify the results, it oversimplifies them. At the very least, don’t offer more choice to your participants than you are willing to report on when you give your report — you are not in a position to decide how they actually feel.

#19 Provide the whole (anonymous) data

There is much to be said for the courage and integrity of organizations like Girlguiding to provide the actual questionnaire data. Seriously, kudos. Many organizations just provide the reports without access to the actual data or even the questions, like the “International News Safety Institute and International Women’s Media Foundation Global Research Project Investigating Harassment and Violence Against Female Media Workers”.

And given the way the kind of questions can strongly influence the answers, the actual questions and replies are important.

But they are only the first step. While the aggregated data are important to scrutinize the way the research was done — which is, by the way, how actual science operates — the actual data set is crucial. Aggregated data allows you only to scrutinize individual questions, but only the complete data set allows you to ask questions to the data yourself.

For example, if the #girlsattitudes survey find that all those who spend more than 101£ on “appearance” (which includes clothes, BTW, have you no decency?) have an outstanding/good school performance it seems that school performance is not independent from income. And that would be an interesting question to ask the data. Likewise, I wonder whether those girls who invest in their appearance are not happier than those who do not (for non-obvious reasons).

But this is something that only a full data set can provide.

Think of it this way — if you have done a good job, you have nothing to fear. If not, other researchers will provide you with feedback that will allow you (or your successors) to do a better job in the future.

Some comments on the #girlsattitudes survey

I have to admit that I never read the whole #girlsattitudes report. I started to read it, but I was unable to get past my feelings of disgust. I mean, the “International News Safety Institute and International Women’s Media Foundation Global Research Project Investigating Harassment and Violence Against Female Media Workers” simply ignored men (even those who completed — “spammed” — the survey that was intended for women only with reports of how they were threatened — it just didn’t fit the narrative and they were discarded from any future reports, way to go equality).

The #girlsattitudes did so many questionable things that I am hard-pressed to decide whether it was groupthink or simply ignorance and self-serving bias that was responsible.

It’s not that I don’t consider equality as important. I love equality — if it ever were true equality that works both ways.

But mostly, it is not.

There is this strange attitude that women are discriminated against and if women were only equal, everything would be fine. I don’t know if this is true. But I certainly know that being a man is not all that it appears to be — for some feminists.

And don’t get me wrong here either — I love being a man (no, not for some ‘privilege’ but for other reasons).

But I think that women have this strange misconception about what it means to be a man. Perhaps hard-core feminists should talk to post-op FTM transsexuals a year after they have made the transition. If hard-core feminists were open to feedback, that would solve some issues. But these are pipe-dreams.

Let’s just say that being a man has some severe disadvantages — and those are frequently neglected. People, especially feminists bent on ‘equality’ only see the prize, never the price (to quote Neil Gaiman’s “Sandman”).

So here are a couple of problems with the #girlsattitudes survey:

  • #0 right from the box, they show a strong bias they do not guard against: They “passionately believe that girls have the potential to be leaders in all walks of life, to be a powerful force for good and to be inspiring advocates for change [but they need guidance to realize their potential that we can provide]”.
  • #1 They look only at girls/women: This is insufficient to make any assessment about inequality. Even when it comes to understanding of financial terms (“Here is a list of financial terms that you may have come across. Which of these are you confident that you understand?”) it tells you nothing until you know how boys fare. Are were really to believe that boys are these stinky bastards, but when it comes to financial matters, they suddenly turn into Donald Trump?
  • #2: They do not assess the advantages women have. No questions about whether they have gotten lesser punishment at school or elsewhere, or whether their overall performance is better than boys.
  • #3 They assume that boys/men are the perpetrators — implicit or explicit. They do not ask who does the negative sexual discrimination when they ask:
    • “girls feel that they experience high levels of sexism in their everyday lives – at school, online and in the media”
    • “Had comments shouted about your appearance while you are at school”
    • “Feeling embarrassed about wearing school sports kit puts me off playing sport”
    • “Women are judged more on their appearance than on their ability”
    • “Girls are judged harshly for sexual behaviour that is seen as acceptable in boys”
    • “Been made to feel stupid because you are a girl”
    • “The way that women are sometimes criticized for their looks on TV puts me off wanting to be on TV myself”
    • “Many people look down on stay-at-home dads”
    • “Sexism is so widespread in our society, it affects most areas of our lives / Sexism is not really a problem for girls these days”
    • “Pictures that “shame” celebrities for not looking perfect, make me anxious about my own appearance / Pictures that “shame” celebrities for not looking perfect, make me feel better about my own appearance”
    • “Magazine pictures of celebrities are often altered to make them look thinner or to remove wrinkles”
    • “Magazine pictures of celebrities are usually true to what they really look like”.
    • “There is too much discussion about women’s weight in the media”
    • “Girls are judged harshly for sexual behaviour that is seen as acceptable in boys.”
    • “Had unkind things said about you on social media”
    • “Been sent photos or content by people you know that you find upsetting”
    • “Experienced someone pretending to be you online”
    • “Had sexist comments made about you”
    • “Too much responsibility is placed on girls for their sexual safety
      Each of these assertions begs the question “by whom?”. Who does the discrimination? Who judges? And I’m willing to bet it’s not only men.
      These are even some (implicit) hints that women might be part of the problem. For example, the (female) participants are asked whether they have: “Uploaded photos of others that you later regretted”, “Said something about someone online that you wouldn’t say to their face”,  “Say rude things back”.
      I mean, even if it’s institutional, like in sentences like: “Sex education at schools is unbalanced / biased in the way that it treats girls and boys” It’s easy to assume that girls/women are discriminated against. But if the whole sex education class would be about women that question would still get a 100% agreement. You need to be specific about who does the discrimination and in whose favor it is!
  • #4 They do not provide an overview of the actual differences by providing the actual distribution between boys/men and girls/women (given that they do not assess boys/men).
  • #5 They do not differentiate within their own group or within the outgroup (boys/men): Seriously, you might get the impression that boys/men have no problems (then why do kill men themselves much more frequently?).
  • #6 There is a strong emphasis on personal perception, not objective measurement.
  • #7 The terms are not clearly defined.
  • #8 The range of options is severely constrained. You can answer anything from “employers do not differentiate between men and women” to “they prefer men”, but that they prefer women? Not an option.
  • #9 They do not assess the scope of the problem, only life-time prevalence.
  • #10 They ask questions about situations the respondents have not encountered, e.g., #11, #12, and #13: They ask their respondents what other people they “know” might think. Not only does this require mind-reading, it weakens their sample.
  • #14 They ask leading questions, e.g., “I worry about the pay gap between men and women” or “When you read about the high proportion of women who have experienced rape or sexual assault, it almost makes it seem like a normal occurrence – and that we shouldn’t fuss about it”. It make the answers meaningless.
  • #15: They ask more than one question at a time, e.g., #14: “When you read about the high proportion of women who have experienced rape or sexual assault, it almost makes it seem like a normal occurrence – and that we shouldn’t fuss about it”. Barring mind-reading, there is no way of knowing what the answers actually mean.
  • #16: They do not offer an “I don’t care” or “I don’t think about this issue this way” option: Most questions provide only an even number of answers. There isn’t even a middle ground.
  • #17 (not sure how they interpret the data)
  • #18 They aggregate interval data into dichotomous data by asserting that girls/women assert that sexism is still an issue (“Three in four girls and young women feel that sexism is so widespread in our society today that it affects most areas of their lives (75%)”), even if the actual distribution looks like this: sexismYes, more agree, but the distribution is not as dichotomous as the “executive summary” makes it appear to be. Its akin to the old saying that “I do know that the slickest way to lie is to tell the right amount of truth – then to shut up.”
  • #19 They do not provide the whole (anonymous) data set: Seriously, kudos for making even the aggregated numbers available (and the questions!). But still, the whole data set would be much more useful. Just imagine if the data would show that the ones who do things girlguiding would see as questionable like using make-up or investing in clothes would be happier. That would be an interesting result.

Despite their assertion that they want an environment where girls “can reach their own conclusions about the world” I suspect that girlguiding is rather conservative-ideological in its outlook. After all, if you assert that you want an environment that is characterized in way that girls “can reach their own conclusions about the world” — why do you condemn girls (7-10) for using nail polish, wearing makeup, high heels, try to make themselves look older/more grown up than they are, or (secondary) students for shaving or waxing their legs, wearing makeup to school, shaving or waxing their bikini line, wearing a padded bra, have a spray tan/use tanning products, alter their school uniform to make it shorter/tighter or wear revealing or fashionable clothes that they find uncomfortable? Is that really the view of independence and girl power you want to convey? That it’s okay to be who you are, as long as they are what you want them to be?

Any smart and curious girl who wanted to switch into and try out a different role would have to answer yes to these questions. Any girl whose mother showed her how to use makeup would have to say yes.

Conclusion

Yes, it’s trite to say, but I say it anyway: Sexism — negative sexual discrimination — is an issue, for all genders. Unfortunately, there is a lot of biases research going on when it comes to negative sexual discrimination against women. Unfortunately, because it’s just a matter of time until these biases become obvious — with the possible consequence that a lot of effort to combat the existing negative sexual discrimination of girls/women gets shelved because people do not like to made a fool of.

But negative sexual discrimination of girls/women is an issue — as is negative sexual discrimination of boys/men. That’s just the thing I find so absolutely strange — somehow we all want a just world, but (also) somehow some groups fighting against negative sexual discrimination of girls/women have gotten into a position where they think that fighting against an unjust world will cost them influence and power. So they resent fighting against an unjust world and focus on their own localized issues.

In a way that’s smart — they keep influence and power this way. If its only girls/women who are discriminated against and they belong to that group, that gives them one hell of an advantage. But overall, society loses this way. The actual issue is not addressed. In some cases, it might even only shift the burden.

For example, schools currently are environments that mirrors society and where kids lash out against each other. School are like prisons and the students need a vent. If girls are exempt from “lashing out against” it might fit the sexist view that men’s roles are as protectors or human shields, but it has nothing to do with achieving equality and creating a better world. It will only lead to a world where the boys below the cool kids in the status hierarchy (i.e., the regular kids, the geeks, the retards and the creeps) are gonna get it twice as hard.

And if you feel outraged that I do not want boys to suffer in place of girls, you are only validating my point that negative sexual discrimination to boys/men does exist and is considered “normal”.

Boys and men are seen as disposable — like written, their feelings don’t matter (as much) and if they kill themselves, hey 4:1 (men:women) seems like an acceptable ratio of suicide.

But I think it’s better to address the real problem, not to cater to an individual subgroup.

It’s easy to put the responsibility and danger on the ‘male stranger’ or the faceless male bully. But these boys and men are someone’s sons, brothers, and husbands. Even if you only understand female pain, understand the pain of a mother seeing her son being made responsible for something he is not guilty of. Or the wife seeing her husband being accused of a crime he did not commit.

Perhaps even worse, you socialize women to become victims early on, needing special protection. You might even create a world where successful men wipe the floor with “supported” women each and every time. After all, any man who is successful is this kind of climate has no female equals.

As always, it will only be a handful of men who make it, but still, you are creating what you try to combat.

And think about what you teach girls/women with your survey. And what you teach boys/men as well. Don’t be willing to let the ship sink further as long as you got seating on the life-boats. Realize that the whole ship is sinking and we should fix that. And please, never (try to) exploit science to back up your ideology.

Science is better than that — and so should be your cause.

1 Trackback / Pingback

  1. Better Surveys | ORGANIZING CREATIVITY

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.