• Gender, RPG Publication, and SCIENCE! (Part II)

    by  • September 26, 2012 • Essays • 9 Comments

    In my last post on gender and RPG publication, I left things on a bit of a cliffhanger. Would there be a difference between men and women when it came to publishing games? Would there be a difference with games beyond the first?

    Read on to find out what I learned!

    Step four: statistical analysis.

    Now that I had a frequency list for male and for female authors, it was time to see whether there was a gender difference in the likelihood of publishing more than one game.

    First, I had to see if there was a difference in the total number of games published by men and women. This was a no-brainer – my column of male data went on for pages beyond the female data column – but it’s good to have hard numbers. In my sample, 85.5% of the games had a male author. 6% had a female author, and 8.5% had an author whose gender I couldn’t identify. That was a big gender difference in the total numbers! However, it didn’t tell me about gender differences beyond the first game.

    To find out about that, I looked at whether the average number of games per author was different for men and for women. Women published an average of 1.44 games, while men published an average of 1.58 games. That’s a small difference in favor of men – but was it a meaningful difference? I did a t-test, a statistical test to see if the averages of two groups are effectively the same. It turned out that the difference was not meaningful (p = .41). Men and women, statistically speaking, have the same average number of publications.

    Finally, I double-checked my conclusions with a chi-squared test. Using this test let me see whether the distributions of male and female designers were the same. For example, if all female designers published either one or three games, and all male designers published exactly two games, the averages would be the same but the distribution would look different. The chi-square showed me that the distribution of games across male and female designers was the same, statistically speaking (p = .68).

    Step five: draw conclusions.

    Here’s what I feel comfortable saying I learned.

    Men are the vast majority of published role-playing game authors. 85.5% of my sample had male names and only 6% had female names. In other words, men publish more than ten times as many RPGs as women do.

    Publishing only one RPG is common. For both men and women, more than half the people in the sample had only published one game.

    After the first book, male and female authors continue to publish at similar rates. The same percentage of men and women went on to publish a second book. Of that group, the same percentage of men and women went on to publish a third, and so on and so forth.

    Male authors had a higher maximum. The most-published man had ten books to his name, while the most-published woman had six. (Go Emily!) This is the natural result of comparing two groups with similar advancement rates, but where one group starts with fewer members. The smaller group – in this case, women – runs out of people before the larger one does. You can’t publish half an RPG!1

    Step six: establish caveats.

    One of the most important things I’ve been trained to do, as a researcher, is to interpret data modestly. This means trying to be clear about the limitations of what I’ve found out, rather than going for the easy generalization.

    Here’s the biggest limit on the study. It only applies to people who published a first game. We have no idea whether people who publish a first game are representative of the role-playing community as a whole. More specifically, we don’t know whether there’s something unusual about the women who actually get a first game published. I suspect there is, both because of conversations I’ve had and because more than 6% of all role-players are women. That’s something I’d want to find out!

    In other words, this study might not actually show that men and women publish games at the same rate. It might show that people publish at the same rate if they exhibit certain personality traits, or if they have certain kinds of social connections, or if they behave in certain ways. If those factors aren’t equally distributed across men and women, then we’re back to seeing a gender difference after all – just in a more subtle way.

    The study also only applies to people who publish under identifiably male or female names. Because this study was quick and dirty, I just left out everyone who publishes under their initials. I suspect women are more likely than men to hide their gender when publishing. Because people who were unidentifiable published more games than identifiable women (8.5% versus 6% of games in the sample), including them could change the outcome significantly.

    Step seven: identify next steps.

    My next steps fall into three categories: making the study better, answering related questions, and working out the implications for our community.

    To make the study better in the short term, I’d do two things. First, I’d reach out to the community to help me identify the gender of my 115 “unknown” authors, so I can include them in the analysis. Second, I’d try to clean up the data itself, like sorting out what to do about multiple editions of the same game.

    As far as other questions go – there are lots of good ones! Here are a few that I could answer with the data I have, more or less:

    • Do we see different patterns for indie and for mainstream games?2
    • Do people who publish under their initials only show different patterns from identifiably male or female authors?
    • Do the percentages of female RPG authors change over time?
    • Are there differences in how long it takes men and women to put out their second game?
    • Do men and women have different patterns of co-authorship?

    I’d also want to try to get some data about how people publish their first game. That would let me find out whether first-time RPG authors are special in some way. It would also provide enough additional data to let me publish this work in a more academic forum. Of course, I’d need a fancy title! How does “Print Runs & Proofsheets: Gender and the Role-Playing Game Publication Process” sound?

    Finally, there’s the question of what we do based on this new data. What it tells me is that if we want to change the numbers of role-playing games written by women, we work on supporting women in publishing their first game. Once a woman publishes, she’s as likely to continue as a man is. I might revise this conclusion once I get more data – but for now, I’m comfortable saying that the first game is where the hurdle is.

    Speaking personally, I’m going to put my money where my mouth is. If you’re a woman and you’re on the fence about publishing your first game, get in touch. We’ll see if we can make something magical happen.

    1. The evidence of certain games I’ve played to the contrary.
    2. Yes, this was the original question. Oops!
    avatar

    About

    Game scholar, game design educator, game designer, and most of all enthusiastic game player!

    http://www.replayable.net

    9 Responses to Gender, RPG Publication, and SCIENCE! (Part II)

    1. avatar
      Finaira
      September 26, 2012 at 18:06

      Out of curiosity (you’re the one who brought up statistics! I blame you!), what was your statistical power? I’m curious with a sample size of that size how big of a difference would have been required to detect significance.

      ((In other news, at least I’m not suggesting you model the results…))

      Thumb up Thumb down +1
      • avatar
        CrucibleofWords
        September 27, 2012 at 17:34

        Seconded! I’d be fascinated to do some numbercrunching on this myself to try and come up with something, if you’re willing to share the .csv.

        Also, as I’ve never really been sure, what’s the difference between “mainstream” and “indie” games? I’m guessing the latter are self-published, but I don’t know for certain.

        Without looking at the exact proportion of first- and second-time publishing, I can’t say for sure whether you’re conclusion is right about the first game being the first hurdle (are the proportions of each gender going beyond one game identical to the nearest %? If not, what’s your “error margin” there?), but my gut would say the next step would be qualitative analysis (yay for focus groups and IPA interviews!) of what each author’s experience was. Was the way Emily approached publication different from the way that single-time authors dealt with it? And potentially most important, we need the “graveyard”, the silent majority of both women and men who don’t get published, and see how their experiences differ. All of this would require a lot of work, but would be worthwhile.

        Thumb up Thumb down 0
        • avatar
          September 28, 2012 at 15:50

          Finaira – I’m writing a third post about power, so you’ll get an answer next week. :)

          Crucible – let me know where I can send the data and you’re welcome to mess with it! As for the proportions of each gender going to the next stage, that’s what the chi-square shows.

        • avatar
          September 28, 2012 at 16:08

          Right! Also! Indie games! Turns out that’s hard to operationalize, which is why I couldn’t actually get the answer to that question.

    2. avatar
      September 26, 2012 at 19:19

      I have every intention to screw with your statistic. Both by writing and writing and writing, and by helping as many women as possible jump that first hurdle. (More on that later.)

      Thumb up Thumb down +3
    3. avatar
      denzi
      September 27, 2012 at 01:20

      As a scientist myself, I am really happy to see hard numbers and statistical analysis, and a conservative discussion of your results. I also think you did a really lovely job creating a rigorous study and then translating it into easy-to-understand language without losing any meaning or rigor. Brava!

      Thumb up Thumb down +1
    4. avatar
      Mo
      September 27, 2012 at 16:49

      Excellent job, Jess! As a woman who has interest in design and publication, but feel effectively disconnected from it, it’s good to see some science going into this… and maybe towards making a place where others don’t end up feeling like I do.

      Thumb up Thumb down +1
    5. Pingback: Gender, RPG Publication, and SCIENCE! (Part III) | Gaming As WomenGaming As Women

    Comments are closed.