• Gender, RPG Publication, and SCIENCE! (Part III)

    by  • October 3, 2012 • Essays • 2 Comments

    If you’re just coming to this series now, I suggest you read about how I got interested in the question of gender and RPG publication rates, and what I found out.

    For those who’ve been following along, you’re about to witness the most important step of the research process.

    Step eight: learn from feedback on your work.

    Between the comments on my post and emails I received, I realized two important things about the study.

    First, I wasn’t clear enough about what the study actually showed. It showed that there was no significant effect of gender on the rate of RPG publication, given this data. That isn’t the same as showing that there is no effect. It just means I didn’t find one!

    This turns out to be important because of the second thing: I neglected to do a power analysis as part of my work. A power analysis tells me whether I would have found a gender difference in publication rates, if there were one. Ordinarily I’d have done one as a matter of course, but I was trying to be quick and dirty – and ended up being a little too hasty.

    So, was my study powerful enough to find out the truth? Well, it turns out the power of my study is only .159. If there is a gender difference, I’ll only detect it 15.9% of the time. That means we shouldn’t be very confident in the answer we got. There very well might be a gender difference in RPG publication rates that this study didn’t pick up!

    This means that my next step isn’t to move on to qualitative analysis, but to do another quantitative analysis that either confirms or refutes this one. Unfortunately, I think the reason the power of the study was so low is that there were so few women who published more than one game. I’m not sure where else to look for data, or if there’s another way to get the power of the study up. If anyone’s got ideas about other sources or approaches I could use, I welcome input in the comments.

    On the other hand, I still feel comfortable saying that the difference in rates of first game publication is both significant and problematic. Watch this space – many of us here at Gaming As Women are thinking about this issue!

    avatar

    About

    Game scholar, game design educator, game designer, and most of all enthusiastic game player!

    http://www.replayable.net

    2 Responses to Gender, RPG Publication, and SCIENCE! (Part III)

    1. avatar
      CrucibleofWords
      October 3, 2012 at 16:58

      Just a thought, is it worth contacting somewhere like DriveThruRPG to see if they can get you a sortable file of some description that can then be .csv’d or .sav’d? It’d also give you access to a lot of indie games that might not have made it onto you original list. It would work out slightly differently to your original study as it would include things like supplements as different entries, but would give you more to work from.

      Also, they make the distinction between free and charged products, so you could examine the gender split among free or charged products, to see if there’s a distinction there.

      Thumb up Thumb down +3
    Comments are closed.