My day job is in game research, so I’m always interested in how we can get better data about games and the experience of play. One of the reasons I contribute to this blog is because it inspires my work! I thought I’d share some of the ideas that have come up because of the conversations that have been happening here.
But first, a caveat.
Research design is a complex process, and I can only give you a glimpse at it here. These projects are in the kicking-around stage, and are by no means ready to go. Until they’re moved to “active research” status, I probably won’t get a chance to do the deep thinking and design needed to make them work for real.
The Impact of Game Art
We spend a lot of time talking about how women are visually represented in games. But what if we could get some measurable data behind that?
We know that gender affects people’s perceptions of competence, and that images are enough to evoke those gender stereotypes. For example, there’s a classic study where researchers had people evaluate resumes; the only difference was whether a male or female picture was attached. Resumes with a male picture were more likely to be “hired” by the subjects – and when the experiment was replicated by sending identical resumes to real-world companies, the same results occurred. There’s also a body of literature on the interaction between attractiveness and success. Unsurprisingly, attractive men are considered to be smart and powerful, while attractive women are considered dumber than their performance suggests. I’d want to review the research on this topic, as I’m not as familiar with it as I am with the competence literature.
So what would I actually do? Step one might be to replicate the resume study, but with character sheets. Create identical character sheets but attach different pictures. I’d have four pictures for each character sheet – male non-sexualized, male sexualized, female non-sexualized, and female sexualized. That would let me evaluate the effect of gender and sexualization separately, and see how they interact.
I’d let people evaluate the characters in a variety of ways. I’d probably want to get at player perceptions of character competence. One way to do that would be to give players a list of characteristics; they would circle the ones they felt were true of that character. Using the right list would also let me see to what extent players considered different characters to have “feminine” and “masculine” traits. Then I’d want to give players a handful of characters (with randomized pictures) and see which one they picked to play in an adventure. In addition to all this quantitative data, I’d record and analyze how players talked about the characters in a more free-form way, so I could get a qualitative look into their thought processes.
In doing this, I’d want to make sure I got at the things that making gaming different. For this study, I think that means two things. First, I’d want to look at the impact of character classes as well as images. We think about characters as ways of doing things in fictional worlds; are certain ways of doing considered more masculine or feminine? I’d expect to see an interaction between character archetype, gender, and sexualization. I’d expect to see gendered differences between, say, tanks and healers. However, I’d also look for differences within genders based on sexualization and class. I’d expect to see sexualized female clerics treated differently from sexualized female mages, just for example.
The second thing I’d look at is a lot less clear. It has something to do with the fact that characters are people you become. I don’t know what exactly that would mean for the study, but I think it’s important to address.
If this study turned up the differences I expect – ones that are consistent with the rest of the literature – it could be a powerful argument for making change in game art, one that can’t just be hand-waved away.
Air Time and Play Impact
One of the most powerful books I’ve ever read is Failing at Fairness, which looks at how girls get less attention in school. The book shares data about how teachers (both male and female) call on girls less often, rebuke them for seeking attention, and give them less useful feedback when they make mistakes.
“Well,” I said to myself, “the book was written a while ago, and I’m pretty committed to feminism. I bet I’m not doing this in my own classes.” Oof! Let’s just say I was embarrassingly wrong, and I’ve had to do a lot of work on my teaching style to make sure my female students get the attention they deserve. Since then, I’ve been particularly interested in this problem.
Of course, this doesn’t just happen in schools. Women get less than their fair share of attention in professional settings, and their contributions are less likely to be taken seriously and acted on. I’d want to find out whether the same applies in games.
The first challenge would be figuring out why it’s worth extending this research to game settings. Why can’t we just say that the research on, say, women’s participation in meetings applies? Yes, replicating research has value, but I’d want to make a case that games are somehow different. I’d start with the argument that characters serve as a filtering device for player behavior. Because characters can differ from their players, both in gender and in social role, we can see different things by asking this question in a role-playing context. I probably wouldn’t ask anyone to cross-play in my first study so I could get a baseline, but it’s one way I’d think about the larger trajectory of the work.
Second, I’d have to figure out how to describe and measure the notion of “play impact.” The Myers figured out a way to evaluate teacher feedback; was the student being given the answer, or pushed to work it out for themselves? I’d have to come up with some way to categorize and describe the different ways in which player input is treated by the group during game. I suspect it goes beyond the usual finding of “a woman’s ideas aren’t accepted by the group until they’re repeated by a man,” but I don’t even have the categories worked out yet.
The biggest challenge would be to decide whether to lean toward naturalistic observation or toward the lab. In the lab, I could control things like the size of the group and what game they were playing, but I’d lose a certain amount of realism. I’d probably go for the lab, though, just so that I could see what happens when you vary the number of women in the group.
I’d also want to look at male-GM, female-GM, and no-GM games – though of course I’d run into problems trying to argue that GM-ed and GM-less games are at all comparable for research purposes.
Finally, I’d want to examine the effect of mechanics. To what extent are women able to use game rules and character design to insist on being heard? Or do women have to play a social game in order to have their characters’ actions matter, even if the rules say they should?
If I were to do this study, I’d expect to find women getting less than their fair share of air time, based on how many women there were in the group. I’d also expect to find female players having less of an impact on the game fiction than their male counterparts, no matter what their character sheet said. Finally, I’d expect to see something interesting around how game rules are used to enforce and undermine this gender play – but I don’t know what!
Even though there’s something cool to find out, though, this study might be too depressing for me to do.
I recently came across a fantastic paper on “experience taking,” or how people can be changed by imaginatively identifying with a character. The idea is that you are changed by the experiences you have as that character, even if they’re fictional. The study was done with written stories, but I think it’s obviously applicable to role-playing games.
I’m not yet sure what I’d want to do with this research, or whether I’d even want to connect it explicitly to gender. One thing I’d want to pursue, though, is something that came up in the original study. It looked at how heterosexual men experienced the world of a homosexual male character. It turns out that if they knew the character was gay at the beginning of the story, they distanced themselves from the experience, but if they found out late in the story, after they’d already had a chance to experience that character’s world, they changed their minds.
To me, that connects to some rather complicated ideas about character creation and representation in role-playing games. What decisions do we make during character creation, and what do we leave until later? How often do we find out things about our characters we didn’t know? What kinds of things do those tend to be? How do we value the instrumental aspects of our characters as opposed to the experiential aspects? How similar is the experience-taking of role-playing games to the experience-taking of fiction in the first place? And how do all of these things vary across games?
If I were going to do work on this, I’d start with an explicitly exploratory study. I’d just go talk to people and watch them play, and also talk to readers. I’d listen carefully for similarities and differences so that I could even begin to frame a role-playing-related version of this theory. I’d also go talk to the authors of this paper, who I happen to know are working on a computer game version of the study, so we could make sure our research worked together to say something interesting and new.
What I’d Do Next
If I were going to pursue any of these studies, I’d hit the books. For each of them, I’ve already identified one or more research papers I’d want to read. Then I’d follow the citation trail to find out what other people are saying about the topic. I’d also be looking at the way people have designed experiments or observational studies, so I can learn from the best practices in working within a specific research field.
It’s helpful for me, though, to do this reading with my own questions in mind. I like to start my research with an outline in my head, like the ones I’ve laid out here. Since I can’t read every paper out there, it helps me figure out which ones I should be spending my time on. It also helps me figure out the ways in which my work is going to be different from what’s already been done.
I’d also start thinking, right from the very beginning, how I would make my work accessible to different audiences. What would I need to do to be persuasive to academics? In which discipline(s)? Am I planning to make this research usable by any other groups? How could I translate and communicate the work for, say, game designers? Role-players? Feminist activists?
Finally, I’d keep my eyes out for collaborators, whether from academia or from the role-playing world. Research is always more fun with friends!
I’m sure more questions will come up as I continue to participate in this blog, and I look forward to kicking them around with all of you.