Failing in Games at Aarhus University

 Last month I was invited to present at the Aarhus University's Interacting Minds mini-conference on "Failing and Confusion in Games and Gaming" along with Jesper Juul, Dennis Ramirez and Charlotte Janasson. Thanks to Andreas Lieberoth for the invite and organising a seriously interesting day :-)

Mine was the final talk of the day, where I presented some of my PhD research on Investigating Game-play: Are Breakdowns in Action and Understanding Detrimental to Involvement? (see pic below). You can find my slides here. I'm waiting to hear back about a journal paper we put together on the findings but in the meantime you can check out my DiGRA paper on why I did not find physiological data to be useful for identifying breakdowns and breakthroughs in game-play. My main argument was that action and understanding breakdowns will contribute to involvement when the player feels responsible for overcoming them but that they will decrease involvement if they take too long to overcome or have major consequences e.g. a loss of progress. There was some interesting discussion in the Q&A afterwards around defining involvement, whether "positive engagement" is a helpful term, the importance of triangulation and how we can avoid players getting into "negative cycles" where breakdowns don't lead to breakthroughs. While I think my work can help explain when certain breakdowns are likely to disrupt involvement, I think there is still plenty of scope to consider how and why some players are able to avoid these negative cycles and others don't.

(Thanks to Andreas Lieberoth for the twitpic)

In terms of the other presentations, I was glad to hear more about some of Andreas' initial work on Quantum Moves (a citizen science game) where they investigated player motivations e.g. in terms of fear of failing i.e. trying to avoid looking bad or achieving mastery challenges. While they chose a different focus, there is definitely some overlap with some work I presented at CHI this year in relation to the Citizen Cyberlab project, looking at why people chose to play citizen science games. I'm definitely looking forward to Andreas visiting next term so we can get into some more discussion about our research.

Jesper then kicked off the main talks by discussing failure in games (he's also written a book about the topic called the Art of Failure). Amusingly, he got different people in the audience to try out Super Hexagon and China Miner - I think I lasted about 10 seconds in the latter! Juul argued that while failure can be a source of learning, it's still an unpleasant experience and pointed out that there is a bit of a paradox going on here - normally we want to succeed but when we play games we seem to be seeking out experiences where we will fail (at least part of the time). I wonder though about how you define failure? I don't think all breakdowns are necessarily failures, often they are part of the challenge, or quickly overcome, whereas the word failure seems to indicate something more serious. What was really interesting was how when he pointed out how games can promise to repair some sort of inadequacy in us, but it is an inadequacy the game actually created in the first place! I think I'm going to have to read his book to get more to grips with the various paradoxes and philosophical arguments outlined in the talk but Jesper also suggested failure in games differs from real life as games offer a certain amount of plausible deniability e.g. "It's just a game", "It wasn't fair", or even "I wasn't trying that hard in the first place". I have thought about "its not fair" comments before - I see them as an indication that involvement has been disrupted, since the player sees the game rather than their own actions as being at fault - but either way I think they indicate a serious breakdown has occurred as player are essentially distancing themselves from the game.

Dennis' talk on his PhD research followed similar lines but he focused a little more of what failure means for learning and educational games. He pointed out that only 20% of players actually reached the end of Hitman Absolution (and apparently only 10% of players will see the end of any game) and argued that it's important to consider the metric being used to assess success within a game. Dennis also discussed various approaches to using games and assessing them - from chocolate covered broccoli e.g. Math Blaster to thinking about model based assessment e.g. Schaffer's epistemic frames. I particularly liked how he pointed out that we can't always infer competence from completion and when he discussed more recent approaches to evaluation that relate to "big data" (though also stressed the importance of talking to players too). For instance, he talked about some work going on at Wisconsin-Madison that was looking at heat maps in terms of how different players move through the game. The fact that progress doesn't always guarantee learning is something I've considered in my PhD research i.e. you can achieve action breakthroughs without understanding (though chances are these will be less satisfying) but it was good to hear more about what that means in terms of assessing learning from an educational point of view.

Charlotte provided a different perspective with her talk on Learning from errors in education. While not focusing on digital games, she provided an interesting account of failure in real-life settings, in this case a vocational school. She made the argument that while not exactly a game, school isn't quite real-life either and vocational schools offer a sort of real-life work game - where failure is considered part of the learning process. Charlotte used an example of the students learning how to clean, cook and prepare flounder (apparently very tricky!). She noted that the instructors treat the school as a practice space where errors are ok, but not if they are made as a result of knowledge you should have acquired already. Further, it seems that developing expertise is about becoming more skilled at paying attention and knowing what to pay attention too. Her talk got me thinking about my work on CHI+MED and errors within a healthcare environment, where I've been interested in how nurses are trained to use infusion devices. But, if errors are an unavoidable part of work practice and learning from them can help you become an expert, then how on earth do you go about supporting this process in an environment where the consequences of errors could literally be life or death?! I guess using a pump isn't normally that complicated but I do wonder about what sort of knowledge nurses have and how they develop expertise in this context.

Overall it was a really good day and it got me thinking a lot about games, failure and errors in the workplace. It was a great opportunity to talk to attendees at the event, catch up with Andreas and Yishay Mor, and enjoy lots of discussion afterwards when we went out for a lovely meal in Aarhus :-)

CHI 2013: Paris (Part I) - MOMA, games and learning, game players and the Games SIG

Last week I went to CHI in Paris - it's been a while since I've been to a major academic conference and I seem to have gotten out of the habit of blogging so I thought I would use this as an excuse to get back into it :-) Plus there was a lot of game sessions that have got me thinking.

It all started last Sunday with the MediCHI workshop. This was a good opportunity to talk to about the work I'm doing on CHI+MED, with respect to medical device safety, and to meet others in the field. The main conference started on Monday with a keynote from Paola Antonelli from MOMA. She gave us an overview of lots of intriguing design projects that MOMA has exhibited and while no specific HCI challenges were made explicit during the talk, I was reminded about how technology, including games, can make people think. Particularly interesting examples include PIG 05049 (Christien Meindertsma) and the Menstruation Machine (Hiromi Ozaki/Sputniko!). She also mentioned their recent games collection - extra points for the inclusion of Passage :-)

In terms of the game-related talks, Erik Harpstead discussed an educational game they had developed (a single player physics game called RumbleBlocks - see below) and how they used metrics to assess learning as part of the ENGAGE project. A toolkit was presented for logging game events and that allows for a replay of game-play so player behaviour can be analysed further. This toolkit seems like it could be really useful but my main question was whether collecting this type of logging data can actually account for situations where players progress but without gaining any real understanding of the principles behind what they are doing. This concern was partially addressed during the talk when the replay analysis indicated that the gameplay mechanics actually contravened one of the learning goals (where students were not lowering the centre of their structures, even though they were building one with wider bases and that were more symmetrical). The misalignment between content and gameplay was seen to potentially explain why there was not a difference between pre and post-test regarding centre of mass and also suggested that the game needs to be redesigned to remedy this issue.

Derek Lomas' talk on optimising learning and motivation in educational games through using crowdsourcing techniques also got my attention. What was particularly interesting about this study was the huge amount of data collected (one study has 10,000 participants the other 70,000 - all who played the online math game Battleship Numberline) and the questioning of the inverted-U hypothesis regarding challenge and engagement. Basically, flow theory suggests that if something is too easy, boredom will occur and if it is too hard, you'll get frustration - so a moderate amount of challenge would be the most engaging. However, the findings from Derek's work actually suggest that people find spent the most time playing when the challenge level was lowest thus indicating that  easier challenges are more engaging . Further, the studies indicated there is a trade-off between engagement and learning i.e. you can't have both... I'm going to have to read the paper for more details but there are several points here that I'd like to consider further. First, I'm questioning whether the length of time spent playing is a good measure of engagement (especially when children might be playing these games during school time - who is controlling the length of play if that is the case?). The terms engagement, motivation and enjoyment were all used interchangeably but I'm not sure they can all be reduced to amount of time spent playing. Surely I can enjoy something I play for less time more than something I might play for longer (e.g. if my motivation was to kill time)? Secondly, I want to look at how well integrated the game mechanics of Battleship Numberline are with the learning content - mainly because I don't like the idea that there needs to be a trade-off between engagement and learning! Further, given Jake Habgood's work on the importance of integrating game mechanics, flow and learning content I don't think there has to be. Finally, the authors also suggested that novelty might be more important than challenge in relation to engagement. This was particularly intriguing as I don't think it's something that has been explored in the literature on games and learning and I'm guessing there might be quite a lot to it.

Within the same session, Stephen Foster talked about designing diverse and sustainable educational games that support competition and meta-cognition. Inspired by the way Chess and Starcarft II players relfect and review their game-play, Stephen presented a game called CompetitiveSourcery based on the pre-existing CodeSpells platoform. The game requires players to compete by designing "spells" in java and using them against each other. Three users were observed over two months as they prepared as a team for a tournament - this included playing the game but also discussing strategies, bebugging each other's code and updating a team wiki. In general, this was a good example of tapping into both micro and macro involvement for the purposes of learning but I was surprised not to see any mention of Gee's discussion player affinity groups that exist around games (though Gee is mentioned in the paper). Plus, the idea that teachers should consider meta-level activities isn't entirely new (see Paul Pivec's BECTA report for the importance of the meta-game) while encourgaing discussion through having a tournament has been done before (e.g. research on Racing Academy). Also, while Stephen makes claims about the sustainability of this approach, there is always going to be an issue concerning whether all players will actually engage in the meta-level activities to the same extent. I'm not sure how you address that though...

There were several other game related sessions including Max Birk talking about the relationship between controller type and personality and Jeff Huang talking about patterns of game-play and skill in Halo. For both these talks I wanted a bit more detail on the methods so I'm going to have to add them to the pile of post-CHI papers to follow-up. In relation to the former, I was a little confused about the relationship between my "real" self, my "ideal" self and my "game" self as I'm not sure any of these can be static constructs but there may be some interesting differences to explore here (do standard controllers really make gamers more neurotic?). In relation to the latter, an awful lot of logging data was collected but I was a little disappointed that "patterns of gameplay" was more about how long people play for and how often than it was about gameplay strategies (but that's only because I'm more interested in player strategies!). Other highlights included Nicholas Graham discussing a tabletop game where one person plays the game and the other orchestrates the experience in real time i.e. builds levels and obstacles. This reminded me a little of Sleep is Death but Tabula Rasa seemed a bit more light-hearted in it's approach to foster open-ended creativity. Tamara Peyton (see below) then spoke about the alternate reality game I Love Bees and showed how leadership emerged from team-play. Interestingly, the players spontaneously used military terms and take on different roles within the team which she classified as General, Lieutenant and Private. I particularly liked that she emphasised that disjuncture can be as important as flow - essentially we should also be thinking about what it means to fail and how failing isn't necessarily a negative experience.

I also attended the SIG on Games and Entertainment and was pleased to see that there really is quite an active games community at CHI. Katherine Isbister and Regina Bernhaupt led the session but handed over the reigns of the SIG to Magy Seif-El-Nasr and Heather Desurvive. The topics that came up ranged from needing to foster links between industry and academia, introducing further games courses at next year's conference, and discussing other venues for games research. It was clear that while some people were interested in the user experience side and methods for assessing game-play others where interested in using games as research tools e.g. for the purposes of collecting data. Regarding the latter, there was a suggestion that there might be a workshop or course next year with a focus on how you might evaluate this kind of large-scale data but I think that will depend on whether someone volunteers to run it! I enjoyed the session overall and found a good way to see the range of game-related interests across the CHI community.

Ok, I think that's enough for today. There is still plenty more to write but I'm going to have to leave it for another post! For now I'm going to leave you with a pic of Charlene Jennett hugging a bear to make one jump on screen :-)

School of gaming starts in New York

This Guardian article on a school founded on the principles of game-based learning claims this is also "proof that games are educational". I think it's going to take more than just the school opening to support this claim though.

They are starting with just the sixth grade right now but you can check out the school website to find out more.

I'm intrigued but not sure I'm a fan of the school being called "Quest to learn" - too many images of warriors, Elves and magic coming to mind...

JURE and EARLI 2009

I was going to write a single blog post titled "Conference season" since that is what the last two weeks have felt like, but I think it might get a stupidly long so I've decided to do two separate ones. I know I've been slack with the posting, especially about the games I've been playing, but now that I'm actually going to be in one place for more than a few days at a time I should be getting back into more regular posts.

So, first off, was EARLI (European Association for Research on Learning and Instruction) in Amsterdam where I presented a roundtable discussion at the JURE (Junior researchers of EARLI) preconference. I'm glad the other presenter at the session was doing games related research (Jantina Huizenga who is working on the Game Atelier project) and though it may not have been the main area of everyone who attended, it was useful to get some different perspectives on my work. I also somehow managed to win an award for "Best roundtable" (thanks to the Groningen votes!) so that was pretty cool :) The main things the discussion got me thinking about was in terms of how to keep interactions as natural as possible during observation sessions and also about whether there is any way I can somehow analyse the complexity of the games being played in order to distinguish between them? I think I have this theory that games like Mario Kart or Wii Fit are somehow simpler than something like Fable II or Spore but I don't really have any real way of distinguishing between them... as yet.

Now EARLI is a massive conference and there were loads of talks to do with areas of education I really don't know much about (e.g. reading and writing). There was some games related stuff, and a lot of talk about motivation, but pretty much all of it was to do with formal education. Though there was a poster by Bjorn Sjoblom on co-located gaming about studying the discourses of players playing a MMOG within an Internet cafe. I liked the fact he was focusing on the co-located aspects of play and that he was using an observational approach.

There wasn't a whole lot of talk about informal learning either and when there was, it seemed to be more about trying to bridge the informal learning experiences at museum or science centres with what goes on at schools. At JURE though, I did come across a poster presented by Marjolein van Herten about informal learning within book discussion groups and had a bit of chat with the author about how difficult it actually is to identify informal learning... I also went to a JURE talk on the computer-based educational games by Claudia Schrader who compared a high-immersive game and a low-immersive application (control) to find that the control group did a lot better on subsequent cognitive tests. Without seeing the game and application used, it's difficult to make any judgements here, but I have a suspicion that learning gains from games take longer to show up (or at least require more than a single session) and may also be retained for longer (which would need a delayed post-test). It's also possible that we are back to considering whether there is a divide between being motivated to play the game, and being motivated to learn the content.

Which brings me on to Shaaron Ainsworth's talk on intrinsic integration within serious games. She presented some work carried out by one of previous PhD students Jake Habgood about how to integrate learning material within the game being designed. The approach adopted was an experimental one where manipulations of the same game (Zombie Division see below) indicated that it is not the notion of intrinsic fantasy that is important but how you integrate the learning within the game's core mechanics. I take this to mean that there shouldn't really be a divide between learning how to play and learning what you want players to learn.



Though I know Shaaron from when I was an undergraduate at Nottingham and it's work I've come across before, it was good to be reminded of it and it made me think about what it I want to get out of my own research. I've chosen not to focus on educational games, but on commercial ones, because I don't think we have a good enough understanding of why people play them, and how this links to the amount of effort they are willing to put in to learn (and master) the game. The discussant at the session also raised some interesting point about how it's time to think about specifying what kinds of gameplay work with what kinds of learning, which seems pretty close to what I want to do. What I really want is to come up with a way to compare and contrast different games in terms of motivation, engagement and informal learning. This will hopefully have implications for how to use and design games within education, or even just mean we are able to assess the informal learning potential of different games.

Even attending talks that weren't directly related to my research area got me thinking about different things. For one, expectancy-value models of motivation kept coming up at different presentations, so I need to look more into that. Errors were also mentioned as a potential source of learning which I think could be relevant to learning during game-play. There was also some discussion of deep and surface approaches to learning, which made we whether you could classify learning within games in the same way - maybe players engaged in shallow learning with Mario Kart and deep learning with Spore? How could I assess this?

One presentation I did find particularly useful (especially because we got to talk about it afterwards) was by Ulises Xolocotzin-Eligio, a PhD student at the LSRI in Nottingham, who is examining the role of perceived emotions within computer supported collaborative learning. He has also used games (e.g. Astroversity) to explore these concepts during co-located play and has kindly forwarded me some of his work to look at in more detail. Looking over my notes for the session, I have written down "think I need to observe people more than once..." as the talk made me realise I probably do want to do more than a one-shot observation session in order to see how the processes of learning and engagement change over time.

Hmm, can see how things are starting to get more complicated in terms of what I want to do, while I haven't even begun to reflect on the HCI conference... But right now I think I need to go away and write up some sort of reading list based on the conference, before I forget it all!

European Conference of Game Based Learning

I was in Barcelona last week for ECGBL. This was my first proper conference and though I wasn't presenting it was pretty cool to be able to get a feel for what other people are doing in the area. In general, I enjoyed the whole thing as I got to hear from different perspectives and meet some interesting people along the way. But it was almost as people were a little too nice - there was little picking apart of ideas or applications, and I didn't hear any particularly difficult questions at the talks I went to. I'm not an advocate of being difficult just for the sake of it, but in line with Ben Sawyer's keynote where he talked about the need for constructive criticism and how the discipline needs to move on from arguments about fun and learning, it was like people were a bit too eager to pat each other on the back. I think I was expecting a little more debate but perhaps I wouldn't feel the same if I had been one of the ones presenting.

That doesn't mean there wasn't loads of interesting stuff that's given me plenty to think about. I was really pleased I got to hear Simon Egenfeldt-Nielsen speak - he gave a really good overview of the research area and gave an example of one the serious games he's worked on as part of the Serious Games Interactive (Global Conflict: Latin America). One of the points he brought up was the fact that increasing the learning focus in serious games seems to decrease the motivational aspects. The other thing he made me think about was how the term "serious game" seems to be used as a synonym for "simulation". For instance, he said something along the lines of games being the "future for training" since they provide "immersive, realistic and meaningful environments". It's just that there seems to be a difference between playing a game where you are essentially training as a journalist and one where you are taking on the role of a criminal in Liberty City. I'm just not sure how to verbalise that difference or how important a distinction it is to make with respect to education. But I have a feeling the more realistic something is, the closer it is to a real life job, the less engaging it's going to be when presented as a game.

Other highlights include a review of theoretical models of player enjoyment by Liz Boyle (including arousal theory and Apter's theory of reversal which I need to look into) which made me think about the difference between sustaining motivation in the long and short term; Nathalie Charlier and Maria Saridaki talking about how teachers can be taught to use a digital-game based learning apporach; Andy Smith's presentation of how we might be able to develop the notion of a "respectful mind" through something like cultural exchange programs in foreign MMPORGs with host families; Gearoid O Suilleabhain talking about how we should consider that there are different types of transfer and we can improve on the ways of testing for it; Nicola Whitton discussing the ARGOSI project to develop an alternative form of induction through ARGs; and Jen Jensen making a distinction between imitation and simulation as a result of the different gaming experiences that new game controllers (e.g. guitar shaped peripherals) seem to provide.

But perhaps the talk that I remember the most in terms of making me think about things differently was by Natasha Boskic, a Serbian now working at the University of British Columbia in Canada. She was working with a team on examining World Without Oil (WWO; an ARG that ran for 32 days in 2007 which encouraged players to consider how they would deal with a future oil crises) to uncover noteworthy themes and issues from the artefacts produced by the players. However, once she began, people's comments and accounts from the game began to remind her of what she went through whilst living in war-torn Serbia in the early 1990s. In the conference paper, she describes dealing with the game as "unbearably traumatic" since "to me, WWO was neither a game or an alternate reality. It was my reality" (pg. 46; Boskic et al., 2008). The tag line "Play it before you live it" didn't exactly help matters and nor did the comments from players about how much "fun" they had playing it. Her experience raised some interesting questions about whether you can teach empathy and understanding about real world crises through play and whether it's even ethical to try and create games out of other peoples' disasters in the first place? Not that the presenter didn't recognise that there weren't good things about WWO (see Rusnak et. al., 2008; from the same conference) but the point was that designers and educators who want to use a game-based learning approach need to be mindful when doing so and sensitive to their potential audience.

So yes, plenty of things to think about while trying to figure out what I want to focus on in terms of my own research.

Writing in Games

I was in London last week for NMK's Writing for Games Event and while the event was interesting in and of itself, it's also got me thinking about the relationship between narrative and game play. The panel was pretty cool, consisting of Katie Ellwood, (who worked on Getaway franchise), Steve Ince (who worked on the BAFTA nominated Broken Sword: The Sleeping Dragon), Adrian Hon and Naomi Alderman (who were both involved in Perplex City, an ARG, with Adrian as designer and producer and Naomi as a writer) and it was fascinating to hear about their different experiences. It did seem like the event was aimed more at people who were looking to break into the games industry as writers, such as Justin who I sat next too and had quite an interesting chat with afterwards. While becoming a games writer is not something I've ever really considered, I do feel I came away with a better idea about the part writing plays in the game design process.

There were a couple of things that struck me. One, was that most of the panel seemed to fall into the roles they had as a result of meeting the right person at the right time. It seems there is no direct or obvious career path, which is possibly why many writers have a background in fiction writing or TV and film. Two, was that the design process seems to very much depend on the team you find yourself in and that there can be a fair bit of tension between game designers and game writers. It was suggested that if you can get the design team and writing team to agree you're probably on to a very good thing but it also sounded like this could be quite hard to do in practice... Especially when the writing is usually seen as secondary to the game.

This seems kind of similar to the idea that narrative can get in the way of the game play. I've been playing GTA4 a bit recently, which I think I enjoy less than I should because I get bored at all the driving around you have to do (and I guess because I'm just not very good good at all that driving). But as usual, I tended to skip over the bits of the story. Maybe I'm just impatient, but I wanted to do something, not have to listen to Roman complain about his debts or go on about his girlfriend Mallorie. What's good is that you can skip these bits, and still know what you're supposed to be doing. However, it just might be possible that I'm just not taking them seriously enough. I mean I wanted to see what the game was like, but it's huge and I don't want to get sucked into playing it so maybe I avoid the story to avoid any further involvement? There are still plenty of games though were you feel like the story is getting in the way of your game play e.g. Trauma Centre: Second Opinion's endless storyboarding and the rather long intro scenes of Prince of Persia: Rival Swords for instance.

Ideally, there should be no conflict between the game story and the game play. One should inform the other, and any cut scenes you have to watch should be integral to the game itself, and to be fair, I think most games do aim for this. Arguably, the writing gives the game a context that makes playing it a richer experience. But games aren't movies intended for a passive audience - they are supposed to be interactive and maybe that's why gamers seem to resent extended cut scenes. There have been rumours that the latest Metal Gear Solid has 90 minute cut scenes in it, and while this is an exaggeration, without the passion for and knowledge of the series that some players have, I find it hard to imagine I could interpret this sort of thing as anything other than an incredibly lengthy intrusion into my game playing.

Maybe it's because writers in the games industry are using ideas from areas such as film and theatre and these just don't work as well when the goal is an interactive experience. I would think that this could be where a lot of educational games fail - I mean it would seem that the most likely place to introduce learning content would be the include it in the narrative, but if gamers skip over these bits, it's unlikely to be a very effective strategy. I doubt games will ever be able to deliver much content but they have other strengths. After reading books by people like James Paul Gee and Ian Bogost, I'm starting to think that what games do well is to provide you with a set of experiences, and if these are meaningful experiences, then maybe that's where the potential for learning through games lies. I guess the next thing to ask is what exactly is a meaningful experience and how can (or do) digital games provide them?