Project Name: Developing Methods for Evaluation of Virtual Worlds and understanding User Experiences

Project Leader

Magy Seif El-Nasr

Project Co-Leader

Regan Mandryk

Researchers

• Cristina Conati (UBC)
• Regan Mandryk (SFU)
• Magy Seif El-Nasr (SFU)
• Bart Simon (Concordia)
• Duane Szafron (Alberta)
• Barry Wellman (Toronto)
• Chris Shaw (SFU)
• Alissa Antle (SFU)
• Alan Kingstone (UBC)

Theme Distribution

• Theme 2 : 40%
• Theme 4 : 30%
• Theme 5 : 30%

Project Description

To develop a good interactive virtual world, a multi-player game or single-player game, developers need to (1) understand the target market and (2) establish a good pipeline and development cycle that integrates testing and iterative design (Fullerton, 2008; Schell, 2008). This is crucial as interactive entertainment products are no longer developed for a small market composed of people who are very similar to designers. For this purpose more time and money is spent to (1) develop better ways of evaluating interactive entertainment products for a target market and (2) understand target markets tastes, behaviors, motivations, perceptions, hobbies, and habits, to mention a few variables. This project is developed to target these two points. Particularly, we aim to develop novel evaluation methods for measuring success of virtual worlds. We also aim to run studies to understand users' activities, motivation, attentional patterns, perceptions and attitudes (Nacke, Ambinder, Canossa, Mandryk, & Stach, 2009). We intend to focus on the evaluation of persistent interactive experiences, such as virtual worlds and massively multiplayer games. Such interactive experiences are developed for more than one user, often thousands of users, and are considered persistent, i.e. the world or game continues to exist even after the user logs off.

While there are many evaluation methods developed by research and industry, these methods and tools have several shortcoming: they are time consuming, need experts to draw conclusive results, and need to be adapted to work for persistent worlds. In this proposal we tackle these problems through building an automatic statistical metrics tracker of user behavior and user trails within a virtual world and a visualization system that allows designers and developers to make easy and fast analysis concerning their designs. We aim to integrate these techniques within a virtual world company, Bardel Entertainment, which currently has a virtual world under development. This will allow us to evaluate our methods and validate industry integration. Another goal of the project is to motivate a research direction towards modeling users by collecting and analyzing user information concerning their habits, perceptions, motivations, attention patterns, attitudes, emotions, and other variables (Thawonmas et al, 2008). Such models will be helpful for entertainment companies to develop better design models that target their audiences.

Excellence of the Research

Development of Highly Qualified Personnel (HQP)

The project will stimulate training for students within interdisciplinary teams as the project is composed of: Design, Interactive Arts and Technology (SIAT), Computer Science (Saskatchewan), Sociology (Toronto and Concordia), Psychology (UBC). The project is also embedded within a virtual world company (which has a long history as an animation company), which will allow students to better develop their own research and tools as well as cultivate good connections and training for future employment.

Networking and Partnerships

Currently, we have confirmed partnership with Bardel Entertainment at a deep level of integration and evaluation of the tools within their virtual worlds, Disney Interactive and IBM at an advisory level. We hope to tap into more of the network's resources in terms of company sites to evaluate the tools once they have been evaluated within Bardel Entertainment.

Knowledge and Technology Exchange and Exploitation

PL has worked within Bardel Entertainment before and thus has a connection and understanding of the personnel within the company. This will facilitate better development, integration and evaluation of the tools developed within this project. As a consequence, we believe the evaluation tools developed can become a technical contribution that will impact other companies.

References to the Literature

Nacke, L., Ambinder, M., Canossa, A., Mandryk, R., & Stach, T. (2009). Game Metrics and. Biometrics: The Future of Player Experience Research. Paper presented at the Future Play.

Thawonmas, R., & Iizuka, K. (2008). Visualization of Online-Game Players Based on Their Action Behaviors. International Journal of Computer Games Technology 2008.

Publications

Presentations

• 2 Minute Madness 2010