Crusader Kings' Tutorial Effectiveness: Usability & Learnability Testing

Background and Goals

Investigate first-time user experience on effectiveness of video game tutorial & whether users determined they would continue to play on their own time. I wanted to understand the video game tutorials ease of use and learnability from the users' perspective.

Methods

To achieve the goal, I adopted Qualitative methodologies. In-person usability testing was conducted at Starbucks for willing participants; a free drink of their choice was given as incentive for their participation. Target audience were PC gamers, and thus, a pre-screening questionnaire was utilized to determine eligibility. Pre-screening questions include: 

• "Are you a PC gamer?"

• "Have you ever played Crusader Kings III before?"

• "Do you have an hour to spare for a research study?"

Upon determining an eligible participant, they were asked to spend 30 minutes playing the video games' tutorial. After the game tutorial, they were given three in-game activities to complete (all of which were included in the tutorial). They were also encouraged to think aloud while interacting with the game as they went through their predetermined tasks. 

Crucial Insights

When asked to complete certain tasks, participants displayed a lack of knowledge with game's UI. They also appeared frustrated while searching for tasks, thus highlighting the need for a restructuring of the current tutorial system. 

​

Users were often visually frustrated and commented on their personal confusion with the game's UI [Ex: two different action buttons having the same name.] Average time take to complete a single task was 5 minutes, while the more complex activity took an average of 8 minutes to complete. 

Research Impact

• Provided insight into the learning curve for this video game, game developers must focus on closing the gap on game's learning curve.

• Developer's should focus on user perspectives when creating tutorials for new game.