Mindgame

The Creation & Validation of 4 IQ Game-Based Assessments

White levitating spheres
White levitating spheres
White levitating spheres

CLIENT

Mindgame

SERVICES

UX Research, Data Collection, Data Analysis, Interviews, Proposal Writing

DATE & YEAR

Jul 20, 2023

Background

IQ has been widely used in fields like education, business, and psychology to assess cognitive abilities, predict future success, and inform recruitment decisions. It is measured using validated tests like the WAIS-IV, Stanford-Binet, and Raven’s Progressive Matrices. Cognitive Components of Intelligence: The WAIS-IV identifies four key cognitive abilities (see image below)—which together provide a full-scale IQ (FSIQ) score to assess strengths and weaknesses.

Challenge

However, there are several challenges with traditional IQ Assessments:

  • Potential risk of discrimination

  • Low predictive reliability when used on minority groups,

  • Negative reactions from participants (e.g. lack of motivation, enthusiasm, and perceived bias)

Gamification of assessments, using game-based assessments (GBAs), has emerged as a promising solution to increase engagement and motivation. While GBAs are becoming popular in educational and personnel selection contexts, insufficient scientific research on their reliability and validity exists. More studies are needed to establish their psychometric soundness in assessing cognitive abilities.

An insufficient number of studies have examined the reliability and validity of GBAs. Additionally, their results suggest several aspects that should be included in research on GBAs such as the examination of their construct validity

Solution

To investigate the construct validity & internal reliability of 4 GBAs with the WAIS-IV in a sample of 34 community-recruited adults.

Key Responsibilities

Research Execution: As the UX Researcher, I planned, performed, and led a series of tests (i.e., WAIS-IV) and collected data on users' IQ, behaviour, preferences and feedback. This was done to ensure a high-quality product in terms of its reliability and validity, allowing a seamless integration of user insights into product development. I directed over 30 tests across these methodologies, tailored to support strategic business goals:

  1. Wechsler’s Adult Intelligence Scale - 4th Edition (WAIS-IV)

    Conducted 30+ Intelligence Quotient (IQ) tests to measure intelligence and cognitive ability in adults and older adolescents. This validated and commonly used assessment was used in order to correlate and analyse the scores with the games.


  2. 4 Game-Based Assessments created by Mindgame

    Executed 30+ tests with participants to measure specific cognitive abilities in adults and older adolescents. Each game lasted between 5 to 10 minutes.


  3. Qualtrics Questionnaire

    Provided a questionnaire to collect users' thoughts and experiences with the 2 assessment methods.


  4. A|B Preference Testing

    Created a preference test for each participant to explain which method of testing cognitive ability they prefer.


  5. Informed Consent Form & Debriefing Session

    Facilitated and communicated with participants before, during, and after the research sessions.


  6. Data Analysis

    With all the data collected, I performed reliability analyses & correlational analyses.

Key Learnings

  1. The Importance of High-Quality Products

Despite how attractive and ‘fun’ a product may look, I appreciated the focus on designing with a strong sense of reliability and validity, not only just the construct validity of the GBAs but also its face validity.

  1. Communication between Business & Users

It’s not often the case that the goals and hopes between business goals and users' perceptions and hopes are in line. Yet, building rapport and trust between users and those in the business and then bridging between the two parties was easily one of my favourite parts of this project.

  1. Next Steps…

This project was developed for my master’s thesis where I worked in line with business goals to ultimately propose to different psychological assessment companies about a collaboration and use of these GBAs.

Next work

Next work

Next work

Next work