SMART TV Cognitive Games Analysis: Can a simple memory game assist medical experts with decisions?


One functionality developed by the ICT4Life technical team as part of the ICT4Life platform is the well-known memory game Find the Pairs, which can be accessed by SMART TV (see figure above).

Thanks to the remote control, the users reveal the tiles in order to find all the pairs. The platform supports three different levels of difficulty the Easy (with 2 pairs/4 tiles),  Medium (with 6 pairs/12 tiles) and Hard (with 10 pairs/20 tiles).

Besides the creation of the tool by the technical partners of the project and its test by the users in the pilots, ICT4Life Consortium partners are also investigating to find out if this simple memory game can provide assistance to medical experts helping them to make decisions about the cognitive status of the user.

In order to study the possible relation between the gaming performance and cognitive status, the Digital Interaction Tracking (DIT) service, which has been integrated to ICT4Life platform, is able to gather data anonymously from each completed game played by the users, in relation with the number of time the user has clicked during the duration of the game. For each game, a variety of scores taking account the speed, the accuracy and the number of errors of each user are extracted.

  • Number of trials: number of pairs revealed. This number allows to calculate the performance of the user in the game as a percentage rate.
  • Time: duration required to complete the game (in seconds). Although the time is proportional to the number of trials, this score is also affected by the movement skills of the patient. Moreover the average duration in seconds required for each tile selection is also extracted.
  • ICT4Life scoring: There are 4 different cases regarding the tile knowledge of the selected pair by the user:
    *The first AND the second selected tiles are unknown. This selection is considered accidental and does not add much value to the total score.
    *The first tile is unknown and the second is known. This adds the most value to the total score concerning the correct selection. It implies that the user opened an unknown tile and remembered where its pair was.
    *The first tile is known and the second unknown. The user re-opens a known tile even though s/he does not know where its pair is.
    *The first AND the second tiles are known. This is a critical selection and adds the most value to the total score concerning the wrong selection.

The scores for each case are illustrated in the table below:


In the next figure, the typical analysis of a hard level game (20 tile/10 pairs), a user’s game has been depicted. In this specific case, the player has made 58 clicks to complete it.


The horizontal axis shows each tile click of the user. The left vertical axis shows the remaining tiles, while the right vertical axis represent time in seconds.

The light gray line displays the remaining closed tiles, which depicts how many available tiles the user has before s/he selects a tile, and the dark gray line displays the remaining unknown tiles at the same time. The corresponding amount of tiles is shown on the left axis. The cyan line displays the time the user took until selecting the tile, and the corresponding times are shown on the right axis.

A medical expert, in a quick view, can see the process of the gameplay of the user. For this specific example, the user, in the first half of the game is trying new tiles, and the unknown tiles are reducing quickly. Moreover, on the second half of the game, the user is taking more time between tile selections, and the remaining closed tiles are reducing more quickly than on the first half. This could be a hint that the user, after has opened most of the tiles, is taking time to remember the position of the correct pairs and is more successful.

Another conclusion can be drawn when looking at the types of errors. The red ‘x’s are showing a score system made up in relation with the two choices of the user (known or unknown tiles), and its range is from -4 to 4 (negative for errors and positive for correct pair choices). The red squares show the Manhattan-distance between the user’s choice and the correct choice, in terms of tiles, for the most significant mistakes (score < -2). In the second half, these distances are smaller, implying that the user was searching in the correct area of the game to find the correct answer, a hypothesis that is consistent with the longest times between the clicks.

This game is currently tested in the University of Pecs. The pilots started on May 2018 and since then more than twenty users have tried it. At the same time, all users have been interviewed by professionals from the University of Pecs using various well-known tests, in order to define motion or cognitive abilities, along with questionnaires for collecting information about their personal status and their quality of life.



Dr. Nicholas Vretos & Dr. Vassilis Solachidis
Information Technologies Institute

CERTH – The Centre for Research & Technology, Hellas

 Follow CERTH on Twitter @CERTHellas

Use of cookies

This website uses cookies to improve your experience. By continuing to browse the site you are agreeing to our use of cookies. ACCEPT