The ICT4Life system was deployed in users’ houses, in all end-users sites, including Madrid (Spain), Paris (France) and Pecs (Hungary). By continuous monitoring using cameras, sensors and wearables – we were ready to support the user in case of an incident, such as falling down, confusion, too high blood pressure – increasing their feeling of safety and independent living. Then, by monitoring their behaviour over a longer period, we were able to detect using machine intelligence techniques, whether there were deteriorations or improvements of their medical condition. This information is especially useful for different stakeholders: the user, the caregiver, and the professionals, but also for prevention strategies and taking the right measures in time. For meeting this goal, the ICT4Life system is sending recommendations to the users, such as information about planned meetings, the prescribed medicines, recommended exercises, helping them to increase their knowledge and autonomy or getting into contact with the right persons.
Different components of the ICT4Life platform serve different goals – sensors for monitoring the physical activity, wearables for monitoring biological signals, the Smart TV for fun and detecting their cognitive status in correlation with cognitive games, digital devices – for staying informed and connected. The set of employed sensors included binary sensors applied on doors for detecting their opening/closing, depth sensors such as Kinect used to detect user behaviour patterns and wearable devices, such as smart bracelets for monitoring biological signals (i.e. heart rate measurements, galvanic skin response). An example of a pilot deployment can be watched on youtube, showing the designated goal(s) of each sensor and the way in which the performed analysis can benefit the user and his/her caregiver, while it is also presented below.
In Figure 1, we included an example of the gathered data of a user during a month, consisting of several statistics, which can provide helpful insights about the general behaviour patterns of the user. In the top corner, a bar graph depicts the relationship between stationary behaviour (sitting) vs. daily motion (the amount of moving around the room), indicating that the user is spending most of his time sitting in an armchair in front of the Kinect camera, while watching TV or reading a book. In case this balance would change in the future, could be a sign that the user lost interest in some of his daily activities, while taking into account also the correlations with other events (i.e. he is at home and does not experience a disease). Then, the next graph in the top right corner includes the heart rate measurements, the mean, the median and the confidence interval, showing that the measurements are below the worry level of this user. Then in the left bottom corner, a bar graph depicts the number of leaving of the house events, detected using binary sensors placed on the entrance door and the line plot shows the number of steps detected using the smart bracelet for each day in a month. Interesting enough the two events are sometimes negatively correlated, as there are days in which the user receives visits of friends or relatives and he reduces the amount of physical activity. Finally, in the bottom right corner of Figure 1, the combined activation of several events is shown, including leaving of the house, visits to the bathroom and night motion events, all useful at evaluating if there are significant changes in the daily routine of the user.
Figure 1 Example of different indicators used to assess the user’s condition during a month period, such as the balance between stationary behaviour and daily motion, the heart rate measurements, the number of steps and number of events detected by binary sensors, such as leaving the house and the number of visits to the bathroom along with night motion .
Regarding the interaction of the user with the smart TV app, E-Seniors’ researchers have noticed interesting points. Overall, the patient’s performance when playing one of the cognitive games of the app strictly depends on his level of attention: when the gaming session took place in the morning or after the patient had rested, he was “fresher” and then more concentrated on the games and this was noticed especially for the memory games. In such cases, the researchers noticed that the patient completed the games in less time than during other gaming sessions when he seemed to be tired or not in the mood for playing. Other remarks can be done regarding the Bingo game. The patient takes quite a lot of time finding on his paper sheet the object that is called by the game. This doesn’t seem to be correlated with the fact that the patient is tired or not. Asking him to focus on the big object that appears on the screen after it is called out by the game can help the patient to find the position of the object in the list of objects on the screen and then on his paper sheet. Finally, in terms of acceptance, the researchers noticed that the patient enjoys playing the games on the Smart TV app, but only if they are played with others. Therefore, the social component is very strong and valuable for the patient, as it really motivates him to play.
The deployment of the ICT4Life system showed promising results, supporting the increasing trend of independent living and contributing to improving the life quality of elderly people living with physical or cognitive disabilities, as stated by the users of the system (i.e. one user said he was pleasantly surprised to see his daily activity graphs, which motivated him to keep improving and stay healthy for as long as possible).
Department of Data Science and Knowledge Engineering