Human behavior understanding using machine intelligence is a challenging task that has received much attention in the machine learning and computer vision communities. Applications can be used in a wide range of fields like video surveillance, ambient assisted living, human-computer interaction and more. One of the main goals of human behavior understanding is to learn what is considered normal in a certain context and, consequently, distinguish between “irregular” or “abnormal” behaviors, for preventing unwanted or dangerous events. For example, walking around the house is a completely normal behavior, but if it is performed during night for a long time, it can be interpreted as an irregular action that may necessitate medical or other action to be taken.
Systems performing abnormal behavior detection have to cope with different challenges: first of all, there is no clear definition of behavior abnormality, since it does not only depends on contextual cues, but also on spatial (location), temporal (time and duration), activity order and more . Secondly, physical constraints, such as occlusions and scene clutter may add noise or ‘pollute’ the data with false alarms.
ICT4Life contains an Abnormal behavior Detector (ABD) module that receives as input raw data from different sensors such as cameras and ambient sensor, carrying out data analysis to discriminate between normal and abnormal daily activities in homes of elderly suffering of Alzheimer’s, Parkinson’s or other type of dementia. ABD aims to provide a statistical analysis of the monitored environment, by extracting data from different cues and channels of information, even beyond just sensors (e.g. medical, demographical). Spatio-temporal information such as trajectories as well as motion features are extracted from video data . Trajectory analysis describes the regions which are frequently occupied, while, additional motion information extracted from these regions, contributes to obtaining high level information such as stationary behaviors (sitting, working at the desk, eating at the table) as well as active ones (walking, exiting the space). Moreover, non-obtrusive sensors such as switch sensors placed on doors and furniture are used to monitor events like the number of visits to the bathroom or leaving the house.
Examples of abnormal behavioral patterns detected by the ABD module: wandering in a confused state or repetitive behaviors, common to patients suffering of Alzheimer’s.
Due to privacy issues, there is no available public video data of patients with Alzheimer’s disease; therefore, in order to train our system, we recorded a dataset at the University of Maastricht. Specifically, we asked various healthy participants to execute tasks which might elicit confused behaviours such as “looking for an item (inexistent) in a room”. It was proved that this situation elicits a similar behaviour, as the one of a person with an early stage of Alzheimer’s, who might not remember the placement of things. The behaviour patterns obtained in this manner were spontaneous, as no indication was given about how to achieve a given task, while resembling behaviours of patients affected by Alzheimer’s disease, when being confused or agitated and anxious.
Machine learning and human behavior understanding achieved promising results in supporting the increasing trend of independent living, both for elderly and people with disabilities. However, much more effort has to be invested by researchers as well as medical communities to work together to overcome today’s systems limitations.
 Bakar, U. A. B. U. A., et al. “Activity and anomaly detection in smart home: A survey.” Next Generation Sensors and Systems. Springer International Publishing, 2016. 191-220.
 D.Dotti, M.Popa, S. Asteriadis. “Unsupervised discovery of normal and abnormal activity patterns in indoor and outdoor environments”. Proceedings of VISAPP 12th International Conference on Computer Vision Theory and Applications. 2017
Department of Data Science and Knowledge Engineering