REPOSITORY > RESULTS

Abstract

Commercially available smartphones, smart glasses, smartwatches, and smart rings are just
a few examples of sensor-packed devices that are enabling the technological revolution
currently underway. To further extend the successful applicability of wearable devices in
sectors such as mobile health, methods for accurate measurements of psycho-physiological
information are required. However, accessing psycho-physiological information using
wearable devices remains challenging. One reason is that the relationship between sensor
data and human psycho-physiological states is not as unambiguous as the relationship
between sensor data and individual physical states is. Thus, we are facing a question: How
can we transform wearable sensor data into valuable human health and behavior
information? Such information has the potential to improve healthcare, decrease healthcare
costs, enhance the automotive industry, enhance sports performance, improve the quality
of life and, ultimately, save human lives.
For a decade, deep learning (DL) has dominated the AI world by achieving a
breakthrough in several areas such as image processing, natural language processing, and
reinforcement learning. Thus, a successful fusion of classical machine learning (ML) and
DL methods could lead to beyond state-of-the-art (SOTA) results for mobile health and
behavior monitoring.
This thesis proposes a general method for combining expert knowledge, classical ML,
and DL to enable the transformation of wearable sensor data into valuable health and
behavior information. The method includes: (i) learning from a large-body of expert-defined
features using classical ML; (ii) a novel DL architecture, named Spectro-Temporal ResNet,
which learns directly from raw sensor data with the potential to discover useful patterns,
previously unknown to experts; and (iii) a fusion of classical ML and end-to-end DL
approaches, which utilizes meta-learning. The method is applied in seven domains of mobile
health and behavior monitoring. These include chronic heart failure detection from heart
sounds, locomotion recognition from smartphone sensors, distracted-driving detection from
physiological and video-based sensors, stress, emotion, and cognitive-load monitoring from
physiological sensors, and blood pressure estimation from ECG sensors. In all these
experimental domains, the proposed method achieved better performance than the baseline
and the SOTA methods, which were analyzed in the studies.
The combination of a variety of sensors enables better modeling of the human psychophysiological
states as different sensors provide information about different parts of the
human nervous system. For example, a sweat sensor can measure the activation of the
sympathetic nervous system, and a heart sensor can measure the activation of the
sympathetic and the parasympathetic nervous system jointly. Additionally, the
combination of a variety of sensors and models enables enhanced robustness to noise in the
data. Finally, the proposed meta-learning enables performance optimization,
personalization, and can consider temporal dependence in the data, improving the
performance of monitoring the human physical, physiological and psychological states.