Smart Bracelet for Emotional Enhancement in Children with Autism Spectrum Disorder ^†

Abstract

People with autism spectrum disorder (ASD) have great difficulties in social interaction and in the management of personal and other people’s emotions. This work aimed at developing an intelligent bracelet, capable of inferring the children’s emotional state, transmitting it to others, and, above all, informing the patients themselves so that they can learn to recognise, control, and work with, as well as to improve their self-knowledge and their relationship with their environment. Electrodermal activity (EDA) and photoplethysmography (PPG) are useful in combined psychophysiological and medical studies to determine the mood of patients. Due to COVID-19, no experiments with subjects could be carried out, although the modules were validated, and a public database was used to test the system’s application. The results concluded that, in general, when an individual is altered or becomes nervous, either positively or negatively (also known as valence) to a stimulus, their heart rate and sweating increase. This is the kind of relationship between physiological signals and external stimuli that the design of these circuits was intended to confirm. Finally, with the indicators of nervous system activity and knowing the behaviour of skin conductance in response to each basic emotion, it can be determined whether the subject is in a situation of pleasure or frustration in response to each reaction.

1. Introduction

Nowadays, there are some devices available in the market focused on the processing of biomedical signals in relation to the emotions of individuals, but they are neither numerous nor affordable to all. Focusing on those applied to autism, again, we find that those that exist are not accessible. The cost ranges from EUR 300 to EUR 2000. The aim of this project was to develop a bracelet able to measure electrodermal activity (EDA) and photoplethysmography (PPG) signals in autistic children but developing an accessible and affordable device for the public. This type of technology is very useful in helping the emotional control and development of children on the autistic spectrum. The processing of their heart rate (HR) and electrodermal activity helps to classify the individual into one of the basic emotions. This information allows both the subject and their tutors or relatives to assess this emotional state and to act, respond, and evolve accordingly. In other words, this device is a tool for the growth, education, and personal development of these individuals.

2. Materials and Methods

The hardware design had an EDA and a PPG module. An Arduino Nano-programmed using the library BSP [1] at 256 Hz of sampling frequency was used for data acquisition and sent them via Bluetooth (HC05). The EDA circuit was presented in [2]. The PPG circuit had a first-order [0.7–28] Hz bandpass filter and a total gain of 100.

The PCB board developed for the prototype is shown in Figure 1. It was made with surface-mounting device (SMD) components, with the intention of reducing the size of the board as much as possible.

Due to the COVID-19 pandemic, it was not possible to test the prototype on volunteers. However, the circuits used during this project (EDA and PPG) had previously been tested in [2,3]; then, we used a database [4] that collected some biological signals— EDA, PPG, EEG, EMG, BVP, breath rate, etc.—when participants were playing the Pacman game, to test the algorithms to apply. This version of the game was designed to introduce the loss of control (LOC) states and measure physiological reactions in volunteers. The controls were reduced to two buttons for the index fingers of both hands. The left one rotated the game avatar clockwise by 90 degrees, and the right one rotated it dual clockwise. The experimental design consisted of introducing the LOC into the game in two-minute blocks, interspersed with unmodified blocks of the same duration. In total, each session lasted 30 min. The blocks with LOC were distributed evenly throughout the experiment, building a series from sequences of three blocks; one LOC and two normal blocks. Modifications to induce LOC consisted of randomly ignoring 15% of the actions typed by the subject and occasionally freezing the screen to produce a delay in the image. After each block, the subjects rated their mental state in terms of valence (pleasure), arousal, and dominance (subjective feelings of control) on a scale presented with the Software Asset Management (SAM) assessment test [5].

Data Processing

The PPG signal was processed in 3 steps: firstly, we calculated the derivative of the signal, then a bandpass filter was applied, and finally, the Wavelet transform. This algorithm calculates the time difference between two consecutive RR segments and determines the individual’s cardiac variability (HRV) [6]. PPG and electrocardiography (ECG) are signals that could be considered redundant, as we were interested in analysing the heart rate and its variability. The derivative was performed to resemble the PPG signal to the ECG and process it with one of the algorithms commonly used to analyse these types of signals, such as the Wavelet. The bandpass filter has a cutoff frequency of 1 Hz and 40 Hz. It does not allow frequencies lower than 1 Hz or higher than 40 Hz.

For the processing of the EDA signal, the variables extracted with Ledalab [7], an open source EDA signal processing package for MATLAB, were used. For the Ledalab processing, software-understandable events were manually added every 10 s to evaluate the EDA signal in 10 s windows within each block. The continuous decomposition analysis (CDA) method was applied, which aimed to recover the underlying signal characteristics of the sudomotor nerve.

3. Results and Discussions

As an example of the results of the statistical processing of the PPG, some of the data obtained for the individual s4 is shown below. The discrete analysis recording of the EDA signal for subject s0 shows that during the course of the entire video game, he underwent four steps of increase in skin conductance. This was consistent over time with the blocks of frustration and with the peak values of phasic activity, as can be seen in Figure 2.

4. Conclusions

The developed device allows the PPG and EDA of individuals to be measured correctly. It has been determined that, when an individual becomes upset or nervous, either positively or negatively to a stimulus, their heart rate and sweating increase. This is the relationship between these signals and external stimuli that this project was intended to confirm. In addition, with the indicators of autonomic nervous system (ANS) activity (acquired from the analysis of cardiac variability obtained through the PPG recording) and knowing the behaviour of skin conductivity in response to each stimulus, it is possible to determine whether the subject is in a situation of pleasure or frustration in response to each observed reaction.