Emotion Detection Using EEG and Voice Signals

Abstract

This paper explores the feasibility of creating a polygraph capable of discerning emotional states in individuals by analyzing skin impedance, EEG waves, and voice changes in response to "YES" and "NO" questions. Utilizing a laboratory MP36 BIOPAC Acquisition Unit with high-resolution A/D sampling, this polygraph employs three channels: Galvanic Skin Response (GSR), EEG signals, and audio input via a microphone. Emotion recognition is based on a majority voting circuit that compares the responses from all three channels. The variation in the amplitude of the skin conductivity is detected by comparing the derivative of the signal with a threshold value, and for the classification of EEG and vocal signals we use feedforward neural networks. To reduce the neurons in the input layers of the networks, the signals are processed using the Discrete Cosine Transform (DCT). Our findings reveal promising results in emotion detection via these multiple channels, offering potential applications in fields like human-computer interaction and emotional state monitoring. 

Country : Romania

1 Rustem Popa

  1. Department of Electronics and Telecommunications, “Dunarea de Jos” University of Galati, Romania

IRJIET, Volume 7, Issue 11, November 2023 pp. 22-26

doi.org/10.47001/IRJIET/2023.711004

References

  1. R. Pflanzer, and W. McMullen, “BIOPAC Student Lab, Lesson 9, Electrodermal Activity and Polygraph”, BIOPAC Systems Inc., CA, USA, 2019.
  2. R. Popa, “Emotion Detection using EEG Signals”, Int. Journal of Progressive Sciences and Technologies, vol. 24, no. 1, pp. 175-182, 2020.
  3. D. Birvinskas, V. Jusas, I. Martisius, and R. Damasevicius, “EEG Dataset Reduction and Feature Extraction using Discrete Cosine Transform”, 2012 UKSim-AMSS 6th European Modelling Symposium, pp. 199-204, DOI 10.1109/EMS.2012.88.
  4. C. Yu, M. Wang, “Survey of emotion recognition methods using EEG information”, Cognitive Robotics, vol. 2, pp. 132-146, 2022.
  5. L. Frangu and R. Popa, "Change Detection in EEG Signals," in 6th International Symposium on Electrical and Electronics Engineering (ISEEE), Galati, Romania, 2019, pp. 1-6, doi: 10.1109/ISEEE48094.2019.9136145.
  6. R. Douglas Fields, “Electric Brain. How the New Science of Brainwaves Reads Minds, Tells Us How We Learn, and Helps Us Change for the Better”, BenBella Books Inc., Dallas, TX, USA, 2020.
  7. X. Wang, Y. Ren, Z. Luo, W. He, J. Hong, and Y. Huang, “Deep learning-based EEG emotion recognition: Current trends and future perspectives”, Frontiers in Psychology,  14:1126994, 2023, doi: 10.3389/fpsyg.2023.1126994.
  8. X. Wang, “Emotion Recognition in Human Voice Speech Based on Machine Learning”, in Y. Chen et al. (Eds.): ICMETSS 2022, ASSEHR 693, pp. 149–157, 2023.
  9. R. Leelavathi, S. Aruna Deepthi, V. Aruna, “Speech Emotion Recognition using LSTM”, International Research Journal of Engineering and Technology, vol. 9, issue 1, pp. 586-594, 2022.